KV Cache int8 cooperate with Flash-attention2 #689

Unanswered

yhyu13 asked this question in Q&A

yhyu13
Nov 15, 2023

HI,

Just wonder how well your implementation of KV Cache int8 PTQ works with flash-attentions's KV Cache partititioning strategy?

Do they all work together? Is there any statictics done on putting all optmizations togethers?

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment