alleryStart learning →KV caching is used in many-shot experiments to
AConvert exemplars into a vector database
BReduce the inference cost per prompt
CIncrease the maximum context window size
DAvoid retraining the model between rounds
Answer & Solution
Correct answer: B. Reduce the inference cost per prompt
KV caching reuses key/value tensors from prior tokens so a long shared prefix (the many-shot exemplars) is computed once.
This makes the inference cost of many-shot regimes manageable.
Keep practising Many Shot in Context Learning
Browse all Many Shot in Context Learning questions for Claude, or explore more Aifoundations topics.
Related questions
Rule on these many shot claimsThe implication for prompt design when shots are abundant isLong-context scaling laws (next-token loss)Many-shot ICL can overcomeMany-shot ICL can perform comparably toThe order of examples in many-shot promptsAbstractive summarization on XSumMachine translation many-shot results