alleryStart learning →Abstractive summarization on XSum
ACannot be evaluated with many-shot ICL approach
BContinues improving monotonically with more shots
CPeaks around 50 shots then performance deteriorates
DImproves only when fine-tuning is also applied
Answer & Solution
Correct answer: C. Peaks around 50 shots then performance deteriorates
XSum performance peaks around 50 shots and declines beyond.
The model also occasionally hallucinates dates and times, which were absent from the in-context examples.
Many-shot is not always monotonic.
Keep practising Many Shot in Context Learning
Browse all Many Shot in Context Learning questions for Claude, or explore more Aifoundations topics.
Related questions
Rule on these many shot claimsThe implication for prompt design when shots are abundant isLong-context scaling laws (next-token loss)Many-shot ICL can overcomeMany-shot ICL can perform comparably toThe order of examples in many-shot promptsMachine translation many-shot resultsThe maximum number of shots evaluated is approximately