Home › Claude › aifoundations › many_shot_in_context_learning › An architectural risk of many-shot ICL exposed b…
An architectural risk of many-shot ICL exposed by the XSum result is
ALoss of multilingual ability in long contexts
BHallucination of facts not present in any exemplar
CCatastrophic forgetting of the pretraining data
DTokenizer mismatch between training and inference
Answer & Solution
Correct answer: B. Hallucination of facts not present in any exemplar
The XSum experiments produced summaries with fabricated dates and times despite none being in the in-context examples. Long prompts can still elicit hallucinated content; many-shot is not a hallucination-proofing trick.
Related questions
The implication for prompt design when shots are abundant isWhy is many-shot ICL practically more attractive than fine-tuning?Long-context scaling laws (next-token loss)Many-shot ICL can overcomeMany-shot ICL can perform comparably toThe order of examples in many-shot promptsAbstractive summarization on XSumMachine translation many-shot results