Home › Claude › aifoundations › many_shot_in_context_learning › The order of examples in many-shot prompts
The order of examples in many-shot prompts
AAffects performance even in the many-shot regime
BOnly matters when temperature is set above zero
CCan be safely shuffled to a fixed sequence always
DHas no effect once the count is large enough
Answer & Solution
Correct answer: A. Affects performance even in the many-shot regime
Section 4.7: ordering still affects many-shot performance. The conventional wisdom that order ceases to matter at scale is shown to be incorrect on some tasks.
Related questions
The implication for prompt design when shots are abundant isAn architectural risk of many-shot ICL exposed by the XSum result isWhy is many-shot ICL practically more attractive than fine-tuning?Long-context scaling laws (next-token loss)Many-shot ICL can overcomeMany-shot ICL can perform comparably toAbstractive summarization on XSumMachine translation many-shot results