Start learning →

Claude Many Shot in Context Learning — practice questions

Explore 14 free Many Shot in Context Learning questions in Claude Aifoundations. Open each question or exercise to read its content, reference answers and available explanation.

Practise Many Shot in Context Learning with Quill →

Many Shot in Context Learning questions and exercises with answers

Showing questions 1–14 of 14.

The central finding is thatRead the answer and worked solution →The model used for many-shot experiments isRead the answer and worked solution →'Reinforced ICL' refers toRead the answer and worked solution →'Unsupervised ICL' refers toRead the answer and worked solution →KV caching is used in many-shot experiments toRead the answer and worked solution →The maximum number of shots evaluated is approximatelyRead the answer and worked solution →Machine translation many-shot resultsRead the answer and worked solution →Abstractive summarization on XSumRead the answer and worked solution →The order of examples in many-shot promptsRead the answer and worked solution →Many-shot ICL can perform comparably toRead the answer and worked solution →Many-shot ICL can overcomeRead the answer and worked solution →Long-context scaling laws (next-token loss)Read the answer and worked solution →The implication for prompt design when shots are abundant isRead the answer and worked solution →Rule on these many shot claimsRead the answer and worked solution →

Worked practice questions

The central finding is that

  1. Few-shot ICL beats many-shot whenever shots exceed 16
  2. Many-shot only helps when full fine-tuning is also done
  3. Many-shot performance is flat regardless of shot count
  4. Many-shot ICL beats few-shot on a wide range of tasks
Show answer and solution

Correct answer: D. Many-shot ICL beats few-shot on a wide range of tasks

Many-shot (hundreds to thousands of shots) consistently outperforms few-shot across translation, summarization, planning, reasoning, and reward modelling. The gap is largest on non-natural-language tasks.

Open this question →

The model used for many-shot experiments is

  1. GPT-3 with the 2k token context window
  2. Claude 2 with the 100k token context window
  3. Gemini 1.5 Pro with 1M token context
  4. Llama 2 with the 4k token context window
Show answer and solution

Correct answer: C. Gemini 1.5 Pro with 1M token context

The 1M-token context of Gemini 1.5 Pro enables many-shot regimes with up to thousands of examples. Earlier models like GPT-3 (2k) and Llama 2 (4k) cannot fit the long prompts.

Open this question →

'Reinforced ICL' refers to

  1. Reinforcement learning fine-tuning on top of ICL prompts
  2. A retrieval-augmented system over expert-written rationales
  3. Using model-generated rationales filtered by correctness
  4. Adding a reward model during inference at every step
Show answer and solution

Correct answer: C. Using model-generated rationales filtered by correctness

Reinforced ICL replaces human-written rationales with model-generated ones, filtered by answer correctness. Reduces dependence on costly human data.

Open this question →

How to practise

  1. Choose a Many Shot in Context Learning question and try to work out your answer.
  2. Open the question to check the correct answer and read its worked solution.
  3. Continue in the app for bite-sized lessons, explanations and review with Quill. You can start without creating an account.

More Aifoundations topics

AlexNet CNN ImageNetLinear Regression and Supervised LearningMath for AI LA ProbabilityInstructGPT RLHFMLP and ActivationsTransformer AttentionWhat Is AI Intro