Home › Claude › aifoundations › instructgpt_rlhf › The base pretrained model fine-tuned for Instruc…
The base pretrained model fine-tuned for InstructGPT is
AGPT-2 with 1.5 billion parameters
BGPT-3 with up to 175B parameters
CT5 with 11 billion parameters total
DLLaMA with 70 billion parameters
Answer & Solution
Correct answer: B. GPT-3 with up to 175B parameters
The starting checkpoint is GPT-3 (up to 175B). InstructGPT is fine-tuned variants of these checkpoints — not pretrained from scratch.
Related questions
The FLAN and T0 datasets are mentioned as baselines because theyWhy is honesty measured via 'truthfulness' rather than directly?Prioritising helpfulness during training while prioritising harmlessness at evaluation is RM models in InstructGPT are sized atInter-annotator agreement among training labelers is roughlyRM training uses all C(K,2) comparisons from a single prompt asThe RM training pairwise loss function uses'PPO-ptx' differs from plain PPO by