Home › Claude › aifoundations › instructgpt_rlhf › The alignment goal is summarised as the model be…
The alignment goal is summarised as the model being
AFast, cheap and small
BOpen, transparent and audited
CMultilingual, multimodal and grounded
DHelpful, honest and harmless
Answer & Solution
Correct answer: D. Helpful, honest and harmless
Following the framework of Askell et al. (2021), InstructGPT aligns toward the HHH triad: helpful, honest, harmless. These shape both training data and evaluation criteria.
Related questions
The FLAN and T0 datasets are mentioned as baselines because theyWhy is honesty measured via 'truthfulness' rather than directly?Prioritising helpfulness during training while prioritising harmlessness at evaluation is RM models in InstructGPT are sized atInter-annotator agreement among training labelers is roughlyRM training uses all C(K,2) comparisons from a single prompt asThe RM training pairwise loss function uses'PPO-ptx' differs from plain PPO by