Home › Claude › aifoundations › instructgpt_rlhf › Why is honesty measured via 'truthfulness' rathe…
Why is honesty measured via 'truthfulness' rather than directly?
ATruthful answers are easier for labelers to score
BThe model's beliefs cannot be observed directly
CHonesty is identical to harmlessness in practice
DHonesty cannot be measured on any NLP dataset
Answer & Solution
Correct answer: B. The model's beliefs cannot be observed directly
Honesty would require comparing the model's output to its internal beliefs, but the model is a black box. The paper proxies honesty via truthfulness — whether stated facts are true — measured on TruthfulQA and hallucination-rate tasks.
Related questions
The FLAN and T0 datasets are mentioned as baselines because theyPrioritising helpfulness during training while prioritising harmlessness at evaluation is RM models in InstructGPT are sized atInter-annotator agreement among training labelers is roughlyRM training uses all C(K,2) comparisons from a single prompt asThe RM training pairwise loss function uses'PPO-ptx' differs from plain PPO byThe PPO objective includes a KL penalty against the SFT model to