Home › Claude › aifoundations › transformer_attention › Why does the decoder need to mask future positio…
Why does the decoder need to mask future positions in self-attention?
ATo match the encoder's bidirectional attention pattern
BTo prevent gradient explosion on long token sequences
CTo preserve the auto-regressive property at inference
DTo reduce computational cost during training time
Answer & Solution
Correct answer: C. To preserve the auto-regressive property at inference
At inference, the decoder generates one token at a time. If position i could attend to positions j > i during training, the train/test distributions diverge. Masking out illegal connections (set to −∞ before softmax) preserves the auto-regressive property.
Related questions
The architectural property that lets self-attention parallelise where RNNs cannot isResidual connections and layer normalisation are applied asMulti-head attention is preferred over a single d_model-dimensional head becauseThe THREE distinct ways attention is used in the Transformer areThe chosen positional encoding scheme usesPosition-wise feed-forward networks in the Transformer are essentiallyWhat is the maximum path length in self-attention layers?The per-layer complexity of self-attention is