alleryStart learning →The decoder differs from the encoder by adding
AJust one more identical encoder-style attention layer
BConvolutional layers between self-attention blocks
CRecurrent units for autoregressive generation steps
DA masked self-attention plus encoder-decoder attention
Answer & Solution
Correct answer: D. A masked self-attention plus encoder-decoder attention
The decoder adds masked self-attention (so position i cannot attend to j > i) and a third sub-layer that does cross-attention over the encoder's output stack.
Keep practising Transformer Attention
Browse all Transformer Attention questions for Claude, or explore more Aifoundations topics.
Related questions
Rebuild the model width from headsThe chosen positional encoding scheme usesPosition-wise feed-forward networks in the Transformer are essentiallyThe per-layer complexity of self-attention isMulti-head attention uses how many heads in the base model?The encoder of the original Transformer uses how many identical layers?Scaled dot-product attention, per the paper, computes the output asThe Transformer architecture replaces recurrence and convolutions with