alleryStart learning →The Transformer architecture replaces recurrence and convolutions with
AStacked self-attention and feed-forward layers
BA single dense layer per token in the sequence
CHand-crafted phrase-structure grammar rules
DA recurrent neural network at every layer
Answer & Solution
Correct answer: A. Stacked self-attention and feed-forward layers
The Transformer is built entirely from stacked self-attention sub-layers and position-wise feed-forward sub-layers, plus residual connections and layer normalisation.
No recurrence, no convolution.
Keep practising Transformer Attention
Browse all Transformer Attention questions for Claude, or explore more Aifoundations topics.
Related questions
Rebuild the model width from headsThe chosen positional encoding scheme usesPosition-wise feed-forward networks in the Transformer are essentiallyThe per-layer complexity of self-attention isThe decoder differs from the encoder by addingMulti-head attention uses how many heads in the base model?The encoder of the original Transformer uses how many identical layers?Scaled dot-product attention, per the paper, computes the output as