Home › Claude › aifoundations › transformer_attention › The Transformer architecture replaces recurrence…
The Transformer architecture replaces recurrence and convolutions with
AStacked self-attention and feed-forward layers
BA single dense layer per token in the sequence
CHand-crafted phrase-structure grammar rules
DA recurrent neural network at every layer
Answer & Solution
Correct answer: A. Stacked self-attention and feed-forward layers
The Transformer is built entirely from stacked self-attention sub-layers and position-wise feed-forward sub-layers, plus residual connections and layer normalisation. No recurrence, no convolution.
Related questions
The architectural property that lets self-attention parallelise where RNNs cannot isResidual connections and layer normalisation are applied asWhy does the decoder need to mask future positions in self-attention?Multi-head attention is preferred over a single d_model-dimensional head becauseThe THREE distinct ways attention is used in the Transformer areThe chosen positional encoding scheme usesPosition-wise feed-forward networks in the Transformer are essentiallyWhat is the maximum path length in self-attention layers?