Home › Claude › aifoundations › transformer_attention › The encoder of the original Transformer uses how…
The encoder of the original Transformer uses how many identical layers?
ATwo stacked layers
BSix stacked layers
CTwelve stacked layers
DTwenty-four stacked layers
Answer & Solution
Correct answer: B. Six stacked layers
Both encoder and decoder use N = 6 identical layers in the original paper. Each layer has multi-head self-attention plus a position-wise feed-forward sub-layer, with residual + layer norm.
Related questions
The architectural property that lets self-attention parallelise where RNNs cannot isResidual connections and layer normalisation are applied asWhy does the decoder need to mask future positions in self-attention?Multi-head attention is preferred over a single d_model-dimensional head becauseThe THREE distinct ways attention is used in the Transformer areThe chosen positional encoding scheme usesPosition-wise feed-forward networks in the Transformer are essentiallyWhat is the maximum path length in self-attention layers?