Start learning →

Claude Transformer Attention — practice questions

Explore 9 free Transformer Attention questions in Claude Aifoundations. Open each question or exercise to read its content, reference answers and available explanation.

Practise Transformer Attention with Quill →

Transformer Attention questions and exercises with answers

Showing questions 1–9 of 9.

The Transformer architecture replaces recurrence and convolutions withRead the answer and worked solution →Scaled dot-product attention, per the paper, computes the output asRead the answer and worked solution →The encoder of the original Transformer uses how many identical layers?Read the answer and worked solution →Multi-head attention uses how many heads in the base model?Read the answer and worked solution →The decoder differs from the encoder by addingRead the answer and worked solution →The per-layer complexity of self-attention isRead the answer and worked solution →Position-wise feed-forward networks in the Transformer are essentiallyRead the answer and worked solution →The chosen positional encoding scheme usesRead the answer and worked solution →Rebuild the model width from headsRead the answer and worked solution →

Worked practice questions

The Transformer architecture replaces recurrence and convolutions with

  1. Stacked self-attention and feed-forward layers
  2. A single dense layer per token in the sequence
  3. Hand-crafted phrase-structure grammar rules
  4. A recurrent neural network at every layer
Show answer and solution

Correct answer: A. Stacked self-attention and feed-forward layers

The Transformer is built entirely from stacked self-attention sub-layers and position-wise feed-forward sub-layers, plus residual connections and layer normalisation. No recurrence, no convolution.

Open this question →

Scaled dot-product attention, per the paper, computes the output as

  1. softmax(QKᵀ / √d_k) · V over the value tokens
  2. A simple element-wise product of Q and V tensors
  3. An average of all value vectors with equal weights
  4. A direct lookup in the embedding table by token id
Show answer and solution

Correct answer: A. softmax(QKᵀ / √d_k) · V over the value tokens

Attention(Q, K, V) = softmax(QKᵀ/√d_k) · V. The √d_k denominator scales dot products to keep softmax in its sensitive region.

Open this question →

The encoder of the original Transformer uses how many identical layers?

  1. Two stacked layers
  2. Six stacked layers
  3. Twelve stacked layers
  4. Twenty-four stacked layers
Show answer and solution

Correct answer: B. Six stacked layers

Both encoder and decoder use N = 6 identical layers in the original paper. Each layer has multi-head self-attention plus a position-wise feed-forward sub-layer, with residual + layer norm.

Open this question →

How to practise

  1. Choose a Transformer Attention question and try to work out your answer.
  2. Open the question to check the correct answer and read its worked solution.
  3. Continue in the app for bite-sized lessons, explanations and review with Quill. You can start without creating an account.

More Aifoundations topics

AlexNet CNN ImageNetMany Shot in Context LearningLinear Regression and Supervised LearningMath for AI LA ProbabilityInstructGPT RLHFMLP and ActivationsWhat Is AI Intro