alleryStart learning →Why does a stack of affine layers collapse without an activation function?
AStacking them runs out of memory on the GPU
BAffine of affine is affine, equivalent to one layer
CThe optimiser refuses to update multiple layers
DStacking is only allowed with skip connections
Answer & Solution
Correct answer: B. Affine of affine is affine, equivalent to one layer
If H = XW¹ + b¹ and O = HW² + b², then O = X(W¹W²) + (b¹W² + b²), a single linear layer.
Nonlinearity is what gives depth its power.
Keep practising MLP and Activations
Browse all MLP and Activations questions for Claude, or explore more Aifoundations topics.
Related questions
Flip and match each activationWhy is the sigmoid prone to the vanishing-gradient problem?For an MLP layer with input X ∈ ℝⁿˣᵈ, hidden width h and output q, the weight shapes areThe derivative of sigmoid(x) equalsThe derivative of the ReLU function isThe tanh function squashes its input to which interval?The sigmoid function squashes its input to which interval?The ReLU activation function is defined as