Home › Claude › aifoundations › mlp_and_activations › For an MLP layer with input X ∈ ℝⁿˣᵈ, hidden wid…
For an MLP layer with input X ∈ ℝⁿˣᵈ, hidden width h and output q, the weight shapes are
AW¹ ∈ ℝᵈˣⁿ and W² ∈ ℝⁿˣᵠ
BW¹ ∈ ℝᵈˣʰ and W² ∈ ℝʰˣᵠ
CW¹ ∈ ℝʰˣᵈ and W² ∈ ℝᵠˣʰ
DW¹ ∈ ℝⁿˣʰ and W² ∈ ℝʰˣⁿ
Answer & Solution
Correct answer: B. W¹ ∈ ℝᵈˣʰ and W² ∈ ℝʰˣᵠ
Hidden layer maps d inputs to h units, so W¹ ∈ ℝᵈˣʰ. Output layer maps h hidden units to q outputs, so W² ∈ ℝʰˣᵠ. Standard MLP forward-pass shapes.
Related questions
Why are sigmoids still used in recurrent networks (e.g., LSTMs) even after ReLU's rise?A 'thresholding' activation taking 0 or 1 cleanly is impractical for gradient-based learniDepth often beats width for the same expressive power becauseWhy is the sigmoid prone to the vanishing-gradient problem?Parametrised ReLU (pReLU) adds which capability over plain ReLU?Why has ReLU largely replaced sigmoid in hidden layers?The universal approximation theorem (Cybenko 1989) for MLPs states thatThe derivative of sigmoid(x) equals