Home › Claude › aifoundations › mlp_and_activations › The universal approximation theorem (Cybenko 198…
The universal approximation theorem (Cybenko 1989) for MLPs states that
AOnly deep networks can learn any function at all
BUniversal approximation requires kernel methods
CMLPs cannot approximate continuous functions
DA single hidden layer can approximate any function
Answer & Solution
Correct answer: D. A single hidden layer can approximate any function
With enough hidden units, a single-hidden-layer MLP can approximate any continuous function. The catch d2l.ai stresses: finding the right weights is the hard part.
Related questions
Why are sigmoids still used in recurrent networks (e.g., LSTMs) even after ReLU's rise?A 'thresholding' activation taking 0 or 1 cleanly is impractical for gradient-based learniDepth often beats width for the same expressive power becauseWhy is the sigmoid prone to the vanishing-gradient problem?For an MLP layer with input X ∈ ℝⁿˣᵈ, hidden width h and output q, the weight shapes areParametrised ReLU (pReLU) adds which capability over plain ReLU?Why has ReLU largely replaced sigmoid in hidden layers?The derivative of sigmoid(x) equals