What does an MLP add to a linear model?
AOne or more hidden layers between input and output
BA larger batch size only at training time
CA higher learning rate at every training step
DA different optimiser without architectural change
Answer & Solution
Correct answer: A. One or more hidden layers between input and output
An MLP stacks fully-connected hidden layers between input and output. With nonlinear activations, it can represent much richer mappings than a single linear layer.
Related questions
Why are sigmoids still used in recurrent networks (e.g., LSTMs) even after ReLU's rise?A 'thresholding' activation taking 0 or 1 cleanly is impractical for gradient-based learniDepth often beats width for the same expressive power becauseWhy is the sigmoid prone to the vanishing-gradient problem?For an MLP layer with input X ∈ ℝⁿˣᵈ, hidden width h and output q, the weight shapes areParametrised ReLU (pReLU) adds which capability over plain ReLU?Why has ReLU largely replaced sigmoid in hidden layers?The universal approximation theorem (Cybenko 1989) for MLPs states that