Home › Claude › aifoundations › mlp_and_activations › Parametrised ReLU (pReLU) adds which capability …
Parametrised ReLU (pReLU) adds which capability over plain ReLU?
AAn exponential decay in the positive input region
BA larger output range that exceeds one in scale
CA built-in mechanism for batch normalisation
DA small linear term in the negative input region
Answer & Solution
Correct answer: D. A small linear term in the negative input region
pReLU(x) = max(0, x) + α·min(0, x). The α term lets a fraction of the negative signal through, preventing 'dead neuron' problems while keeping the simplicity.
Related questions
Why are sigmoids still used in recurrent networks (e.g., LSTMs) even after ReLU's rise?A 'thresholding' activation taking 0 or 1 cleanly is impractical for gradient-based learniDepth often beats width for the same expressive power becauseWhy is the sigmoid prone to the vanishing-gradient problem?For an MLP layer with input X ∈ ℝⁿˣᵈ, hidden width h and output q, the weight shapes areWhy has ReLU largely replaced sigmoid in hidden layers?The universal approximation theorem (Cybenko 1989) for MLPs states thatThe derivative of sigmoid(x) equals