alleryStart learning →Why are ReLU networks faster to train than tanh networks?
AReLU is supported only on modern GPU hardware
BReLU does not saturate for positive inputs
CReLU has a cheaper memory access pattern
DReLU avoids the need for any learning rate
Answer & Solution
Correct answer: B. ReLU does not saturate for positive inputs
Saturating nonlinearities (tanh, sigmoid) produce near-zero gradients for large |x|, slowing learning.
ReLU's gradient is 1 for x > 0, so signal passes through cleanly.
Keep practising AlexNet CNN ImageNet
Browse all AlexNet CNN ImageNet questions for Claude, or explore more Aifoundations topics.
Related questions
Fill in the AlexNet numbersImage similarity in AlexNet's last hidden layer is measured byThe SGD update rule for AlexNet usesRemoving any single middle convolutional layer of AlexNet costs roughlyAlexNet was trained for roughly how many epochs on ImageNet?At test time dropout is approximated byAlexNet uses overlapping pooling withAlexNet's second form of data augmentation is