The SGD update rule for AlexNet uses
ARMSprop with a decay rate of 0.99
BMomentum 0.9 plus weight decay 0.0005
CAdam optimiser with default beta values
DPlain SGD without any momentum term
Answer & Solution
Correct answer: B. Momentum 0.9 plus weight decay 0.0005
Update: v_(t+1) = 0.9·v_t − 0.0005·ε·w_t − ε·gradient. Momentum 0.9, weight decay 0.0005, learning rate ε divided by 10 when validation error plateaued.
Related questions
Image similarity in AlexNet's last hidden layer is measured byRemoving any single middle convolutional layer of AlexNet costs roughlyAlexNet was trained for roughly how many epochs on ImageNet?At test time dropout is approximated byAlexNet uses overlapping pooling withAlexNet's second form of data augmentation isAlexNet's first form of data augmentation isWhy are ReLU networks faster to train than tanh networks?