Home › Claude › aifoundations › linear_regression_and_supervised_learning › The LMS (Widrow–Hoff) update rule on a single tr…
The LMS (Widrow–Hoff) update rule on a single training example is
Aθⱼ := θⱼ · y⁽ⁱ⁾
Bθⱼ := θⱼ + α · (y⁽ⁱ⁾ − h_θ(x⁽ⁱ⁾)) · xⱼ⁽ⁱ⁾
Cθⱼ := θⱼ − α · y⁽ⁱ⁾
Dθⱼ := xⱼ⁽ⁱ⁾ − y⁽ⁱ⁾
Answer & Solution
Correct answer: B. θⱼ := θⱼ + α · (y⁽ⁱ⁾ − h_θ(x⁽ⁱ⁾)) · xⱼ⁽ⁱ⁾
LMS / Widrow–Hoff: θⱼ := θⱼ + α(y − h(x))xⱼ. Update is proportional to the ERROR — large error ⇒ big update; near-zero error ⇒ no update. This is the gradient of the squared-error loss.
Related questions
The resulting per-example log-likelihood is, up to constantsWhy is STOCHASTIC gradient descent often preferred over BATCH gradient descent on very larClosed-form normal equations require XᵀX to beLogistic regression uses which hypothesis function?The PROBABILISTIC justification for the least-squares cost assumes the noise ε⁽ⁱ⁾ in y⁽ⁱ⁾ Why is the least-squares optimisation for LINEAR regression guaranteed to converge to the The design matrix X for n training examples with d features (plus intercept) has shapeThe closed-form 'normal equations' solution for least-squares linear regression is