Home › Claude › aifoundations › linear_regression_and_supervised_learning › The PROBABILISTIC justification for the least-sq…
The PROBABILISTIC justification for the least-squares cost assumes the noise ε⁽ⁱ⁾ in y⁽ⁱ⁾ = θᵀx⁽ⁱ⁾ + ε⁽ⁱ⁾ is
ABernoulli(p)
BUniform on [0, 1]
CIid Gaussian N(0, σ²)
DExponential with rate 1
Answer & Solution
Correct answer: C. Iid Gaussian N(0, σ²)
Under iid Gaussian noise N(0, σ²), maximising the likelihood is equivalent to minimising the sum of squared errors — that's why least-squares is 'the' regression loss.
Related questions
The resulting per-example log-likelihood is, up to constantsWhy is STOCHASTIC gradient descent often preferred over BATCH gradient descent on very larClosed-form normal equations require XᵀX to beLogistic regression uses which hypothesis function?Why is the least-squares optimisation for LINEAR regression guaranteed to converge to the The design matrix X for n training examples with d features (plus intercept) has shapeThe closed-form 'normal equations' solution for least-squares linear regression isThe LMS (Widrow–Hoff) update rule on a single training example is