Home › Claude › aifoundations › linear_regression_and_supervised_learning › The resulting per-example log-likelihood is, up …
The resulting per-example log-likelihood is, up to constants
AProportional to −(y⁽ⁱ⁾ − θᵀx⁽ⁱ⁾)²
BProportional to |y⁽ⁱ⁾|
CIndependent of θ
DProportional to log(θᵀx⁽ⁱ⁾)
Answer & Solution
Correct answer: A. Proportional to −(y⁽ⁱ⁾ − θᵀx⁽ⁱ⁾)²
log p(y|x; θ, σ) under Gaussian noise = -1/(2σ²)·(y - θᵀx)² + const. Maximising the log-likelihood w.r.t. θ is equivalent to minimising the squared-error sum — derives least-squares as MLE.
Related questions
Why is STOCHASTIC gradient descent often preferred over BATCH gradient descent on very larClosed-form normal equations require XᵀX to beLogistic regression uses which hypothesis function?The PROBABILISTIC justification for the least-squares cost assumes the noise ε⁽ⁱ⁾ in y⁽ⁱ⁾ Why is the least-squares optimisation for LINEAR regression guaranteed to converge to the The design matrix X for n training examples with d features (plus intercept) has shapeThe closed-form 'normal equations' solution for least-squares linear regression isThe LMS (Widrow–Hoff) update rule on a single training example is