Home › Claude › aifoundations › linear_regression_and_supervised_learning › The closed-form 'normal equations' solution for …
The closed-form 'normal equations' solution for least-squares linear regression is
Aθ = (X y)⁻¹
Bθ = (XᵀX)⁻¹ Xᵀ y
Cθ = X⁻¹ y
Dθ = Xᵀ y
Answer & Solution
Correct answer: B. θ = (XᵀX)⁻¹ Xᵀ y
Setting ∇_θ J(θ) = 0 yields XᵀX θ = Xᵀy, hence θ = (XᵀX)⁻¹ Xᵀy provided XᵀX is invertible (full column rank).
Related questions
The resulting per-example log-likelihood is, up to constantsWhy is STOCHASTIC gradient descent often preferred over BATCH gradient descent on very larClosed-form normal equations require XᵀX to beLogistic regression uses which hypothesis function?The PROBABILISTIC justification for the least-squares cost assumes the noise ε⁽ⁱ⁾ in y⁽ⁱ⁾ Why is the least-squares optimisation for LINEAR regression guaranteed to converge to the The design matrix X for n training examples with d features (plus intercept) has shapeThe LMS (Widrow–Hoff) update rule on a single training example is