Home › Claude › aifoundations › linear_regression_and_supervised_learning › Closed-form normal equations require XᵀX to be
Closed-form normal equations require XᵀX to be
AHave negative determinant
BSingular
CInvertible (i.e., X has full column rank — no perfectly collinear features)
DDiagonal
Answer & Solution
Correct answer: C. Invertible (i.e., X has full column rank — no perfectly collinear features)
If XᵀX is singular (features collinear or n < d), the inverse doesn't exist. Practical fix: ridge regression θ = (XᵀX + λI)⁻¹ Xᵀy, which is always invertible for λ > 0.
Related questions
The resulting per-example log-likelihood is, up to constantsWhy is STOCHASTIC gradient descent often preferred over BATCH gradient descent on very larLogistic regression uses which hypothesis function?The PROBABILISTIC justification for the least-squares cost assumes the noise ε⁽ⁱ⁾ in y⁽ⁱ⁾ Why is the least-squares optimisation for LINEAR regression guaranteed to converge to the The design matrix X for n training examples with d features (plus intercept) has shapeThe closed-form 'normal equations' solution for least-squares linear regression isThe LMS (Widrow–Hoff) update rule on a single training example is