Home › Claude › aifoundations › linear_regression_and_supervised_learning › Why is the least-squares optimisation for LINEAR…
Why is the least-squares optimisation for LINEAR regression guaranteed to converge to the GLOBAL minimum under (small enough) gradient descent?
ABecause J(θ) is non-convex and randomness helps
BBecause J(θ) is a convex quadratic function — it has one global minimum and no other local optima
CBecause the data is always Gaussian
DBecause gradient descent always converges everywhere
Answer & Solution
Correct answer: B. Because J(θ) is a convex quadratic function — it has one global minimum and no other local optima
J(θ) is convex quadratic in θ (sum of squared affine functions). All local minima of a convex function are global; gradient descent with small enough α converges to it.
Related questions
The resulting per-example log-likelihood is, up to constantsWhy is STOCHASTIC gradient descent often preferred over BATCH gradient descent on very larClosed-form normal equations require XᵀX to beLogistic regression uses which hypothesis function?The PROBABILISTIC justification for the least-squares cost assumes the noise ε⁽ⁱ⁾ in y⁽ⁱ⁾ The design matrix X for n training examples with d features (plus intercept) has shapeThe closed-form 'normal equations' solution for least-squares linear regression isThe LMS (Widrow–Hoff) update rule on a single training example is