A COMPARATIVE ANALYSIS OF GRADIENT DESCENT AND LEAST SQUARES IN LINEAR REGRESSION

Abstract

This paper explores and compares the performance of Gradient Descent (GD) and the Least Squares (LS) method for solving linear regression problems. Theoretical aspects, computational efficiency, and accuracy are analyzed. While LS provides a closed-form solution, GD offers scalability advantages for large datasets. Empirical results demonstrate the trade-offs between these methods in terms of convergence rate, numerical stability, and computational cost.