IntermediateTopic 57 of 104Engineering Mathematics
Least squares
Finds the best-fit solution to overdetermined systems by minimising squared error; used in regression and calibration.
In depth
When Ax=b has more equations than unknowns, least squares minimises ‖Ax-b‖² and solves the normal equations AᵀAx=Aᵀb. It fits lines or curves to noisy data and underlies regression and sensor calibration.
Key points
- Residual = measured - predicted
- QR is more stable than normal equations
- Weighted versions account for noise
ExampleFitting a straight line to 20 temperature readings uses least squares.
What is Least squares?
Finds the best-fit solution to overdetermined systems by minimising squared error; used in regression and calibration.
Why is it important to understand least squares?
Least squares finds the best fit to noisy or overdetermined data, which is how most calibration and parameter estimation is done.
Why minimise the sum of squares rather than the absolute errors?
Squares are differentiable, which gives a closed-form solution, and they penalise large errors more heavily. Absolute-error fits need other methods.
What does a residual tell me?
The difference between an observed value and the fitted value. Patterns in residuals reveal a wrong model, not just noise.
How is Least squares used in practice?
Typical applications include fitting a straight calibration line to strain gauge readings, estimating model parameters from experimental data in a control project and finding a consistent set of measurements from redundant surveys.
What should I learn before Least squares?
Study "Orthogonality and projections" first (topic 56 of 104 in Engineering Mathematics). It covers the groundwork this topic builds on.
What are the key points of Least squares?
Residual = measured - predicted. QR is more stable than normal equations. Weighted versions account for noise.
Can you give an example of Least squares?
Fitting a straight line to 20 temperature readings uses least squares.
What comes after Least squares?
The next topic is "Singular value decomposition", which builds directly on this one.
How difficult is Least squares?
It is classed as Intermediate in Engineering Mathematics (topic 57 of 104). It is an intermediate topic. Work through the beginner topics in Engineering Mathematics first.