Statistics · Maths EE idea · Accessible
Where does the least-squares line come from?
A research question to start from
How can the least-squares regression line be derived by calculus and by geometry, and how sensitive is it to outliers compared with alternatives?
A starting point, not your question: change the case, the comparison or the limit until it is yours. The research-question builder helps you check it.
Why it works as a maths EE
A method studied from the inside, with an evaluation of its weakness.
Mathematics you would need
- Partial derivatives or completing the square
- Sums and means
- Projections (vector view)
- Robust regression ideas
Much of this goes beyond the DP course. That is expected in a maths EE, but you must understand and explain everything you use.
One possible line of attack
- Derive the line by minimising squared residuals.
- Show the vector projection view.
- Compare with a median-based line on data with outliers.
Scope and difficulty
Accessible. Accessible.
Pitfalls
- Running regressions on data with no method focus.
- Calculus steps skipped.
Where to start reading
Search a library catalogue or a university's open lecture notes for: least squares derivation partial derivatives; Theil-Sen estimator outliers. Prefer textbooks, lecture notes and journal articles to a single website, and cite everything you use (how to reference a maths EE).