Linear regression and residual analysis (IB Math AI HL)

Key ideas

• Residual = observed y − predicted y. Points above the line have positive residuals and points below have negative ones.
• A good linear model has residuals scattered randomly about 0, with no curve and no spreading out.
• A U-shape or an arch in the residual plot means the relationship is curved: try a quadratic, exponential or power model.
• r² close to 1 means the model explains most of the variation in y, not that the relationship is linear or causal.
• Use the regression of y on x to predict y from x. Predictions outside the range of the data (extrapolation) are unreliable.

Worked example

For x = 1, 2, 3, 4, 5, 6 the values of y are 2.0, 3.1, 4.9, 7.4, 10.2, 13.9. Find the regression line of y on x and r², calculate the residuals, and comment on whether a linear model is suitable.

1. GDC linear regression: y = 2.38x − 1.41, with r ≈ 0.983, so r² ≈ 0.966.
2. Predicted values: 0.97, 3.35, 5.73, 8.11, 10.49, 12.87.
3. Residuals (observed − predicted): 1.03, −0.25, −0.83, −0.71, −0.29, 1.03.
4. The residuals are positive at both ends and negative in the middle: a U-shaped pattern, not random scatter.
5. So although r² is high, the data curve upwards and a linear model is not the best choice. A quadratic model fits much more closely (R² ≈ 0.9998).

Answer: y = 2.38x − 1.41, r² ≈ 0.966; the residuals form a U-shape, so a curved (for example quadratic) model is more suitable.

Exam-style HL AI questions on this topic

• Data Collection and Non-Linear Regression questions — 23 questions with mark schemes, plus free worked examples
• Linear Combinations of Random Variables & the Central Limit Theorem questions — 22 questions with mark schemes, plus free worked examples
• All HL AI Statistics and Probability questions by topic

Do it on your GDC

A GDC is allowed on every AI paper. Keystrokes for the TI-84 Plus CE, TI-Nspire CX II and Casio fx-CG50, with a checked worked example:

• Regression on your calculator
• Correlation r and Spearman's rank on your calculator
• Try it: correlation and regression calculator (free, shows the working)

See it on a graph

• Regression with r, R² and residuals
• Residual plots
• Open the free Grapher

Maths IA ideas that use linear regression and residual analysis

• Do students who sleep less react more slowly?
• Is deforestation in [a country] slowing down?
• Do more doctors mean fewer infant deaths?
• Are popular songs getting faster or shorter? Statistics on music data
• IA statistics, step by step: pearson's correlation and regression
• All HL AI IA ideas

Related in Statistics and Probability

• Normal and Poisson distributions
• Hypothesis testing — t-test and chi-squared
• Markov chains and transition matrices

Revise Linear regression and residual analysis for the exam

• HL AI revision notes — every unit summarised, with the key results to learn
• HL AI past papers — specimen papers and timed practice papers
• Formulas for Statistics and Probability — on the one-page HL AI formula sheet (free PDF), with the ones not in the IB booklet marked
• HL AI formula booklet by topic — every formula in the IB booklet's notation, with a free PDF
• IB Maths prediction papers — full papers in the IB format, with markschemes

TOK link: Does big data make mathematical theory less necessary?

Stretch yourself: original probability problems at Senior level (ages 16 to 18), with hints and full solutions: Problem S107 · Problem S101 · Problem S29 · Problem S102. All 20 →