Who answers online surveys? Measuring non-response bias

AI SLAI HL Solid Also in: Statistics

Research question

When the same questions are sent as an optional online form and asked in person to a random sample, do the two sets of answers differ significantly, and what does that suggest about who chooses to respond?

Adapt it: change the place, the data or the comparison until the question is yours.

Free: the A–E checklist an examiner uses, by email ↓

Why it makes a good exploration

Non-response bias affects every real poll. Comparing a self-selected sample with a random one lets you measure it rather than just mention it.

The mathematics you'll need

Random versus self-selected samples
Comparing proportions and means between samples
Chi-squared or t-tests with conditions
Response rates and their effect
Weighting to correct a sample (new)

Course labels show where a technique sits; using maths from outside your course is fine if you explain it clearly and say it is new to you.

The statistics, step by step

Worked with every number shown, with what examiners look for and the common mistakes: t-test: two-sample and paired. Then run the same steps on your own data in Analyse my data, or start from the statistics workflow.

Where the data comes from

Two collection methods for the same harmless questions; anonymity, consent and school approval.

Desmos graphing calculator — Free graphing and regression (y₁ ~ ax₁ + b) — fit models to your data and show residuals.

Cite every source in a footnote where you use it and in your bibliography. Check the licence of any dataset you download.

A possible outline

Design the questions and two collection methods.
Collect both samples.
Compare answers statistically.
Try weighting the online sample.
Reflect on implications for school surveys.

Pitfalls that cost marks

Differences in wording between methods.
Small in-person sample.
Questions people feel pressured to answer in person.

Showing personal engagement

Use a real school question.
Predict which group will be more positive.
Advise the school on its next survey.

See Criterion C: personal engagement for what examiners look for.

Which course is it for?

Course	Fit	Maths to lean on
AA SL	Not a natural fit	The core technique sits in the AI course or at HL; an AA SL student could use it only as clearly explained new mathematics.
AA HL	Not a natural fit	The mathematics is mainly from the AI course; at AA HL the exploration would need an AA-level approach (calculus, proof or probability theory) to reach the top of Criterion E.
AI SL	Good fit	Random versus self-selected samples; Comparing proportions and means between samples
AI HL	Good fit	Random versus self-selected samples; Comparing proportions and means between samples

Level: Solid. Needs some independent work beyond class examples. See how the IA differs between AA and AI, SL and HL.

How this idea reaches the top bands

Personal engagement (C)

Design every part yourself: the question, the sampling frame, the wording and the analysis plan. Piloting the questionnaire and changing it is engagement an examiner can see.

Reflection (D)

Reflect on bias (who answered, who didn't, how wording steered answers) and on what a significant result can and can't show about the whole population. For this idea, start with: differences in wording between methods — say how it affects your answer.

Use of mathematics (E)

SL: A sampling method justified, a sample size worked out from the test you plan, the test's conditions checked, one calculation shown and the p-value interpreted in context.

HL: Confidence intervals, a justified choice between tests, a test of a model's fit, or a statistical estimate of how much bias could change the conclusion.

Criteria A and B (presentation and communication) work the same way for every idea: see the guides to Criterion A and Criterion B.

Taking it further

Estimate how large non-response bias would need to be to reverse a published school-survey result.

Extending it for HL

Add confidence intervals for the proportions you report, and estimate how large a non-response bias would have to be to overturn your result.

See a complete IA, marked

Our annotated exemplar Do students who sleep less react more slowly? (AI SL) asks a different question, but shows how a complete surveys exploration is structured and marked, with an examiner's comment on every criterion. Free excerpts and the full marking table are on its page.

Before you start: the checklist an examiner uses

Every check for Criteria A–E in a 4-page PDF, the mistakes that cost the most marks and a self-assessment grid. We'll email it with a short IA tip every few days, timed to your deadline if you give it. Free — no account, no payment.

Get the free checklist

Turn this idea into your IA

Plan this IAOpens the planner with this question, the maths and data sources filled in. Criteria checklistCheck your plan or draft against the A–E descriptors and self-grade before you hand it in. AI feedback on a draftCriterion-by-criterion feedback with evidence from your own text. Coaching, not rewriting. Examiner reviewPete, an IB examiner, reads your draft and talks it through with you (€100).