Does the wording change the answer? A split-sample survey experiment

AI SLAI HLAA SL Solid Also in: Statistics, Probability

Research question

If half your sample is randomly given one wording of a question (or a high 'anchor' number before an estimate) and half another, is there a statistically significant difference in their answers?

Adapt it: change the place, the data or the comparison until the question is yours.

Free: the A–E checklist an examiner uses, by email ↓

Why it makes a good exploration

It turns a survey into a controlled experiment. Random allocation lets you draw a cause-and-effect conclusion that ordinary surveys can't.

The mathematics you'll need

Random allocation and why it matters
Comparing two groups: means or proportions
t-test or chi-squared test, with conditions checked
Sample size needed to detect a chosen effect
HL: confidence interval for the difference

Course labels show where a technique sits; using maths from outside your course is fine if you explain it clearly and say it is new to you.

The statistics, step by step

Worked with every number shown, with what examiners look for and the common mistakes: t-test: two-sample and paired · Confidence intervals for a mean. Then run the same steps on your own data in Analyse my data, or start from the statistics workflow.

Where the data comes from

Two versions of an anonymous form allocated at random (e.g. by coin toss). Choose harmless questions (estimates of a quantity, opinions on a school matter).

Desmos graphing calculator — Free graphing and regression (y₁ ~ ax₁ + b) — fit models to your data and show residuals.

Cite every source in a footnote where you use it and in your bibliography. Check the licence of any dataset you download.

A possible outline

Choose the effect to test and state the hypothesis.
Work out the sample size.
Randomise and collect.
Run the test and check conditions.
Reflect on what the effect means for other surveys.

Pitfalls that cost marks

Non-random allocation (e.g. one wording per class).
A difference found but the test's conditions not checked.
Sensitive questions.

Showing personal engagement

Choose wordings you think will matter.
Predict the size of the effect.
Present the result to whoever runs school surveys.

See Criterion C: personal engagement for what examiners look for.

Which course is it for?

Course	Fit	Maths to lean on
AA SL	Good fit	Random allocation and why it matters; Comparing two groups: means or proportions
AA HL	Not a natural fit	The mathematics is mainly from the AI course; at AA HL the exploration would need an AA-level approach (calculus, proof or probability theory) to reach the top of Criterion E.
AI SL	Good fit	Random allocation and why it matters; Comparing two groups: means or proportions
AI HL	Good fit	Random allocation and why it matters; Comparing two groups: means or proportions

Level: Solid. Needs some independent work beyond class examples. See how the IA differs between AA and AI, SL and HL.

How this idea reaches the top bands

Personal engagement (C)

Design every part yourself: the question, the sampling frame, the wording and the analysis plan. Piloting the questionnaire and changing it is engagement an examiner can see.

Reflection (D)

Reflect on bias (who answered, who didn't, how wording steered answers) and on what a significant result can and can't show about the whole population. For this idea, start with: non-random allocation (e.g. one wording per class) — say how it affects your answer.

Use of mathematics (E)

SL: A sampling method justified, a sample size worked out from the test you plan, the test's conditions checked, one calculation shown and the p-value interpreted in context.

HL: Confidence intervals, a justified choice between tests, a test of a model's fit, or a statistical estimate of how much bias could change the conclusion.

Criteria A and B (presentation and communication) work the same way for every idea: see the guides to Criterion A and Criterion B.

Taking it further

Test whether the effect differs between groups (a two-way table).

Extending it for HL

This idea already has HL mathematics in it: confidence interval for the difference. Add confidence intervals for the proportions you report, and estimate how large a non-response bias would have to be to overturn your result.

See a complete IA, marked

Our annotated exemplar Do students who sleep less react more slowly? (AI SL) asks a different question, but shows how a complete surveys exploration is structured and marked, with an examiner's comment on every criterion. Free excerpts and the full marking table are on its page.

Before you start: the checklist an examiner uses

Every check for Criteria A–E in a 4-page PDF, the mistakes that cost the most marks and a self-assessment grid. We'll email it with a short IA tip every few days, timed to your deadline if you give it. Free — no account, no payment.

Get the free checklist

Turn this idea into your IA

Plan this IAOpens the planner with this question, the maths and data sources filled in. Criteria checklistCheck your plan or draft against the A–E descriptors and self-grade before you hand it in. AI feedback on a draftCriterion-by-criterion feedback with evidence from your own text. Coaching, not rewriting. Examiner reviewPete, an IB examiner, reads your draft and talks it through with you (€100).