SAT Math: Evaluating statistical claims

Evaluating statistical claims: observational studies and experiments.

Problem-Solving and Data Analysis · Evaluating statistical claims: observational studies and experiments
Both modules
8 practice questions · 2 with free solutions
Calculator allowed (Desmos)

Practise evaluating statistical claims →Adaptive SAT module

What the test covers

Random assignment supports cause; random sampling supports generalizing

Key ideas

Random assignment to treatments lets you conclude cause and effect.
Random sampling from a population lets you generalize to that population.
Neither: only an association among the participants.

Common mistakes

Volunteers are not a random sample, even if they are randomly assigned.
Correlation alone never shows causation.

Do it in Desmos

Ask two questions: was there random assignment, and was there random sampling?

Worked example

A researcher asked for volunteers at a gym. The 120 volunteers were randomly assigned to follow either a new stretching routine or their usual routine for 8 weeks, and the new-routine group showed a greater gain in flexibility. Which conclusion is best supported?

The new routine causes greater flexibility gains for all gym members.
The new routine causes greater flexibility gains for people like these volunteers, but the result cannot be generalized to all gym members.
No conclusion can be drawn because the sample was smaller than 200.
The new routine is associated with flexibility gains, but no cause-and-effect conclusion can be drawn.

Answer: B: The new routine causes greater flexibility gains for people like these volunteers, but the result cannot be generalized to all gym members.

Random assignment to the two routines supports a cause-and-effect conclusion. The participants were volunteers, not a random sample of gym members, so the result applies only to people like the volunteers.

Practice questions

Try each question before opening the solution. Desmos or your own calculator is allowed on every SAT Math question. Solutions to the first 1 are open to everyone; the rest need only a free account. Every answer here was checked independently before it was published.

Question 1

A researcher studied a random sample from students at a college. The participants were randomly assigned to use a new study app or not, and the group using a new study app showed a greater improvement in test scores. Which conclusion is best supported?

No conclusion about test scores can be drawn from this study.
Using a new study app is associated with greater improvement in test scores among students at the college, but cause and effect cannot be concluded.
Using a new study app causes greater improvement in test scores for all students at the college.
Using a new study app causes greater improvement in test scores for people like the participants, but the result cannot be generalized to all students at the college.

Show the answer and solution

Answer: C: Using a new study app causes greater improvement in test scores for all students at the college.

Random assignment supports cause and effect; random sampling supports generalizing to the population. This study used random assignment and used a random sample, so: Using a new study app causes greater improvement in test scores for all students at the college.

Question 2

A researcher studied volunteers from employees of a company. The participants chose whether to use a sleep routine or not, and the group using a sleep routine showed a greater improvement in reaction time. Which conclusion is best supported?

Using a sleep routine causes greater improvement in reaction time for all employees of the company.
Using a sleep routine causes greater improvement in reaction time for people like the participants, but the result cannot be generalized to all employees of the company.
Using a sleep routine is associated with greater improvement in reaction time among the participants only; neither cause nor generalization is supported.
Using a sleep routine is associated with greater improvement in reaction time among employees of the company, but cause and effect cannot be concluded.