TOK knowledge questions about mathematics

Statistics, data and evidence: TOK knowledge questions

Statistics turns data into claims that decide medical, legal and political questions. These questions look at p-values, correlation, samples, averages, graphs and probability, and at how each can mislead as well as inform.

Questions on this page

Each question has claims, counterclaims, real-life situations from mathematics and pointers for your essay and exhibition. Check dates and figures in a reliable source before you use them, and cite that source. Link a question to the prescribed title or exhibition prompt you are working on, in your own words: see using maths examples in your essay and choosing exhibition objects. The study headings (scope, perspectives, methods and tools, ethics) are our own.

What does the replication crisis reveal about how statistics turns data into knowledge?

Knowledge and the knower Knowledge and politics Methods and tools

A common rule in research has been to call a result 'significant' if its p-value is below 0.05. When researchers tried to repeat well-known studies, many results did not hold up. This question asks what went wrong: the mathematics, the way it was used, or the system that rewarded it.

Claims

  • The mathematics of significance testing is sound; the crisis came from misuse, such as testing many hypotheses and reporting only the ones that worked.
  • Replication is how science corrects itself, so the crisis shows statistical methods working as intended over time.

Counterclaims

  • A fixed threshold such as 0.05 invites people to treat a continuous measure of evidence as a yes-or-no verdict, so the method itself encourages error.
  • Journals and careers reward surprising significant results, so the statistics cannot be separated from the incentives of the people using them.

Real-life situations from mathematics

Reproducibility Project: Psychology. In 2015 the Open Science Collaboration reported repeating 100 published psychology studies. Almost all the originals had reported significant results, but only about a third of the repeats did.

The ASA statement on p-values. In 2016 the American Statistical Association published an unusual formal statement warning that a p-value does not measure the probability that a hypothesis is true, and that scientific conclusions should not rest on whether it passes a single threshold.

Use this in your TOK work

Essay. Very strong for titles about the reliability of the human or natural sciences, or about methods and trust.

Exhibition. A printed statistics table with 'p < 0.05' highlighted makes a pointed object for a prompt about how we decide what counts as evidence.

The mathematics behind it

When can a statistical correlation justify a claim about cause?

Knowledge and the knower Methods and tools

Everyone learns that correlation does not imply causation, yet many of our most important causal beliefs, such as that smoking causes cancer, rest largely on statistical association. This question asks what extra reasoning turns a correlation into knowledge of a cause.

Claims

  • When an association is strong, consistent across studies, follows the right time order and has a plausible mechanism, a causal conclusion is justified even without an experiment.
  • Statistical methods can adjust for other variables, which helps separate real causes from coincidences.

Counterclaims

  • A hidden third variable can always be suggested, so correlational evidence never gives the certainty of a controlled experiment.
  • Deciding that the evidence is 'enough' is a judgement that can be influenced by interests, as the history of tobacco research shows.

Real-life situations from mathematics

Smoking and lung cancer. In 1950 Richard Doll and Austin Bradford Hill published a study linking smoking with lung cancer. The statistician R. A. Fisher argued the link might come from a common genetic factor. In 1965 Bradford Hill set out considerations, such as strength and consistency, for judging when an association is likely to be causal.

Ice cream and drowning. Ice cream sales and drownings rise and fall together over a year, a standard example of two effects of a common cause: warm weather.

Use this in your TOK work

Essay. Fits titles about evidence, causation and certainty in the human and natural sciences.

Exhibition. A scatter graph from a news article claiming one thing causes another is an easy object to analyse critically.

The mathematics behind it

Can a larger set of data ever give less reliable knowledge than a smaller one?

Knowledge and politics Knowledge and technology Methods and tools

It seems obvious that more data means better knowledge. Statistics teaches that how data is collected can matter more than how much there is. This question asks how sampling shapes what data can tell us.

Claims

  • A small random sample can represent a population well, while a huge biased sample repeats its bias with false confidence.
  • Statistical theory lets us state how uncertain a well-designed sample is, which is itself knowledge.

Counterclaims

  • With very large data sets, some biases can be detected and corrected, so size is still valuable.
  • Truly random samples are rare in practice, so the choice is often between different kinds of imperfect data.

Real-life situations from mathematics

The 1936 Literary Digest poll. The magazine collected about 2.4 million replies and predicted that Alf Landon would beat Franklin Roosevelt in the US presidential election. Roosevelt won by a landslide. George Gallup's far smaller but better-chosen sample called the result correctly.

Online surveys. Surveys anyone can answer online attract people with strong views and spare time, so their results can be misleading however many people respond.

Use this in your TOK work

Essay. Fits titles about evidence, quantity versus quality, or how we know about large groups.

Exhibition. A survey form from a website or a magazine is an everyday object for a prompt about how knowledge is collected.

The mathematics behind it

How should we judge a statistical claim reported in the news?

Knowledge and language Knowledge and politics Perspectives

Headlines often turn numbers into simple stories: risks 'double', a group is 'twice as likely'. The numbers may be correct while the impression is wrong. This question asks what a reader needs to know to turn a reported statistic into reliable knowledge.

Claims

  • Asking a few standard questions (compared with what, out of how many, measured how) lets any reader evaluate most claims.
  • Official statistics agencies and fact-checkers make reliable figures available, so careful readers can check claims.

Counterclaims

  • Most people do not have the time or training to check, so in practice they trust or distrust a source rather than the numbers.
  • Choosing which true figure to report is itself a perspective; even accurate statistics can frame a story.

Real-life situations from mathematics

The 1995 contraceptive pill scare. In October 1995 UK regulators warned that some newer contraceptive pills doubled the risk of blood clots. The relative risk was correct, but the absolute risk was very small. Many women stopped taking the pill, and an increase in unplanned pregnancies and abortions was recorded the following year.

Relative and absolute risk. A rise from 1 in 10,000 to 2 in 10,000 is a 100% increase in relative terms but one extra case in 10,000 in absolute terms. Both figures are true; they tell different stories.

Use this in your TOK work

Essay. Fits titles about trust, the media, or how language shapes knowledge.

Exhibition. A newspaper headline about risk is one of the best exhibition objects: ask what the number does and does not tell you.

The mathematics behind it

Can a single average represent a group fairly?

Knowledge and politics Perspectives

Means, medians and percentages compress many people into one number. That makes comparison possible, but it can hide differences that matter. This question asks when summarising helps knowledge and when it distorts it.

Claims

  • Averages make patterns visible that are impossible to see in raw data, and choosing the right average for the purpose gives fair knowledge.
  • Statistics can show its own limits, for example by reporting spread and subgroup figures alongside averages.

Counterclaims

  • The same data can support opposite conclusions depending on how it is grouped, so summaries can mislead even when correctly calculated.
  • Choosing which average to report can be a political decision, as in debates over mean versus median income.

Real-life situations from mathematics

Berkeley admissions, 1973. Overall figures for graduate admissions at the University of California, Berkeley, appeared to show women admitted at a lower rate than men. A 1975 analysis by Bickel and colleagues showed that within most departments women did as well or better; women had applied more often to departments that admitted fewer applicants. This is an example of Simpson's paradox.

Mean and median income. In countries with a few very high earners, mean income is well above median income, so the 'average' person earns less than the average. Which figure is quoted changes the story.

Use this in your TOK work

Essay. Fits titles about simplification, representation or the human sciences.

Exhibition. A school report showing a class average could make a personal object for a prompt about whether knowledge can be objective.

The mathematics behind it

What does a probability mean when an event happens only once?

Knowledge and the knower Knowledge and politics Scope

A fair coin lands heads half the time over many throws. But what does it mean to say there is a 30% chance a particular election goes one way? This question asks whether probabilities describe the world, our knowledge of it, or both.

Claims

  • A probability for a single event measures how strongly the evidence supports it, and such probabilities can be checked over many forecasts: of all the events called '30%', about 30% should happen.
  • Probability lets us act sensibly under uncertainty, which is knowledge even if it is not certainty.

Counterclaims

  • When a single forecast 'fails', people often conclude the forecaster was wrong, so probabilities are easily misunderstood and may give an illusion of knowledge.
  • Different reasonable models give different probabilities for one-off events, so the number may say more about the modeller than the world.

Real-life situations from mathematics

Election forecasts in 2016. Before the 2016 US presidential election, the forecasting site FiveThirtyEight gave Donald Trump roughly a 29% chance of winning. When he won, many people said the forecast had been wrong, although events with a 29% chance happen often.

Weather forecasts. A forecast of a 70% chance of rain is judged by checking, over many days, whether it rained on about 70% of the days with that forecast.

Use this in your TOK work

Essay. Fits titles about uncertainty, prediction or how we evaluate experts.

Exhibition. A betting slip or a forecast graphic with a percentage is a striking object for a prompt about certainty.

The mathematics behind it

Can the same data support opposite conclusions depending on how it is shown?

Knowledge and language Knowledge and politics Perspectives

Axes can start at zero or not, scales can be linear or logarithmic, and time periods can be chosen carefully. Each choice can be defended, and each changes what a reader sees. This question asks whether visualising data is a neutral act.

Claims

  • Different scales reveal different true features of the data, so using several views gives fuller knowledge, not distortion.
  • Good conventions, such as labelled axes and stated sources, let readers interpret a graph correctly.

Counterclaims

  • Most readers look at the shape of a graph, not its axes, so the designer's choices shape belief in ways readers do not notice.
  • Graphs are often made to persuade, so they always carry the perspective of whoever made them.

Real-life situations from mathematics

COVID-19 graphs on log scales. In 2020 many news outlets showed case numbers on logarithmic scales, which make steady exponential growth look like a straight line. Some readers took a straightening line to mean the epidemic was slowing when it was not.

Truncated axes. A bar chart whose vertical axis starts at 95% instead of 0% can make a difference of two percentage points look enormous. The numbers are correct; the impression is not.

Use this in your TOK work

Essay. Fits titles about representation, bias or the role of images in knowledge.

Exhibition. Two versions of the same graph, one on a linear and one on a log scale, make an excellent object pair.

The mathematics behind it

Should new evidence change what we believe according to a fixed mathematical rule?

Knowledge and technology Knowledge and the knower Methods and tools

Bayes' theorem says exactly how a rational person should update a belief when new evidence arrives. Applied to medical tests, it gives results many people find surprising. This question asks whether rational belief can be reduced to a formula.

Claims

  • Bayes' theorem is a consequence of the rules of probability, so ignoring it means reasoning inconsistently.
  • It explains common mistakes, such as ignoring how rare a condition is, and so improves real decisions.

Counterclaims

  • The formula needs prior probabilities, which are often guesses, so its output can be no better than the judgement that went in.
  • People's beliefs also depend on trust, values and context that are hard to put into numbers.

Real-life situations from mathematics

Screening tests. If a disease affects 1 in 1,000 people and a test detects it 99% of the time with a 1% false-positive rate, most people who test positive do not have the disease. Studies have found that many doctors misjudge this kind of question.

Spam filters. Early email spam filters used Bayes' theorem to update the probability that a message is spam from the words it contains, an everyday example of rule-based updating.

Use this in your TOK work

Essay. Fits titles about evidence, belief and changing our minds.

Exhibition. A home test kit's leaflet with accuracy figures is a strong object for a prompt about how we know what we know.

The mathematics behind it

Other areas: Proof · Nature of maths · Infinity · Models · Technology · The knower · Culture · Ethics · all 49 knowledge questions