Technology, algorithms and AI · Ethics
Can an algorithm be biased if it only follows mathematics?
Algorithms decide what we see online, who gets loans and, in some places, who is released on bail. They are built from mathematics that looks neutral. This question asks where bias can enter, and who is responsible for it.
Claims
- Algorithms apply the same rule to everyone and can be audited, so they can be fairer than human judgement, which is inconsistent and often biased.
- Bias comes from data and design choices, not from the mathematics, so better data and checks can fix it.
Counterclaims
- If an algorithm learns from past decisions that were unfair, it repeats that unfairness while appearing objective.
- Different mathematical definitions of fairness can be impossible to satisfy at the same time, so someone must choose, and that choice is ethical, not mathematical.
Real-life situations from mathematics
COMPAS risk scores
In 2016 the news organisation ProPublica analysed a risk-scoring tool used in US courts and reported that it wrongly flagged Black defendants as high risk more often than white defendants. The tool's makers disputed the analysis, and the debate showed that different measures of fairness can conflict.
Exam grades in 2020
With exams cancelled in 2020, England's exams regulator used a statistical model to adjust teachers' predicted A level grades, and about four in ten were lowered. After protests the government reverted to teachers' grades. The IB also used a model for its May 2020 results and faced complaints from students and schools.
Check dates and figures in a reliable source before you use them, and cite that source.
Use this in your TOK work
Essay. Very strong for titles about technology, ethics and objectivity. The 2020 exam grades are a personal and relevant example for IB students.
Exhibition. A screenshot of a recommendation feed or a results slip from 2020 is an object close to students' lives.
Link it to the prescribed title or the exhibition prompt you are working on, in your own words. See using maths examples in your essay and choosing exhibition objects.
Themes and study heading
Knowledge and technology Knowledge and politics Ethics
Responsibility in making and using mathematics.