IA idea · Statistics & hypothesis testing

Can statistics tell two authors apart?

AI SLAI HLAA SL Solid Also in: Art & music

Research question

Is there a significant difference between the mean sentence lengths of [two authors] in comparable texts, and can sentence and word length identify which author wrote an unattributed passage?

Adapt it: change the place, the data or the comparison until the question is yours.

Why it makes a good exploration

Stylometry has been used to settle real authorship disputes. Using free public-domain texts, you can test whether two authors differ — and then try to identify a “mystery” passage.

The mathematics you'll need

Sampling from a text
Descriptive statistics and box plots
Two-sample t-test (AI)
Normal approximation and conditions
Chi-squared test on word-length categories

Course labels show where a technique sits; using maths from outside your course is fine if you explain it clearly and say it is new to you.

Where the data comes from

Download texts from Project Gutenberg; sample random chapters and count sentence lengths with a spreadsheet or short script.

Project Gutenberg — 70,000+ free public-domain books as plain text — ideal for letter and word frequency counts.

Cite every source in a footnote where you use it and in your bibliography. Check the licence of any dataset you download.

A possible outline

Choose authors from the same era and genre.
Design a random sampling method for sentences.
Compare distributions and test means.
Use your findings to classify a passage.
Reflect on editing, genre and definitions of a sentence.

Pitfalls that cost marks

Comparing a novel with a play.
Non-random sampling (e.g., only first chapters).
Assuming normality without checking.

Showing personal engagement

Choose authors you have read in English class.
Test your own essays against a classmate's (with permission).
Discuss real stylometry cases.

See Criterion C: personal engagement for what examiners look for.

Taking it further

Combine several features into a simple classifier and test its accuracy on passages you hold back.

Turn this idea into your IA

Plan this IAOpens the planner with this question, the maths and data sources filled in. Desmos workspacePlot your data, fit models and keep your graphs with your plan. AI feedback on a draftCriterion-by-criterion feedback with evidence from your own text. Coaching, not rewriting. Examiner reviewPete, a practising IA examiner, reads your draft and talks it through with you (€100).