Statistics · Maths EE idea · Accessible
Spearman vs Pearson
A research question to start from
When do Spearman's rank and Pearson's correlation coefficients disagree, and which better detects a monotonic but non-linear relationship?
A starting point, not your question: change the case, the comparison or the limit until it is yours. The research-question builder helps you check it.
Why it works as a maths EE
Two methods compared with a proof of the Spearman formula and designed test cases.
Mathematics you would need
- Correlation coefficients
- Sums of squares of integers
- Monotonic functions
- Ties in ranks
Much of this goes beyond the DP course. That is expected in a maths EE, but you must understand and explain everything you use.
One possible line of attack
- Derive Spearman's formula from Pearson's applied to ranks.
- Construct datasets where they disagree.
- Evaluate which to use when.
Scope and difficulty
Accessible. Accessible.
Pitfalls
- Formulas used without derivation.
- Only one dataset.
Where to start reading
Search a library catalogue or a university's open lecture notes for: Spearman formula derivation from Pearson; monotonic non-linear correlation. Prefer textbooks, lecture notes and journal articles to a single website, and cite everything you use (how to reference a maths EE).