Two treatments for kidney stones are compared. Treatment B succeeds more often overall: 83% against 78%. Yet for small stones, Treatment A does better. For large stones, Treatment A does better again. A treatment that wins in every group and still loses in the total sounds impossible. It is real data, and it has a name: Simpson's paradox.
The table comes from a 1986 study of kidney-stone treatments by Charig and colleagues, which is the standard textbook example. Treatment A was open surgery and Treatment B was closed surgery, 350 patients each.
Small stones: A succeeded in 81 of 87 cases (93%), B in 234 of 270 (87%). Large stones: A succeeded in 192 of 263 (73%), B in 55 of 80 (69%). Everyone: A 273 of 350 (78%), B 289 of 350 (83%).
Look at who got which treatment. Doctors sent most of the hard cases, the large stones, to Treatment A: 263 of its 350 patients. Treatment B got mostly the easy ones: 270 of 350 had small stones. A's overall figure is dragged down by its tougher caseload, and B's is lifted by an easier one.
The success rates inside each group are fixed at the real values. The only thing you control is how many of each treatment's 350 patients have large stones. Start at the real study, then drag.
Set both sliders to the same value and the paradox vanishes: A wins overall, exactly as it does in each group. Pull them apart and the overall bars can swap places, because the overall rate is a weighted average and the weights are the mix of patients.
An overall success rate is each group's rate multiplied by that group's share of the patients, added up. Change the shares and the total changes, even when nothing about the treatment itself has changed. Stone size is a hidden third factor: it affects who gets which treatment and also affects whether the treatment works. Statisticians call such a factor a confounder.
Here the answer is the split table. Stone size is something that was decided before the treatment, and it changes both the choice and the outcome, so comparing like with like is the fair test. But there is no rule that says to always split the data. If the extra variable is something the treatment itself causes, such as a side effect that happens afterwards, splitting can hide the very effect you want to measure. The data alone cannot tell you which table to believe. You need to know the story of how the numbers came about.
In the fall of 1973 the University of California, Berkeley admitted about 44% of its 8,442 male graduate applicants and about 35% of its 4,321 female applicants, which looked like clear bias. In 1975, P. J. Bickel, E. A. Hammel and J. W. O'Connell published a study in Science examining the figures department by department. Within departments there was no large bias against women; women had applied more often to departments that rejected more of everyone. The aggregated gap came largely from where people applied, the same mechanism as the kidney stones.
Whenever a headline compares two overall numbers, ask what the groups inside them look like, and whether the two sides are made of the same kind of people. The statistician Edward Simpson described the effect in 1951, and the name "Simpson's paradox" was attached to it by Colin Blyth in 1972. Averages are honest arithmetic, but they hide the mix they were averaged over.