Howdy! Let's finish up with some more scattered observations from Peter Bevelin's Seeking Wisdom.
8) Just saying ‘four out of five’ says nothing. So from all, or just a sample of five?
We've gotten dangerously accustomed to talking about results from surveys or studies as if they automatically represent some kind of truth. It wouldn't be such an issue if these results never strayed from the trivial or mundane - four out of five Americans put on their left sock first! - but unfortunately the results are often about more important questions, forcing us in the audience to remain skeptical until we can confirm the validity of the methodology.
9) Roulette wheels have been shown to pull in a profit even without a house edge. Players tend to pull back bets during a bad run, or stick with a double down strategy long after the returns have slowed.
Comments like this reinforce my skepticism about amateur statistical analysis (or maybe I'm just annoyed at the low standards for 'thinking' about roulette wheels). I'm sure someone somewhere watched real gamblers at a real roulette board and observed this strange phenomenon ("have been shown"). The link to betting behavior rings true; the stock market is my star witness.
But here's the thing - every single bet made on a roulette board favors the house. No exceptions. This is the only relevant reason why roulette wheels profit in the sense that no matter what else happens at the wheel, the house retains the probability edge. Could the house win without this built-in advantage? Possibly. But I think bettors would wise up, and the house would respond. If you disagree, sit at a blackjack table and openly count cards.
10a) Just enrolling low-performing students into a remedial course can bias toward ‘good results’ via regression to the mean.
10b) The body is great at healing itself, so many treatments will look effective just by leveraging the high base rate of natural success.
These subtleties of statistical behavior likely wouldn't be met with a standing ovation from the education or medical fields. But let's say you take one hundred equally capable students, then grade their semester performance on the standard 'curve'. Just by pure chance - by which I mean, factors unrelated to underlying academic ability - you'd have five percent of those students earn a failing grade. If you put them in a remedial course, each student has at least a 95% chance of doing better. For some reason, people use these results to conclude that remedial coursework has value.
The medical field has a very strong grasp of this idea but seems resigned to an obvious counterpoint - what else can we do? Fluids and bed rest are almost always a great idea, but insurance won't cover it, which means doctors aren't reimbursed for good advice; I always appreciate the online resources that demarcate symptoms into 'continue monitoring' and 'go to the hospital' categories.
11a) In the context of certain events like a large earthquake, probabilities are a distraction. In geology such an event is inevitable, and a better strategy would be to encourage anyone in the area to be fully prepared.
11b) Forensic evidence alone will produce false positives. Let’s say a city has a 500K population, with false matches in 1/20000 tests. If there is 1 sick person, that means the infected party plus 25 test positive - so any match is a 1/26 chance of being sick.
I like these quips because understanding them is the best way to demonstrate mastery of applied probability. The earthquake example is an easy starting point, reminding us that certain events occur at magnitudes that make estimating frequencies an inappropriate mode of thinking. The next is more mathematically challenging but critical; a high school student who understood it would have no immediate need for additional probability coursework (probably).
12) I've exposed your lies, baby, the underneath no big surprise...
If there's one thing to keep in mind, it's that the deceit of statistical thinking is often only just out of sight. Lift every rock, but don't be surprised when you see that, hiding all along...
...OK, fine.
That's not from this book, that's from Muse's 'Plug In Baby'. But we need to end a riff-off somewhere, and I'm out of Courtney Barnett songs. So what better way than Muse?
Thanks for reading.
