« No happy clappy TV: Clinton & interviewer on Fox News | Main | John Peel Day: Teenage dreams, so hard to beat »
October 9, 2006
Why the mind hates probabilities (and gets out-smarted by science marketers)
A saying among pilots states: You start with a bag full of luck and an empty bag of experience. The trick is to fill the bag of experience before you empty the bag of luck.
Well, too bad that's not how probability works. However, it seems to me that it characterises the general perception of what "probability" and "risk" mean pretty well. When you've played the lottery for twenty years, it is not finally time for your turn to win. History doesn't matter, and there is no bag of luck that is full or empty.
I conjecture that the human mind isn't made to deal with probabilities in a rational fashion. What?, you might ask. Us, with our brains that regularly beat every machine-learner at recognizing complex patterns? Let me give you an example: Do you know about the Baader-Meinhof effect? (Un-fittingly, they were a terrorist group in 60/70's Germany.) What the term refers to is that you'd often notice that your mom would call you on the phone, just when you were thinking about her. Things like that would happen a couple of times, and you begin to think that there must be something supernatural at play. Can your thoughts make your mom dial your number? Do your dreams cause hiccups a thousand miles away?
Or, Murphy's Law: the lane you're in always moves slower. Unfortunately, you forget to make a mental note every time your lane is faster!
It's simply our brain that simply pays special attention when these strange coincidences occur, while we do not take note when something usual happens. Some psychologists say that we mostly learn from the unexpected. So in a way, listening up when the unpredicted takes place is beneficial to learning from experience. But it clearly biases our conscious perception of the world around us towards the unusual.
And of course there's a second kind of skew in our judgment: we're biased to arrive at favorable conclusions.
One of the problems with that reasoning is that you can't really say much from a sample of size one (her friend). From this limited evidence, you can't tell how much more likely cigarettes make it to run into problems when you're getting old. But you can tell that smoking is bad for you when you look at thousands of patients, and thousands of healthy people. And that's why we do studies rather than believing our instincts.
So, I may conclude that the equipment we work with is a little weak on the "objective statistics" side. Guesswork and calculated probability based on evidence can be pretty far off. It's unfortunate that people generally get little formal education in interpreting statistics. What does it mean when a study finds that some drug significantly enhances weight-loss?
Not much! "Significant" in the context of a clinical study only means that the tested drug most likely makes a difference overall. For example, it could be that this new slim-down substance allows you to lose 70 grams more over the course of a month! That's certainly not worth spending £100 on, is it?
Drugs companies happily abuse the statistic terminology in their favor. Science journalism happily copies the stats-speak. The reporters forget that among non-scientists, "significant" certainly means something else, namely "grave", "important", "notable". That's why I tend to use the term "reliable" in my papers when speaking of statistical significance.
In short: Don't always believe what your guts tell you, especially if you have a vested interest in believing one way or the other. And when you read about statistics, don't forget that "significant" only means something like "makes a difference". What matters is how big this difference is.
Posted by dr at October 9, 2006 11:38 PM
Trackback Pings
Please use the following TrackBack URL:
http://www.davids-world.com/~dr/cgi-bin/mt/mt-tb.cgi/169