Bad Statistics and A Good Headline Make Easy Science
30th September 2008
Today the Journal of the American Medical Association is reporting a letter that describes a simple statistical analysis of traffic fatalities on election day. As detailed at WebMD, the researchers found an 18% increase in the number of traffic fatalities on the Tuesday election day from 1976 through 2004 as compared to the number of traffic fatalities on the Tuesdays before and after those election Tuesdays. On average there were 158 traffic fatalities on election Tuesday compared to 134 fatalities on the preceding and following Tuesdays. The authors of the study advised alertness, buckling up, subsidized public transportation, voting centers within walking distances, tamper-proof remote voting, or more traffic enforcement on Election Day.
Let’s begin with the obvious. While a real quantitative increase does exist, with databases this large a mean difference of just a few points will always be statistically significant which is enough for JAMA in this instance although reliance on mere significance is the mark of the beast in standard research methods. It would be interesting to know, for example, what the daily standard deviation in traffic fatalities is and compare this effect size (184 / 134 or 118%) and see if it exceeds random variation. It would also be nice if the study authors had adjusted their results for the increase in number of miles driven. In fact, it is commonplace to adjust sheer number of fatalities against some standard like number of miles travelled. That these authors did not make such adjustment makes the research and JAMA’s publication of it sheer nonsense. More people are driving on election day and probably drive longer distances than on an average day. To not adjust for this is transparently clever, biased, and deceitful. It affords them the opportunity to look like concerned and responsible citizens warning of a danger . . . that does not exist.
You cannot always trust scientific journal publications and accept their reports at face value.
Please realize that I’m not saying to drive like an idiot on election day, Tuesdays, or any day of the week. Things to remember: Just don’t buy what those gypsy girls at JAMA are selling today.
Posted in Applications, Idiot's Persuasion Guide | Comments Off