The journal, Psychological Science, continues with inSincerity as it publishes a paper which demonstrates that many other papers the journal published, should not have been published. When you can do that, then you are persuasive which is an odd pursuit for a journal with the word ‘Science’ in its title. Consider the new paper that would kill many old papers.
Geoff Cumming presents the case for what he calls the New Statistics (more on that General Semantics Persuasion Play®™© a bit later). Cumming must be reading the Persuasion Blog and all the old stat and method sources I read back in the day because he’s as cranky about contemporary science as I am. Under the heading, Research Integrity, Cumming bullets three issues.
Researchers in psychological science and other disciplines are currently discussing a set of severe problems with how we conduct, analyze, and report our research. Three problems are central:
• Published research is a biased selection of all research;
• data analysis and reporting are often selective and biased; and
• in many research fields, studies are rarely replicated, so false conclusions persist.
Cumming also hammers my favorite Idol of Research: Statistical Significance.
Despite warnings in statistics textbooks, the word significant is part of the seductive appeal: A “statistically significant” effect in the results section becomes “significant” in the discussion or abstract, and “significant” shouts “important.” Kline (2004) recommended that if we use NHST, we should refer to “a statistical difference,” omitting “significant.” That is a good policy; the safest plan is never to use the word significant.
Cumming develops the points in an argument that is so obvious one wonders why he has to make it, but if you read enough journal science, you know exactly why he makes the argument: Nobody follows it. The best proof of the disconnect is to read the journal Psychological Science.
As a top of the Blog example, just recall the Heil Geringfugig post that examined an observational design of German last names for their relationship to organizational position. The researchers found that people with powerful last names (King, Prince, etc.) were “over-represented” in leadership positions compared to people with less powerful last names (Cook, Baker, etc.). From a database of over 200,000 cases, the researchers found an increased odds ratio of 1.125 for this name effect. That is only ¼ of a Small Windowpane and in an observational design that is not replicated anywhere, yet the authors hide behind a p < .05 symbolism and tell us a truth that does not exist but that Psychological Science gladly prints.
Cumming continues the tradition I’ve noted in the Persuasion Blog and Primer (does the name Ioannidis ring a bell or a ding-a-ling?) as my fellow travelers detail the obvious and continuing mis-science of contemporary researchers more interested in pop press hits, correctly spelled last names, and 100% indirects. (Employment at the Center for Science in the Self Interest.) And, I suspect that this latest foray with Falling Apples versus Fallen Apples will have the same impact prior sallies produced: Nothing.
Persuasion explains why science won’t do science while still calling itself science. If only on that one point of Statistical Significance – make miscellaneous those niceties about registering research before data collection, publishing the entire dataset, or disclosing all data analysis – you know why Cumming’s argument will continue to be ignored. Most research ends with data that is a dog who only knows how to play dead. Researchers will not give up Statistical Significance because it is the most reliable persuasion play they have when the data won’t hunt, fetch, or even roll over.
In its way, Statistical Significance can be added to the list of Great Lies (I gave at the office, the check is in the mail, I’m from the government and I’m here to help, and others that are not polite for the PB). Everyone knows it is a lie, but it sounds so good in the situation that everyone believes it just long enough. And, since everyone has stared at a computer screen of dead dog data, everyone is willing to participate in the deception. We all will need to tell that Lie sometime.
So expect to see Psych Science publish work that switches persuasion for science in every issue. Just look for statistical significance, the first Great Lie of Persuasive Science.
Geoff Cumming. The New Statistics: Why and How. Psychological Science, first published on November 12, 2013.
P.S. The New Statistics? Cumming admits there is nothing New in his New Statistics other than:
The strategies and techniques are not themselves new, but for many researchers, adopting them would be new, as well as a great step forward.
I guess New Statistics sounds better than any title I might suggest, but it says more about the depth of the denial. We have to pretend that we are New when we do the right thing rather than the wrong thing?