The latest persuasion news in brief from the third issue of 2011 from the Journal of Personality and Social Psychology. Things are going to start happening now!
I believe that JPSP has made history with this issue – an author appeared on the Colbert Report in support of the publication of his research! Doubtless people will hoard this issue as a collector’s item for one article which means that people won’t read the other articles in the issue. Therefore, I’m only going to Phonebook the infamous three here and then provide individual posts on two other reports that bear much more attention and thought.
Is Peer Review at JPSP Only Psychological?
Wagenmakers, Wetzels, Borsboom, & van der Maas provide a simple, calm, and rational presentation of how to think with statistics while doing psychological science. They politely dismember Daryl Bem’s report on the psi effect (precognition or knowing the future) with a straightforward application of the scientific method with an emphasis upon basic statistical principles. They note various obvious errors in methods with Bem’s work – an artful manipulation of p values, careless capitalization on post hoc testing, the biased use of exploratory and confirmatory standards – and his thinking – if psi is true, what else should be true and false. In so doing, the authors follow the standards of science that allegedly drive the peer review process at JPSP which should have precluded publication of Bem’s paper and Judd and Gawronski’s editorial comment.
Wagenmakers, E.–J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011). Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100(3), 426-432.
Does psi exist? D. J. Bem (2011) conducted 9 studies with over 1,000 participants in an attempt to demonstrate that future events retroactively affect people’s responses. Here we discuss several limitations of Bem’s experiments on psi; in particular, we show that the data analysis was partly exploratory and that one-sided p values may overstate the statistical evidence against the null hypothesis. We reanalyze Bem’s data with a default Bayesian t test and show that the evidence for psi is weak to nonexistent. We argue that in order to convince a skeptical audience of a controversial claim, one needs to conduct strictly confirmatory studies and analyze the results with statistical tests that are conservative rather than liberal. We conclude that Bem’s p values do not indicate evidence in favor of precognition; instead, they indicate that experimental psychologists need to change the way they conduct their experiments and analyze their data.
The Queen of Tomorrow Lives in Cornell
Daryl Bem presents a series of experiments that provide evidence for psi, precognition, or for the rest of us, knowing the future. Really. Psi exists. In JPSP. Bem presents an entertaining rhetorical case for psi with his artful narrative, but the science? (And if you want to learn how to write rhetorical science, read this report. It makes those epi posers look like . . . uh . . . posers.) Beyond the really bad use of statistical analysis, the inconsistent and contradictory results, and the sheer argumentativeness of the report, we’ve got an outstanding example of Psychological Science, oops, JPSP science here. Bem wants us to believe his psi is real. If this is true, then what else should be true? The Queen of Tomorrow!
Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100(3), 407-425.
The term psi denotes anomalous processes of information or energy transfer that are currently unexplained in terms of known physical or biological mechanisms. Two variants of psi are precognition (conscious cognitive awareness) and premonition (affective apprehension) of a future event that could not otherwise be anticipated through any known inferential process. Precognition and premonition are themselves special cases of a more general phenomenon: the anomalous retroactive influence of some future event on an individual’s current responses, whether those responses are conscious or nonconscious, cognitive or affective. This article reports 9 experiments, involving more than 1,000 participants, that test for retroactive influence by “time-reversing” well-established psychological effects so that the individual’s responses are obtained before the putatively causal stimulus events occur. Data are presented for 4 time-reversed effects: precognitive approach to erotic stimuli and precognitive avoidance of negative stimuli; retroactive priming; retroactive habituation; and retroactive facilitation of recall. The mean effect size (d) in psi performance across all 9 experiments was 0.22, and all but one of the experiments yielded statistically significant results. The individual-difference variable of stimulus seeking, a component of extraversion, was significantly correlated with psi performance in 5 of the experiments, with participants who scored above the midpoint on a scale of stimulus seeking achieving a mean effect size of 0.43. Skepticism about psi, issues of replication, and theories of psi are also discussed.
Dissonance? I Got No Stinkin’ Dissonance
The JPSP editors publish both Bem’s “nothing up my sleeve” show and the Wagenmakers team “the man behind the curtain” expose. The editors then provide a comment article that explains there is no contradiction. Here are the key lines.
After a rigorous review process, involving a large set of extremely thorough reviews by distinguished experts in social cognition, we are publishing the following article by Daryl J. Bem . . .
What makes these findings so remarkable and certainly controversial is that they turn our traditional understanding of causality on its head. A central assumption in lay and scientific conceptions of causality is that a cause precedes its effect, not the other way round. The claim presented in the current article that participants’ responses were influenced by randomly generated stimuli that followed these responses poses a serious challenge to traditional views of causality. Needless to say, such a challenge to firmly held convictions is destined to ignite a lot of controversy.
Yet, as editors we were guided by the conviction that this paper—as strange as the findings may be—should be evaluated just as any other manuscript on the basis of rigorous peer review. Our obligation as journal editors is not to endorse particular hypotheses but to advance and stimulate science through a rigorous review process.
Judd, Gawronski, and those distinguished social cognition experts may be awakening now to the realization that they participated in a Daryl Bem dissonance experiment.
“Yeah. It’s just self perception!”
Judd, C. M., & Gawronski, B. (2011). Editorial comment. Journal of Personality and Social Psychology, 100(3), 406.