Correcting the Past: Failures to Replicate Psi
🛡️ Critical replication ↗📌 Appears in:
Plain English Summary
Can people somehow study for a test *after* taking it and still boost their scores? That's essentially what Daryl Bem claimed in his famous 2011 experiments on precognition. A team of four researchers decided to put this to the ultimate test, running seven tightly controlled experiments with over 3,200 participants — no peeking at results early, no wiggling the methods. The verdict? Six out of seven experiments found absolutely nothing. The combined effect was essentially zero. A broader look at all 19 known replication attempts told the same story. Here's the real kicker, though: the only thing that predicted positive results was whether Bem himself ran the experiment. When he did, effects appeared; when anyone else tried, they vanished. This became a landmark study in the replication crisis — raising hard questions about whether subtle researcher choices, rather than psychic powers, drove the original findings.
Research Notes
The most comprehensive replication and meta-analytic challenge to Bem's retroactive recall paradigm, with the largest combined sample of any single replication effort. Discusses researcher degrees of freedom as a possible explanation for original positive findings. A cornerstone of Controversy #2 (Bem FTF) and the broader replication crisis.
Across seven experiments (N=3,289), the retroactive facilitation of recall paradigm from Bem's (2011) Experiments 8 and 9 was replicated using computer-standardized delivery, predetermined sample sizes, and no data inspection before stopping. Six of seven experiments found no evidence of precognition; the combined effect was d≈0.01 with Bayesian BF=70.48 providing 'extreme' support for the null. A meta-analysis of all 19 known replication attempts (N=4,091) yielded an overall effect of d=0.04, 95% CI [-0.00, 0.09], indistinguishable from zero. The only significant moderator was whether Bem himself conducted the experiment (d=0.29 vs. d=0.02 for all others).
Links
Related Papers
Replication Of
Cites
- Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi — Wagenmakers, Eric-Jan (2011)
- Meta-Analysis of Free-Response Studies, 1992–2008: Assessing the Noise Reduction Model in Parapsychology — Storm, Lance (2010)
- Extraordinary Claims Require Extraordinary Evidence: The Case of Non-Local Perception, A Classical and Bayesian Review of Evidences — Tressoldi, Patrizio E (2011)
- False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant — Simmons, Joseph P (2011)
Companion
- Failing the Future: Three Unsuccessful Attempts to Replicate Bem's 'Retroactive Facilitation of Recall' Effect — Ritchie, Stuart J (2012)
- Results from a Confirmatory Replication Study of Bem (2011): Precognitive Detection of Erotic Stimuli? — Wagenmakers, Eric-Jan (2012)
- Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice — LeBel, Etienne P (2011)
- Back from the Future: Parapsychology and the Bem Affair — Alcock, James E (2011)
- Bem's 'Feeling the Future' (2011) Five Years Later: Its Impact on Scientific Literature — Silva, Bruno A (2017)
- Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project — Kekecs, Zoltan (2023)
Meta Analyzed By
More in Skeptical
Cognitive Styles and Psi: Psi Researchers Are More Similar to Skeptics Than to Lay Believers
Searching for the Impossible: Parapsychology's Elusive Quest
False-Positive Effect in the Radin Double-Slit Experiment on Observer Consciousness as Determined with the Advanced Meta-Experimental Protocol
Cross-Examining the Case for Precognition: Comment on Mossbridge and Radin (2018)
N,N-Dimethyltryptamine and the Pineal Gland: Separating Fact from Myth
📋 Cite this paper
Galak, Jeff, LeBoeuf, Robyn A, Nelson, Leif D, Simmons, Joseph P (2012). Correcting the Past: Failures to Replicate Psi. Journal of Personality and Social Psychology. https://doi.org/10.1037/a0029709
@article{galak_2012_correcting,
title = {Correcting the Past: Failures to Replicate Psi},
author = {Galak, Jeff and LeBoeuf, Robyn A and Nelson, Leif D and Simmons, Joseph P},
year = {2012},
journal = {Journal of Personality and Social Psychology},
doi = {10.1037/a0029709},
}