Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project
📄 Original study ↗📌 Appears in:
Plain English Summary
Remember Daryl Bem's famous 2011 claim that people can sense the future? This massive project put it to the ultimate test. Twenty-nine experts, including both believers and skeptics, co-designed the study together so nobody could complain about unfair methods. Then ten labs across nine countries ran over 37,000 trials with 2,115 participants. The result? Participants guessed correctly 49.89% of the time, essentially dead-on chance (50%). The statistical evidence strongly favored 'nothing is happening here.' What makes this study special is its extreme transparency: open data, external audits, tamper-proof software, and real-time reporting. The conclusion is striking: Bem's original precognition effect disappears once you lock down researcher wiggle room and publication bias.
Research Notes
The most methodologically transparent replication of Bem's FTF Exp 1 to date — unique for its adversarial consensus design process involving both proponents and skeptics. Central to Controversy #2 (Bem FTF debate) as it directly tests the 'pure bias theory' prediction that positive psi findings vanish under rigorous controls.
A pre-registered, multi-laboratory replication of Bem's (2011) Experiment 1 on precognition, co-designed by a consensus panel of 29 experts including both proponents and opponents. Across 10 laboratories in 9 countries, 2,115 participants completed 37,836 forced-choice erotic trials. Hit rate was 49.89% (chance = 50%), yielding BF₀₁ ≈ 72 — strong Bayesian evidence for the null model. The 90% HDI for population success rate was [49.57%, 50.40%], with >99% probability below 50.6%. Credibility-enhancing tools included direct data deposition, born-open data, real-time reports, external audits, and tamper-evident software. The authors conclude that Bem's original effect does not survive when researcher degrees of freedom and publication bias are controlled.
Links
Related Papers
Replication Of
Cites
- Feeling the Future: A Meta-Analysis of 90 Experiments on the Anomalous Anticipation of Random Future Events — Bem, Daryl J (2015)
- Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi — Wagenmakers, Eric-Jan (2011)
- Must Psychologists Change the Way They Analyze Their Data? — Bem, Daryl J (2011)
Companion
- Correcting the Past: Failures to Replicate Psi — Galak, Jeff (2012)
- Failing the Future: Three Unsuccessful Attempts to Replicate Bem's 'Retroactive Facilitation of Recall' Effect — Ritchie, Stuart J (2012)
- Feeling the Future Again: Retroactive Avoidance of Negative Stimuli — Maier, Markus A (2014)
- A Preregistered Multi-Lab Replication of Maier et al. (2014, Exp. 4) Testing Retroactive Avoidance — Maier, Markus A (2020)
Also by these authors
Telecommunication Telepathy: A Meta-Analysis
Anomalous Cognition: An Umbrella Review of the Meta-Analytic Evidence
Explicit Anomalous Cognition: A Review of the Best Evidence in Ganzfeld, Forced-choice, Remote Viewing and Dream Studies
More in Precognition
Experimental Investigation of Precognition in Yoga Practitioners
Sentiment and Presentiment in Twitter: Do Trends in Collective Mood "Feel the Future"?
Predictive Physiological Anticipation Preceding Seemingly Unpredictable Stimuli: An Update of Mossbridge et al.'s Meta-Analysis
Precognition as a Form of Prospection: A Review of the Evidence
Perspectives on Precognition
📋 Cite this paper
Kekecs, Zoltan, Palfi, Bence, Szaszi, Barnabas, Szecsi, Peter, Zrubka, Mark, Kovacs, Marton, Bakos, Bence E, Cousineau, Denis, Tressoldi, Patrizio, Schmidt, Kathleen, Grassi, Massimo, Evans, Thomas Rhys, Yamada, Yuki, Miller, Jeremy K, Liu, Huanxu, Yonemitsu, Fumiya, Dubrov, Dmitrii, Roer, Jan Philipp, Becker, Marvin, Schnepper, Roxane, Ariga, Atsunori, Arriaga, Patricia, Oliveira, Raquel, Poldver, Nele, Kreegipuu, Kairi, Hall, Braeden, Wiechert, Sera, Verschuere, Bruno, Giran, Kyra, Aczel, Balazs (2023). Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project. Royal Society Open Science. https://doi.org/10.1098/rsos.191375
@article{kekecs_2023_transparent_psi,
title = {Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project},
author = {Kekecs, Zoltan and Palfi, Bence and Szaszi, Barnabas and Szecsi, Peter and Zrubka, Mark and Kovacs, Marton and Bakos, Bence E and Cousineau, Denis and Tressoldi, Patrizio and Schmidt, Kathleen and Grassi, Massimo and Evans, Thomas Rhys and Yamada, Yuki and Miller, Jeremy K and Liu, Huanxu and Yonemitsu, Fumiya and Dubrov, Dmitrii and Roer, Jan Philipp and Becker, Marvin and Schnepper, Roxane and Ariga, Atsunori and Arriaga, Patricia and Oliveira, Raquel and Poldver, Nele and Kreegipuu, Kairi and Hall, Braeden and Wiechert, Sera and Verschuere, Bruno and Giran, Kyra and Aczel, Balazs},
year = {2023},
journal = {Royal Society Open Science},
doi = {10.1098/rsos.191375},
}