Editors' Introduction to the Special Section on Replicability in Psychological Science: A Crisis of Confidence?
π Original studyπ Appears in:
Plain English Summary
This editorial kicked off one of the most dramatic self-reckonings in modern science. In 2011-2012, psychology hit a full-blown 'crisis of confidence,' and the editors pinpoint three things that lit the fuse. First, a prominent researcher named Stapel was caught fabricating data wholesale. Second, Daryl Bem published a study claiming to find evidence for ESP (precognition, specifically) in a top journal, which triggered both fascination and widespread ridicule. Third, researchers Simmons and colleagues showed that by using common but flexible data analysis tricks, you could make completely fake effects look statistically significant way more often than the supposed 5% false alarm rate. That last one was a gut punch: it meant the problem wasn't just fraud or fringe topics β ordinary, accepted research practices were quietly inflating false results across the board. By 2012, surveys revealed how widespread these questionable research practices really were, suspiciously too many published results clustered right at the magic threshold of statistical significance, and bitter public fights broke out over failed attempts to replicate famous findings. The special journal section introduced here gathers fifteen articles tackling both the diagnosis and potential cures β from replication attempts and Bayesian statistics (a different way of weighing evidence) to pre-registration (publicly declaring your analysis plan before collecting data) and open data sharing. What makes this piece especially fascinating is how a single ESP paper became a catalyst that forced all of psychology to look in the mirror.
Research Notes
Key editorial framing the replication crisis in psychology, explicitly citing Bem's precognition study as a primary catalyst alongside the Stapel fraud and Simmons et al.'s false-positive demonstration. Essential context for understanding why psi research provoked methodological self-examination across all of psychology. Directly relevant to Controversy #2 (Bem) and #10 (meta-debate on research soundness).
An editorial introduction to the Perspectives on Psychological Science special section on replicability, chronicling the 'crisis of confidence' that unfolded in 2011-2012. Three catalysts are identified: the Stapel fraud case, Bem's (2011) ESP publication followed by widespread mockery, and Simmons et al.'s (2011) demonstration that flexible data analysis produces false-positive rates far exceeding 5%. The editors note 2012 brought further evidence of the problem: QRP prevalence surveys, suspicious clustering of p-values just below .05, and acrimonious disputes over failed social priming replications. The special section's 15 articles span diagnosis and treatment, from replication failures to Bayesian reanalyses, pre-registration, and data sharing.
Links
Related Papers
Cites
Companion
- Correcting the Past: Failures to Replicate Psi β Galak, Jeff (2012)
- Failing the Future: Three Unsuccessful Attempts to Replicate Bem's 'Retroactive Facilitation of Recall' Effect β Ritchie, Stuart J (2012)
- Too Good to Be True: Publication Bias in Two Prominent Studies from Experimental Psychology β Francis, Gregory (2012)
- Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi β Wagenmakers, Eric-Jan (2011)
- A Bayes Factor Meta-Analysis of Bem's ESP Claim β Rouder, Jeffrey N (2011)
- An Agenda for Purely Confirmatory Research β Wagenmakers, Eric-Jan (2012)
- Commentary: Reproducibility in Psychological Science: When Do Psychological Phenomena Exist? β Heino, Matti T. J (2017)
- Power failure: why small sample size undermines the reliability of neuroscience β Button, Katherine S (2013)
Extended By
Cited By
- Bayesian and Classical Hypothesis Testing: Practical Differences for a Controversial Area of Research β Kennedy, J.E (2014)
- Addressing Researcher Fraud: Retrospective, Real-Time, and Preventive Strategies β Including Legal Points and Data Management That Prevents Fraud β Kennedy, James E (2024)
- Why Science Is Not Necessarily Self-Correcting β Ioannidis, John P.A (2012)
Also by these authors
Meta-Analyses Are No Substitute for Registered Replications: A Skeptical Perspective on Religious Priming
Results from a Confirmatory Replication Study of Bem (2011): Precognitive Detection of Erotic Stimuli?
A Practical Solution to the Pervasive Problems of p Values
More in Methodology
Paranormal belief, conspiracy endorsement, and positive wellbeing: a network analysis
Planning Falsifiable Confirmatory Research
Quantum Aspects of the Brain-Mind Relationship: A Hypothesis with Supporting Evidence
Paranormal beliefs and cognitive function: A systematic review and assessment of study quality across four decades of research
Experimental evidence of non-classical brain functions
π Cite this paper
Pashler, Harold, Wagenmakers, Eric-Jan (2012). Editors' Introduction to the Special Section on Replicability in Psychological Science: A Crisis of Confidence?. Perspectives on Psychological Science. https://doi.org/10.1177/1745691612465253
@article{pashler_2012_replicability_crisis,
title = {Editors' Introduction to the Special Section on Replicability in Psychological Science: A Crisis of Confidence?},
author = {Pashler, Harold and Wagenmakers, Eric-Jan},
year = {2012},
journal = {Perspectives on Psychological Science},
doi = {10.1177/1745691612465253},
}