Skip to main content

Why Science Is Not Necessarily Self-Correcting

πŸ“„ Original study
Ioannidis, John P.A β€’ 2012 Modern Era β€’ methodology

πŸ“Œ Appears in:

Plain English Summary

We like to think science fixes its own mistakes, but does it really? Ioannidis digs into the numbers and the picture is grim: only about 1% of psychology papers even try to repeat earlier work, and a tiny fraction of those are truly independent do-overs. With most studies running on weak statistical power (around 35%), somewhere between 30% and 95% of published 'significant' findings could simply be wrong and nobody's checking. He identifies thirteen roadblocks to self-correction, from journals refusing to publish replication attempts to researchers cherry-picking their best-looking results. This paper became a touchstone for both critics and supporters of parapsychology research, since both sides point to these same problems to argue their case. Fixes exist, but each comes with trade-offs.

Research Notes

Foundational metascience paper for evaluating psi research credibility. The impediments Ioannidis catalogues β€” low power, rare replication, publication bias, allegiance bias β€” are the same ones debated in parapsychology. Both critics (applying these to psi claims) and proponents (applying them to mainstream null results) invoke this analysis.

Self-correction is widely assumed to be a defining hallmark of science, but how often does it actually occur? Reviewing empirical evidence from psychology and biomedicine, Ioannidis argues that self-correction requires active replication effort β€” yet only ~1% of psychology papers are replications, fewer than 0.2% are independent direct replications, and most yield confirming results (perpetuated fallacies). With average power of 35% and modest bias, unchallenged fallacies may constitute 30–95% of published significant findings. A taxonomy of six discovery–replication paradigms quantifies the problem. Thirteen impediments to self-correction are catalogued, including publication bias, selective reporting, underpowered studies, and editorial bias against replication. Proposed reforms each carry unintended risks unless pursuit of truth remains the overriding priority.

Links

Related Papers

Also by these authors

More in Methodology

πŸ“‹ Cite this paper
APA
Ioannidis, John P.A (2012). Why Science Is Not Necessarily Self-Correcting. Perspectives on Psychological Science. https://doi.org/10.1177/1745691612464056
BibTeX
@article{ioannidis_2012_science_not_self_correcting,
  title = {Why Science Is Not Necessarily Self-Correcting},
  author = {Ioannidis, John P.A},
  year = {2012},
  journal = {Perspectives on Psychological Science},
  doi = {10.1177/1745691612464056},
}