Is the Methodological Revolution in Psychology Over or Just Beginning?
📄 Original study📌 Appears in:
Plain English Summary
When Bem published his "feeling the future" experiments in 2011, psychology had a wake-up call about research rigor. But Kennedy argues pre-registering studies was just the start. He identifies eight more weak spots both mainstream psychology and psi research (the study of psychic phenomena) still haven't fixed -- sloppy record-keeping, unvalidated software, no safeguards against cheating, and cherry-picked statistics among them. His fix? Borrow the tough playbook from FDA-regulated drug trials. His punchline is deliciously strategic: once psychologists see these flaws laid bare, they'll rush to adopt reforms rather than risk their work looking as shaky as the psi studies that sparked the crisis.
Research Notes
Most systematic single-paper inventory of methodological weaknesses in psi research, applying FDA-level clinical trial standards. Directly extends Kennedy (2013) on retrospective meta-analysis and anticipates Kennedy (2017, 2024) on fraud standards. Central to the meta-debate (Controversy #10).
Prompted by Bem's (2011) 'feeling the future' experiments, psychology recognized the need for pre-registered confirmatory research. Eight additional methodological deficiencies remain unaddressed in both fields: deficient study registration, bias from dropouts and incomplete data, lack of software validation, absence of fraud prevention measures, inappropriate statistical methods for confirmatory studies, overlooked Bayesian inferential errors, weaknesses of retrospective meta-analysis versus prospective meta-analysis, and statistical dependence problems in outcome variables. Drawing on regulated medical research standards (FDA clinical trial guidance), specific practices are recommended for each deficiency. When confronted with the choice between psi and methodological flaws, psychologists will inevitably choose reform — making proactive adoption of these standards the efficient path forward.
Related Papers
Cites
- Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect — Bem, Daryl J (2011)
- Can Parapsychology Move Beyond the Controversies of Retrospective Meta-Analyses? — Kennedy, J.E (2013)
- Estimating the Reproducibility of Psychological Science — Open Science Collaboration (2015)
- Lessons from the First Two Years of Operating a Study Registry — Watt, Caroline (2015)
Same Research Program
- Experimenter Fraud: What Are Appropriate Methodological Standards? — Kennedy, J.E (2017)
- Conclusions about Paranormal Phenomena — Kennedy, J.E (2013)
- Planning Falsifiable Confirmatory Research — Kennedy, James E (2024)
- Addressing Researcher Fraud: Retrospective, Real-Time, and Preventive Strategies — Including Legal Points and Data Management That Prevents Fraud — Kennedy, James E (2024)
Companion
Also by these authors
More in Methodology
Paranormal belief, conspiracy endorsement, and positive wellbeing: a network analysis
Quantum Aspects of the Brain-Mind Relationship: A Hypothesis with Supporting Evidence
Paranormal beliefs and cognitive function: A systematic review and assessment of study quality across four decades of research
Experimental evidence of non-classical brain functions
Self-Ascribed Paranormal Ability: Reflexive Thematic Analysis
📋 Cite this paper
Kennedy, J.E (2016). Is the Methodological Revolution in Psychology Over or Just Beginning?. Journal of Parapsychology.
@article{kennedy_2016_psi_research_ready,
title = {Is the Methodological Revolution in Psychology Over or Just Beginning?},
author = {Kennedy, J.E},
year = {2016},
journal = {Journal of Parapsychology},
}