Options for Prospective Meta-Analysis and Introduction of Registration-Based Prospective Meta-Analysis
📄 Original study ↗📌 Appears in:
Plain English Summary
Meta-analyses -- studies that pool results from many experiments -- have a sneaky problem: researchers often decide which studies to include after they already know the results, which is a bit like picking your lottery numbers after the draw. This paper lays out three ways to fix that, and the third option is the real star. Called "registration-based prospective meta-analysis," it locks in which future studies count before anyone collects data. The authors test-drive this approach on ganzfeld ESP experiments, making it the first time parapsychology has tried this kind of rigorous pre-commitment. It is a clever move to settle decades of heated debate about whether psi meta-analyses are trustworthy.
Research Notes
Directly addresses parapsychology's protracted meta-analytic debates by proposing registration-based prospective meta-analysis as a practical solution. The ganzfeld exemplar is the first such application to psi research. Key paper for the methodology/meta-debate controversy (#10).
Retrospective meta-analyses resemble exploratory rather than confirmatory research because inclusion criteria, statistical methods, and moderator decisions are made after analysts know study outcomes. Three options for prospective meta-analysis are compared: (1) preregistered plans, still allowing retrospective decisions about unanticipated variations; (2) preplanned multi-center projects, optimal but requiring funding rarely available in behavioral science; (3) registration-based prospective meta-analysis, where inclusion/exclusion of independently-initiated studies is decided at pre-registration, before data collection. An exemplar applying Option 3 to ganzfeld ESP studies at the Koestler Parapsychology Unit illustrates the approach.
Links
Related Papers
Cites
- Examining Psychokinesis: The Interaction of Human Intention With Random Number Generators—A Meta-Analysis — Bösch, Holger (2006)
- Reexamining Psychokinesis: Commentary on the Bösch, Steinkamp and Boller Meta-Analysis — Radin, D (2006)
- Meta-Analysis That Conceals More Than It Reveals: Comment on Storm et al. (2010) — Hyman, Ray (2010)
- Why Most Published Research Findings Are False — Ioannidis, John P.A (2005)
- Can Parapsychology Move Beyond the Controversies of Retrospective Meta-Analyses? — Kennedy, J.E (2013)
- Is the Methodological Revolution in Psychology Over or Just Beginning? — Kennedy, J.E (2016)
- Does Psi Exist? Lack of Replication of an Anomalous Process of Information Transfer — Milton, Julie (1999)
- Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability — Nosek, Brian A (2012)
- Meta-Analysis of Free-Response Studies, 1992–2008: Assessing the Noise Reduction Model in Parapsychology — Storm, Lance (2010)
Companion
- Registered Reports: A Method to Increase the Credibility of Published Results — Nosek, Brian A (2014)
- Why Science Is Not Necessarily Self-Correcting — Ioannidis, John P.A (2012)
- Theoretical Risks and Tabular Asterisks: Sir Karl, Sir Ronald, and the Slow Progress of Soft Psychology — Meehl, Paul E (1978)
- Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability — Nosek, Brian A (2012)
Also by these authors
More in Methodology
Paranormal belief, conspiracy endorsement, and positive wellbeing: a network analysis
Addressing Researcher Fraud: Retrospective, Real-Time, and Preventive Strategies — Including Legal Points and Data Management That Prevents Fraud
Quantum Aspects of the Brain-Mind Relationship: A Hypothesis with Supporting Evidence
Paranormal beliefs and cognitive function: A systematic review and assessment of study quality across four decades of research
Experimental evidence of non-classical brain functions
📋 Cite this paper
Watt, Caroline A, Kennedy, James E (2017). Options for Prospective Meta-Analysis and Introduction of Registration-Based Prospective Meta-Analysis. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2016.02030
@article{watt_kennedy_2017_prospective_meta,
title = {Options for Prospective Meta-Analysis and Introduction of Registration-Based Prospective Meta-Analysis},
author = {Watt, Caroline A and Kennedy, James E},
year = {2017},
journal = {Frontiers in Psychology},
doi = {10.3389/fpsyg.2016.02030},
}