Decline Effects: Types, Mechanisms, and Personal Reflections
🧐 Skeptical/Critical ↗📌 Appears in:
Plain English Summary
Ever noticed how a splashy scientific finding shrinks when other labs try repeating it? Protzko and Schooler lay out four flavors of this 'decline effect': the original was a fluke, a real finding got inflated by tiny studies, the effect only works under conditions nobody pinned down, or — here's the wild part — it genuinely fades over time even when everything is held constant. Schooler experienced this firsthand: his verbal overshadowing effect (describing a face in words makes you worse at recognizing it) shrank across a 30-lab replication. Refreshingly, the two authors disagree. Protzko calls most 'genuine' declines statistical mirages; Schooler entertains stranger explanations echoing quantum observer effects. Both prescribe pre-registration and large coordinated replications.
Research Notes
Chapter 6 in Lilienfeld & Waldman (Eds.), Psychological Science Under Scrutiny (Wiley, pp. 85-107). Unique dual-voice structure: Protzko is explicitly skeptical of unconventional mechanisms while Schooler entertains observer-related explanations akin to quantum measurement effects. Directly relevant to parapsychology's replication debates and the question of whether psi effects genuinely decline or were never real.
A taxonomy of four types of declining effect sizes in science: false positive (no true effect; e.g., Mozart effect), inflated (true effect exaggerated by small N and selective reporting), under-specified (true effect but boundary conditions unknown), and genuinely decreasing (true effect diminishes over time with variables held constant). General mechanisms include underpowered studies, publication bias, and selective reporting. Schooler reports his verbal overshadowing effect declined from ~25% impairment to markedly smaller effects in a 30-lab registered replication. Protzko argues most genuinely decreasing effects are Type I errors of a mega-analytic framework. Both advocate pre-registration and prospective multi-site replication.
Links
Related Papers
Companion
Cites
- False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant — Simmons, Joseph P (2011)
- The Garden of Forking Paths: Why Multiple Comparisons Can Be a Problem, Even When There Is No "Fishing Expedition" or "P-Hacking" and the Research Hypothesis Was Posited Ahead of Time — Gelman, Andrew (2013)
- Replication Unreliability in Psychology: Elusive Phenomena or "Elusive" Statistical Power? — Tressoldi, Patrizio E (2012)
- Experimenter Effects and the Remote Detection of Staring — Wiseman, Richard (1997)
Also by these authors
More in Methodology
Paranormal belief, conspiracy endorsement, and positive wellbeing: a network analysis
Planning Falsifiable Confirmatory Research
Addressing Researcher Fraud: Retrospective, Real-Time, and Preventive Strategies — Including Legal Points and Data Management That Prevents Fraud
Quantum Aspects of the Brain-Mind Relationship: A Hypothesis with Supporting Evidence
Paranormal beliefs and cognitive function: A systematic review and assessment of study quality across four decades of research
📋 Cite this paper
Protzko, John, Schooler, Jonathan W (2017). Decline Effects: Types, Mechanisms, and Personal Reflections. Psychological Science Under Scrutiny: Recent Challenges and Proposed Solutions (Wiley). https://doi.org/10.1002/9781119095910.ch6
@article{protzko_2017_decline_effects,
title = {Decline Effects: Types, Mechanisms, and Personal Reflections},
author = {Protzko, John and Schooler, Jonathan W},
year = {2017},
journal = {Psychological Science Under Scrutiny: Recent Challenges and Proposed Solutions (Wiley)},
doi = {10.1002/9781119095910.ch6},
}