Timeline of the Replication Crisis
How concerns about statistical power, publication bias, and repeatability changed psychology and economics, from Cohen's 1962 analysis to the SCORE results of 2026.
- Category
- Social Sciences
- Events
- 16 events 1962 – 2026
- Generated
- by gpt-6
1962: Cohen documents low statistical power
Jacob Cohen examined studies in the Journal of Abnormal and Social Psychology and found that many had limited ability to detect effects. His analysis made sample size and statistical power central concerns well before the modern replication debate.
https://en.wikipedia.org/wiki/Power_(statistics)1979: Rosenthal describes the file-drawer problem
Robert Rosenthal examined how unpublished studies with nonsignificant results could distort conclusions drawn from the published literature. The file-drawer problem connected individual publication decisions with biased estimates of research effects.
https://en.wikipedia.org/wiki/Publication_bias2005: Ioannidis models why published findings can fail
John Ioannidis published "Why Most Published Research Findings Are False." His mathematical argument showed how low prior probabilities, weak statistical power, and bias could undermine the reliability of positive findings; it was not a direct count of all published research.
https://en.wikipedia.org/wiki/Why_Most_Published_Research_Findings_Are_False2011: Researchers demonstrate false-positive flexibility
Joseph Simmons, Leif Nelson, and Uri Simonsohn published "False-Positive Psychology," showing how flexible choices in data collection and analysis could produce apparently significant results. They recommended disclosure practices to make those choices visible.
https://en.wikipedia.org/wiki/Data_dredging2012: A survey measures questionable research practices
Leslie John, George Loewenstein, and Drazen Prelec surveyed psychologists about research practices such as selective reporting. The aggregate responses helped shift attention from isolated misconduct cases toward incentives and routine decisions that could bias the literature.
https://en.wikipedia.org/wiki/Research#Research_ethics2013: The Center for Open Science is founded
Brian Nosek and Jeffrey Spies established the Center for Open Science. The nonprofit developed infrastructure and coordinated projects intended to make research materials, data, and analytical decisions more transparent and easier to evaluate.
https://en.wikipedia.org/wiki/Center_for_Open_Science2013: Journals introduce registered reports
Cortex and other early adopters introduced publication formats that reviewed study questions and methods before results were known. In-principle acceptance reduced the incentive to publish only significant findings when researchers followed an approved protocol.
https://en.wikipedia.org/wiki/Registered_report2014: Many Labs publishes coordinated replications
The first Many Labs project published replications of 13 effects across 36 samples. Its common protocols made it possible to compare findings across research settings and study variation between sites rather than relying on one attempted replication.
https://en.wikipedia.org/wiki/Replication_crisis2015: A collaboration repeats 100 psychology studies
The Open Science Collaboration published replications of 100 studies from three psychology journals. Thirty-six percent of replications produced statistically significant results, and estimated effects were generally smaller; the sample was not a random survey of all psychology.
https://en.wikipedia.org/wiki/Reproducibility_Project2016: Statisticians clarify the limits of p-values
The American Statistical Association issued a statement explaining that a p-value did not measure the probability that a hypothesis was true or the importance of an effect. The statement challenged mechanical use of significance thresholds in scientific interpretation.
https://en.wikipedia.org/wiki/P-value2016: Economists repeat laboratory experiments
Colin Camerer and collaborators repeated 18 laboratory experiments from two leading economics journals. Eleven produced significant effects in the original direction, while effect sizes were generally smaller, extending coordinated replication beyond psychology.
https://en.wikipedia.org/wiki/Experimental_economics2017: The Psychological Science Accelerator forms
Christopher Chartier founded a distributed network of psychology laboratories to coordinate research across countries. The Psychological Science Accelerator sought more diverse samples and shared infrastructure for large collaborative studies.
https://en.wikipedia.org/wiki/Crowdsourced_psychological_science2018: Researchers retest high-profile social science findings
Colin Camerer and collaborators published replications of 21 social science experiments from Nature and Science. Thirteen produced significant results in the original direction, and the study compared replication outcomes with researchers' predictions.
https://en.wikipedia.org/wiki/Replication_crisis2018: Many Labs 2 compares 28 findings internationally
The Many Labs 2 collaboration found significant effects in the original direction for 14 of 28 findings. Variation in replicability was more closely associated with the effect being tested than with the location of the replication sample.
https://en.wikipedia.org/wiki/Replication_crisis2021: Researchers compare registered and conventional reports
Courtney Soderberg and collaborators reported that reviewers rated registered reports more highly on several measures of research rigor and quality. The observational comparison supported study of publication reform without establishing that the format alone caused the differences.
https://en.wikipedia.org/wiki/Registered_report2026: SCORE reports multiple dimensions of repeatability
The Systematizing Confidence in Open Research and Evidence program published results on reproducibility, robustness, and replication in social science. Its new-data component retested 274 claims from 164 papers; about half replicated, with substantially smaller effect sizes.
https://en.wikipedia.org/wiki/Replication_crisis