Of forking paths and tied hands: Selective publication of findings, and what economists should do about it

Kasy, MaximilianORCID: https://orcid.org/0000-0002-8679-5179 (2021) Of forking paths and tied hands: Selective publication of findings, and what economists should do about it. Journal of Economic Perspective, 35 (3), pp. 175-192.

Full text not available from this repository.

Abstract or Table of Contents

A key challenge for interpreting published empirical research is the fact that published findings might be selected by researchers or by journals. Selection might be based on criteria such as significance, consistency with theory, or the surprisingness of findings or their plausibility. Selection leads to biased estimates, reduced coverage of confidence intervals, and distorted posterior beliefs. I review methods for detecting and quantifying selection based on the distribution of p-values, systematic replication studies, and meta-studies. I then discuss the conflicting recommendations regarding selection resulting from alternative objectives, in particular, the validity of inference versus the relevance of findings for decision-makers. Based on this discussion, I consider various reform proposals, such as deemphasizing significance, pre-analysis plans, journals for null results and replication studies, and a functionally differentiated publication system. In conclusion, I argue that we need alternative foundations of statistics that go beyond the single-agent model of decision theory.

Item Type: Article in Academic Journal
Classification Codes (e.g. JEL): A14, C80, L82
Research Units: IHS general publications
Status: Published
Date Deposited: 02 Aug 2021 09:15
Last Modified: 02 Aug 2021 09:15
Identification Number or DOI: 10.1257/jep.35.3.175
ISSN: 0895-3309, 1944-7965
URI: https://irihs.ihs.ac.at/id/eprint/5693

Actions (login required)

View Item View Item