Abstract

Publication bias has emerged in the last decade as a major impediment for the accumulation of scientific knowledge. In this paper, I study the consequences of publication bias for the use of scientific evidence by policymakers. I delineate a model where a decision maker uses published results to decide on which treatments to allocate. I show that publication bias distorts the optimal allocation of treatments through several mechanisms. First, under publication bias, more ineffective treatments are implemented than what would be deemed optimal without publication bias. Second, under publication bias, the allocation of treatments does not converge to the optimal one as more studies are added to the evidence base. Third, policy-makers can undo some of this bias by ranking programs that they wish to implement. I show that publication bias makes the actual allocation less efficient that the unbiased one. I also show that there are cases in which this approach is severely biased because publication bias does not preserve the ranking of treatment effectiveness. I find evidence for all these sources of bias in an empirical application where I compare the treatment allocation obtained using the results of pre-registered replications to the treatment allocation that comes out of published studies.

Reference

Sylvain Chabé-Ferret (Toulouse School of Economics), “Treatment (Mis)Allocation under Publication Bias”, IAST Lunch Seminar, Toulouse: IAST, December 1, 2020, 14:00–15:00, room Zoom Meeting.