The Applied Statistics Workshop (Gov 3009) meets all academic year, Wednesdays, 12pm-1:30pm, in CGIS K354. This workshop is a forum for advanced graduate students, faculty, and visiting scholars to present and discuss methodological or empirical work in progress in an interdisciplinary setting. The workshop features a tour of Harvard's statistical innovations and applications with weekly stops in different fields and disciplines and includes occasional presentations by invited speakers. Free lunch is provided.
Isaiah Andrews (MIT) presents.
Title: "Identification of and Correction for Publication Bias"
(Joint with Maximilian Kasy)
Abstract: Not all empirical results are published, and the probability that a given result is published may depend on the result.
Such selective publication can lead to biased estimators and distorted inference. We discuss identification of selectivity, and in particular of the conditional probability of publication as a function of a study's results. We propose two approaches to identification, the first based on systematic replication studies, and the second based on meta-studies. Having identified the form of selectivity, we propose median-unbiased estimators and associated confidence sets. We apply our methods to recent large-scale replication studies in experimental economics and experimental psychology, where we find strong evidence of selection based on statistical significance. We also apply our methods to a meta-study of minimum wage effects, where we find larger publication probabilities for studies reporting a negative effect on employment, and to a meta-study of de-worming programs, where our findings are ambiguous.