Social Science Statistics Blog

« How to teach methods | Main | Automated Text Analysis of Political Scientists »

26 March 2009

Small samples across the country

A recent paper by Shetty, DeLeire, White, and Bhattacharya looked at the effect of workplace smoking bans across the country. This paper follows previous papers that looked at single smoking bans and attempted to identify the effect of smoking bans on health outcomes by comparing the area with the ban to a similar area without a ban. The particular contribution of this paper is that it compared all smoking bans across the country at once to conclude that smoking bans have considerably smaller effects on heart attacks and mortality than previous articles suggested.

The analysis used region (county) fixed effects to determine the effect of smoking bans within regions, so it is conceptually similar to differences-in-differences analyses that previous articles had used to address this same issue. The authors have taken pains to make sure that the results are not a result of modeling assumptions by including time-varying covariates in the model. This is where one of my pet peeves shows up since the assumption in fixed effects models is that there are no unobservable time-varying effects, but this assumption can be checked, not just by incorporating additional time-varying covariates, but also by including linear time trends that vary by region. The linear time trend provides a test of the assumption that there are no time trends that vary by region, independently of the included covariates.

Those are my peeves, one of the gems in this paper is that they took the time to simulate possible pairwise comparisons between regions (i.e. simulating the results of previous studies). These simulations indicate that one is just as likely to get the large health-improving effects seen in earlier studies as large health-detracting effects. This opens the possibility that the papers that were published came from the extreme of the distribution of pairwise outcomes.

Posted by Martin Andersen at March 26, 2009 9:25 PM