Social Science Statistics Blog

« February 9, 2009 | Main | February 16, 2009 »

15 February 2009

How many people could improved COBRA cover?

One feature of the recently-passed stimulus legislation is a temporary federal subsidy program for individuals to purchase transitional health insurance coverage in the event they lose their job. This coverage -- called COBRA after the legislation that created the program in the 1980's -- is generally available to displaced workers in firms with more than 20 employees, although some states have also adopted policies that allow employees of smaller firms to buy into transitional coverage. The catch, however, is that workers must pay the full premium amount plus 2 percent for administrative expenses.

The new federal program allows for subsidies of up to 65 percent of the cost of health insurance, which is aimed to provide a boost to individuals trying to make ends meet and remain insured while they're unemployed. But the question is, how many people could this program potentially cover in a given year?

Below I've created a population flow diagram using R's diagram package. The underlying data are drawn from the 2006 Medical Expenditure Panel Survey (MEPS) -- a nationally-representative survey of American households conducted each year by the Agency for Health Care Research and Quality. To construct the diagram I took the health insurance coverage status of non-elderly adults and children in January 2006, and compared this to their coverage status in December of that year.

Clearly, the potential for reducing the number of uninsured is large -- according to these estimates, 6.4 million adults and 1.4 million children lost employer-based group insurance between January and December 2006.* If just one-half of these individuals took up a subsidy and were able to continue that group coverage, the number of uninsured in 2006 could have been reduced by about 4 million. Moreover, this is almost certainly an underestimate, since there are also individuals who lost employer-based coverage prior to January, as well as individuals who were intermittently uninsured during the year, who may not show up in the diagrammed flows but could also have benefited from access to subsidized transitional coverage. Finally, I would also note the potential for spillover and cost-offsetting effects within Medicaid, as the estimated 1.2 million who went from group coverage to public coverage could have also retained their employer-based insurance, saving states and the federal government the costs of these extra Medicaid/SCHIP enrollees.

* Note, however, that I have made no attempt to produce confidence intervals around these figures, though in principle this would be straightforward to do with R's survey package. If anyone knows how to easily input these into the figures (I couldn't figure out a way), please let me know!

Posted by John Graves at 4:51 PM

Bayesian Propensity Score Matching

Many people have realized that conventional propensity score matching (PSM) method does not take into account the uncertainties of estimating propensity scores. In other words, for each observation, PSM assumes that there is only one fixed propensity score. In contrast, Bayesian methods can generate a sample of propensity scores for any observation, by either monitoring the posterior distributions of the estimated propensity scores directly or predicting propensity scores from the posterior samples of the parameters of the propensity score model.

Then matching on thus obtained propensity scores, we should expect to get a distribution of estimated treatment effects. This will also provide us with an estimation of the standard error of the treatment effect. The Bayesian S.E. will be larger than the S.E. based on PSM estimate, as it takes into account more uncertainties. This conjecture is indeed confirmed by a recent paper written by Lawrence C. McCandless, Paul Gustafson and Peter C. Austin, "Bayesian propensity score analysis for observational data", which appears in Statistics in Medicine (2009; 28:94-112). The authors show that, the Bayesian 95% credible interval for the treatment effect is 10% wider than conventional propensity score C.I.

It seems that we should expect Bayesian propensity score matching (BPSM) perform better than PSM in cases where there are a lot of uncertainties in estimating the propensity scores. Before running into any simulations, however, the question is: what are the sources of the uncertainties in estimating propensity scores? From my point of view, there is at least one source of uncertainties, the uncertainties due to omitted variables. I do not think BPSM can do any better than PSM in solving this issue. But maybe, BPSM can model the error terms and so provide better estimations of the propensity scores? The above authors argue that when the association between treatment and covariates is weak (i.e., when the betas are smaller), the uncertainties in estimating propensity scores are higher. Weak association means smaller R-square or larger AIC, etc. Is this equivalent to larger bias due to omitted variables?

Another type of uncertainty related to BPSM, but not to propensity scores, is the uncertainty due to matching procedure. This is avoidable or negligible. Radically, we can just abandon the matching method and resort to linear regression model to predict the outcomes. Or we can neglect the bias from matching procedure, because when we only care about ATT and there is sufficient number of control cases, the bias is negligible, according to Abadie and Imbens 2006. ("Large Sample Properties of Matching Estimators for Average Treatment Effects." Econometrica 74 (1): 235 - 267.)

Of course, the logit model for the propensity scores could be wrong as well. But this can be manipulated in the simulations. Now my question is: how should we do the simulations to evaluate the performance of BPSM vs. that of conventional PSM?

Posted by Weihua An at 12:06 AM