Social Science Statistics Blog

check the paper by Altonji et al in the JPE that takes an interesting approach to this problem

Yes, very insightful to recognize that U really needs to be thought of as the linear composite of multiple potential confounders rather than a single hypothetical confounder. The substantive implication for this if a model does not explain a large portion of variance to begin with is that the subjective assessment of the possible effect sum of multiple potential confounders can be *much* larger than a single confounder, thereby invalidating judgments from sensitivity analyses that are based on speculation about a single hypothetical confounder.

> But what if U should be taken to stand not for
> a single but for a whole collection of
> unobserved confounders?

You can actually think of U as capturing a whole array of confounders and not only a single confounder. In fact, on can show that within the Rosenbaum sensitivity framework, the non-parametric Manksi bounds are an extreme case of hidden bias arising from U.

Jens

You are right, Jens, but that's not what I'm driving at. Rather, the problem I mentioned concerns gauging the critical associations of U in _substantive_ terms. In practice, most every inference falls apart if we are willing to allow for the existence of a sufficiently large unobserved U (hardly any no-assumption bound doesn't include zero). The question is: how much confounding is a lot of confounding? In other words, when does the size of a critical U put us at ease and when should it ring alarms? Rosenbaum and Rubin have suggested an interpretation that''s clearly very helpful, but it's also clearly not going to solve the issue for all applications.

Hi Felix,

yes this remains a problem I agree. But I think it will necessarily boil down to a judgment call; the magnitude of the hidden bias that turns over the findings has to be evaluated for each particular application. In particular it will matter on how many observed confounders one has conditioned already.

For example, consider the seminal Cornfield et al (1959) analysis that first assessed sensitivity to hidden bias necessary to invalidate the relationship between smoking and lung cancer. They found that hidden bias alone would have to be a nearly perfectly correlated with lung cancer and about 10 times more prevalent in smokers than non-smokers. That such a variable exists is highly unlikely (in particular from a genetic perspective). The burden of proof is then on the critics to find this magic omitted variable.

In other cases that may not be so clear of course. But I would not trust any findings that break down at low gamma values already (like 1.5 or 2) unless one has conditioned on many many confounders.

Best,
Jens