I have not come across a more satisfactory statistical solution for circumstances in which SUTVA is violated. In an interesting new paper, Manski provides some bounds on treatment effects in the presence of social interactions. Unfortunately, these bounds are often uninformative, since when SUTVA is violated random assignment to treatment arms does not identify treatment effects. Sinclair suggests using multi-level experiments to empirically identify spillover effects. This approach (which relies on multiple rounds of randomization to test if treatment effects are overidentified, as we would expect if there were no spillovers) is appealing, as the process of diffusion within networks is of great scientific interest. However, it does not help identify treatment effects when spillovers are present. Neither can we simply assume that effects estimated under SUTVA represent upper bounds on the true effects, because it is possible that interference across units intensifies the treatment effects rather than diluting them. Manski's paper seems like a useful foray into an open area of research. Let me know of other work on methods for causal inference in network-like situations where interference across units is likely.

Here is a typical model:

There are two typical features of these models that seem relevant. First, most include a lagged dependent variable (LDV) to account for persistence in the responses. If I was going to vote for McCain the last time you called, I'll probably still want to do that this time. Makes sense. Second, we include a unit-specific effect, alpha, to account for all other relevant factors. Dynamic panel models tend to identify their effects with a simple differencing by running the following model:

Which eliminates the unit-specific effect by the differencing, but our parameters remain, ready to be estimated. I should note that there are some identification issues left to solve and the differences between estimators in this field mostly have to do with how to instrument for the differenced LDV.

Reading these models, I have two questions. One, is there a reason to expect that we need both a LDV and a unit-specific effect? This means that we expect that there is a shock to a unit's dependent variable that is constant across periods. I find this a strange assumption. I understand a unit-specific shock to the initial level and then using LDV thereafter, but in every period?

Two, the entire identification strategy here is based on the additivity of the model, correct? If we were to draw a directed acyclic graph of these models, it would be trivially obvious that we could never identify this model nonparametrically. I understand that we sometimes need to use models to identify effects, but should these identifications depend so heavily on the functional form? It seems that this problem is tied up in the first. We are allowing for the unit-specific effect as a way to free the model of unnecessary assumptions, yet this forces our hand into making different, perhaps stronger assumption to get identification.

Please clear up my confusion in the comments if you are more in the know.

Despite its shortcomings, cross-level or ecological inference remains a necessary part of many areas of quantitative inference, including in United States voting rights litigation. Ecological inference suffers from a lack of identification that, most agree, is best addressed by incorporating individual-level data into the model. In this paper, we test the limits of such an incorporation by attempting it in the context of drawing inferences about racial voting patterns using a combination of an exit poll and precinct-level ecological data; accurate information about racial voting patterns is needed to trigger voting rights laws that can determine the composition of United States legislative bodies. Specifically, we extend and study a hybrid model that addresses two-way tables of arbitrary dimension. We apply the hybrid model to an exit poll we administered in the City of Boston in 2008. Using the resulting data as well as simulation, we compare the performance of a pure ecological estimator, pure survey estimators using various sampling schemes, and our hybrid. We conclude that the hybrid estimator offers substantial benefits by enabling substantive inferences about voting patterns not practicably available without its use.

Both the paper and the technical appendix are on the course website.

The Applied Statistics workshop meets each Wednesday in room K-354, CGIS-Knafel (1737 Cambridge St). We start at 12 noon with a light lunch, with presentations beginning around 12:15 and we usually wrap up around 1:30 pm. We hope you can make it.

The function μ is referred to as the link function. Common choices for μ include both the identity and log link. One common question is why one would choose to use a GLM with, for example, a log link instead of estimating via OLS the regression model:

There are two principle objections to the OLS method. First, in the presence of heteroskedasticity it is difficult to transform predicted values of ln(y) into predicted values of y, although it is possible. Second, the OLS method throws out any data coming from observations with y=0.

Unfortunately, the choice of a link function is comparatively easy (in my view) compared with the next step of choosing an appropriate function for the variance of y given x, which must be prespecified in most GLMs*. In my work I have focused on choosing variance functions that are proportional to some power of the variance:

The trick, then, is to choose the correct power with various powers of the mean corresponding to Poisson (k=1), Gamma (k=2), and Wald (k=3), for example. In health econometrics this can be accomplished by using a modified Park test (due to Manning and Mullahy). In this procedure one first computes tentative parameter estimates for a GLM based on one's prior beliefs about the appropriate variance function (I typically use Gamma-like regressions for this). The linear predictors from the tentative regression can be used to get raw-scale residuals by applying the inverse link function. The modified Park test is to then regress the squared raw-scale residuals on a constant and the linear predictor in a GLM with a log link and the coefficient on the linear predictor then indicates which variance structure is most appropriate.

Now for the question. In health utilization data one often has data with a large number of zeros, for example, less than 10% of my sample uses mental health services in any given year. While GLMs are typically well behaved, in the presence of so many zeros this need not be the case. One common practice is to then use a "two part" model in which one uses an initial probit or logit regression to estimate the probability of any utilization and then estimate the second stage GLM model among users only. My question relates to the appropriate sample to use for the modified Park test--users or everybody? It turns that in this case it matters since when I look at everyone I get evidence in support of Gamma-like regressions (i.e. k=2 in my Park test), but when I only consider users in the Park test I get estimates of k=2.6, or so, which is more consistent with Wald-type variances.

My strong suspicion is that the latter approach is more appropriate since the GLM is only estimated among users, but I've hunted in the literature and found no specific advice on this point and many examples that seem to indicate that the test should be done on everybody.

* One exception is the Extended Estimating Equations method proposed by Basu and Rathouz (implemented as pglm in Stata).

It would make sense. Controversial bills require a lot more ink -- pork, special cases, exceptions -- to reel in support. Uncontroversial bills can be written succinctly and pass as is.

To assess this I scraped bills from OpenCongress, which maintains the full text, voting results and amendment history of House and Senate Resolutions. You can even comment on specific portions of bills. There's already a bunch of neat comments on potential loopholes in H.R. 3962.

I downloaded the text and voting results for all 152 House resolutions passed by the 111th House. A boxplot of page length against support appears below. Each page length group represents roughly 20% of House resolutions. The plot shows the suspected trend, that longer bills have less support. One-page bills almost always pass unanimously!

I think this formulation suggests that answering a "why" question requires both "causes of effects" and "effects of causes" approaches; we need to search over a number of possible causes to identify likely causes, but we also need to test the effectiveness of each likely cause before we can say much about the causal effect. We probably still can't answer questions like "what caused World War I" but maybe this gets us somewhere with more tractable types of "why" questions.

Unfortunately, when treatment effects are heterogeneous, the identified local average effect does not provide direct information about the wider population. This is problematic since treatment effects are likely to be heterogeneous in social science applications. In fact, this heterogeneity is one of the reasons why identifying causal effects is so difficult (individuals' self-selection into a treatment status based in part on anticipated treatment effects induces endogeneity problems).

A number of demographers have discussed the problem of extrapolating local average treatment effect estimates to the broader population. Greg Duncan, in his presidential address to the Population Association of America, stated that although causal inference is "often facilitated by eschewing full population representation in favor of an examination of an exceedingly small but strategically selected portion of a general population with the 'right kind' of variation in the key independent variable of interest.... a population-based understanding of causal effects should be our principal goal." Robert Moffitt writes that although "some type of implicit weighting is needed" to help us understand how to trade off internal and external validity, "this problem has not really been addressed in the applied research community." Some researchers have suggested using bounds for average treatment effects that are not point-identified (for example, Manski). Of course, the usefulness of bounding techniques depends on the tightness of the bounds, which in turn depends on what assumptions we are willing to impose - and it is exactly scholars' discomfort with prevailing assumptions (e.g., lack of correlation between the error and the treatment indicator) that drove the current focus on non-representative population subgroups. It seems to me that there is still work to be done to connect subpopulation causal estimates to broader population trends. I would be interested to hear of work in this area that you think is promising.

Networks are ubiquitous in science and have become a focal point for discussion in everyday life. Formal statistical models for the analysis of network data have emerged as a major topic of interest in diverse areas of science, as many scientific inquiries involve collections of measurements on pairs of objects. Probability models on graphs date back to 1959. Along with empirical studies in social psychology and sociology from the 1960s, these early works generated an active network community and a substantial literature in the 1970s. This effort moved into the statistical literature in the late 1970s and 1980s, and the past decade has seen a burgeoning network literature in statistical physics and computer science. The growth of the World Wide Web and the emergence of online networking communities such as Facebook and LinkedIn, and a host of more specialized professional network communities has intensified interest in the study of networks and network data. In this talk, I will review a few ideas that are central to this burgeoning literature. I will emphasize formal model descriptions, and pay special attention to the interpretation of parameters and their estimation. I will conclude by describing open problems and challenges for machine learning and statistics.

The setup is that there are many markets in which buyers and sellers are distinct types of actors, for example, the market for spouses has, until recently, been such a market (although I make no claim as to which side of the market is buying and which is selling). This market, in the form of college applications, was analyzed by Gale and Shapley in a famous 1962 paper in which they proved that there was a solution to this type of matching problem.

Unfortunately, this setup does not permit the applied researcher (or poor grad student) much traction for identifying the effect of being assigned to a particular residency program.

One solution comes from some work by Morten Sorensen on matching in venture capital. His idea is to model the decision process leading to investments by venture capitalists in early stage companies and at the same time to model his outcome of interest (company goes public) thus allowing for correlations between the respective error terms of the attractiveness / matching model and the outcome equation. Sorensen makes the point that this method makes use of the characteristics of other investors and investments in the market as instruments in order to address the fact that "better" investors may invest in "better" companies.

While in principle this method is attractive, it is computationally difficult and does not convince everyone--the faculty member I was talking to agreed that in principle this method is attractive, but that the results would be more credible with an instrumental variable that affects the probability of being assigned to a particular program. However, he also pointed out the value of these structural models--they provide estimates that may be valid over a broader range of values and can be used to do policy experiments that one may not be able to do with a model identified by instrumental variables.

Reading a few articles in the "physics of politics" as a political scientist, one has the sense of observing an alternate universe. For example: a paper on the effect of election results on party membership in Germany that has no references to work outside of physics; features many exotic (to me at least) terms like Wegscheider potentials, the Sznajd model, and the Kronecker symbol; and takes a time-series approach to causation that I suspect would be unacceptable to most reviewers in political science and economics these days.

In general, it's clear that physicists doing work on political phenomena (or "sociophysics" more generally) are primarily interested in exploring the individual-level social interactions that might underpin the macro-order we observe in, e.g., regularities in turnout or vote share distributions. As such, political institutions (which are the major preoccupation of political scientists) necessarily disappear from the model and are typically not even mentioned, even when they would seem to be of first-order importance in explaining a particular phenomenon. (Another example of the alternate universe: a paper that argues that party vote shares in Indonesia follow a power law, but which does not describe or mention the electoral system.) These omissions seem foolish on first reading, but it's clear that they reflect a different choice of explanatory variable: physicists seek their explanations in micro-interactions, and we seek them primarily in political institutions. It's probably both of course, but models can only be so complex.

Despite my overall sense of disorientation in reading these papers, there were also somewhat surprising moments of familiarity. Physics heavily influenced economics in an earlier period of colonization, and much of what we read in economics and political science descended from those models. In reading these newer physics papers, there is therefore a sense of distant kinship, the knowledge of a common ancestor several generations back.

I wonder about the scope for collaboration between physicists and social scientists. Based on my admittedly very cursory reading of one area in which physicists have ventured, it's hard to know whether the potential gains from trade are sufficient to overcome the apparent difference in goals. For all I know there already is a lot of productive collaboration going on -- if you know of something interesting, share it in the comments!

Identifying the sources of randomness underlying our data is important, because they have implications for our analysis. Särndal, Swensson, and Wretman show that the variance of a parameter from a ordinary regression model estimated using sample data can be decomposed into two elements, one based on the sampling design and one based on the model. In the case of a census, the extra variance introduced from the design is zero, and thus the total variance of the estimated parameter is the variance of the "BLUE" estimator. Otherwise, accounting for the sampling design in the analysis should improve inference.

I am probably overly skeptical and I am very sympathetic to heroic attempts to solve difficult problems of causal inference to answer important questions. Still, it seems that having multiple instruments can become an embarrassment of riches. A good instrument is so hard to come by that having too many starts to lend evidence against an empirical argument rather than for it.

While a copy on your computer is very handy, a desk copy of this book is essential if you are interested in machine learning or data mining. The book is also a sight to behold. You can buy a copy at Amazon or Springer.