Social Science Statistics Blog

« The BLOG inference engine | Main | Applied Statistics - Michael Hiscox and Nicholas Smyth »

7 December 2005

Fun with R²

Mike Kellermann

This semester, I have been one of the TFs for Gov 2000 (the introductory statistics course for Ph.D. students in the Government Department). It the first time that I've been on the teaching staff for a course, and it has been quite an experience so far. We've spent the past month or so introducing the basic linear model. Along the way, Ryan Moore (the other TF) and I have had some fun sharing the best quotes that we've come across about everyone's favorite regression output, R²:

Nothing in the CR model requires that R² be high. Hence a high R² is not evidence in favor of the model, and a low R² is not evidence against it. Nevertheless, in empirical research reports, one often reads statements to the effect that "I have a high R², so my theory is good," or "My R² is higher than yours, so my theory is better than yours." (Arthur Goldberger, A Course in Econometrics, 1991)

Thus R² measures directly neither causal strength nor goodness of fit. It is instead a Mulligan Stew composed of each of them plus the variance of the independent variable. Its use is best restricted to description of the shape of the point cloud with causal strength measured by the slopes and goodness of fit captured by the standard error of the regression. (Chris Achen, Interpreting and Using Regression, 1982)

Q: But do you really want me to stop using R²? After all, my R² is higher than all of my friends and higher than those in all the articles in the last issue of APSR!
A: If your goal is to get a big R², then your goal is not the same as that for which regression analysis was designed. The purpose of regression analysis and all of parametric statistical analyses is to estimate interesting population parameters....
If the goal is just to get a big R², then even though that is unlikely to be relevant to any political science research question, here is some "advice": Include independent variables that are very similar to the dependent variable. The "best" choice is the dependent variable; your R² will be 1.0. (Gary King, "How not to lie with statistics,"AJPS, 1986).

So this is old news, right? Maybe not. Quite possibly the thing that has surprised me the most so far is just how much students want R² to tell them how good their model is. You could almost see the anguish in their faces as we read these quotes to them, particularly among those who have taken some statistics in the past. The question I want to throw out is, why is R² such an attractive number? Why do we want to believe it? Maybe our cognitive science colleagues have some insight....

Posted by Mike Kellermann at December 7, 2005 5:00 AM