In a recent paper entitled, "Scaling the Critics: Uncovering the Latent Dimensions of Movie Criticism with an Item Response Approach," Arthur Spirling (IQSS, Harvard University) and Michael Peress (University of Rochester) study the way that movie critics--like Roger Ebert  and Owen Gleiberman--make their decisions.  They introduce a new econometric modeling strategy that allows for each critic to vary not just in the types of movies they enjoy, but also how sensitive critics are to the general quality of the films they review.  

The data used by Spirling and Peress consist of reviews drawn from approximately fifty expert film critics and 1000 movies, using the web site Rotten Tomatoes and the National Society of Film Critics as sources.  According to Spirling, "We show that after quality, there are basically three dimensions to film reviewing: critics care about how dramatic or 'art-house' a movie is, versus it being mostly for sci-fi ‘nerds’ or for comedy and action-loving ‘jocks.'"  Spirling goes on to say that, "What’s remarkable is that these three dimensions explain almost all the variation between individuals’ enjoyments of the films they watch."  In terms of the critics themselves, the paper notes that some are easier to please than others: Roger Ebert emerges as the "most generous" reviewer, while it turns out to be "hard work to impress Amy Taubin, who writes columns for The Village Voice."

The authors also show that the elite critics they study seem to be very in-tune with certain cultural crowds: according to Spirling, "When the critics approve of an indie movie, it tends to do well at the box office."   
 
Spirling and Peress’s framework can be applied in a number of situations where individuals choose to approve or disapprove of an item, including a consumer’s decision to use a certain product, the college admissions process, and policy decisions made by legislators. A version of their paper will appear shortly in the Journal of the American Statistical Association.

Please click here to see the full text of the paper.