May 2008
Sun Mon Tue Wed Thu Fri Sat
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

Authors' Committee

Chair:

Andy Eggers (Gov)

Members:

Weihua An (Soc)
Kevin Bartz (Stats)
Sebastian Bauhoff (HealthPol)
John Graves (HealthPol)
Justin Grimmer (Gov)
Jens Hainmueller (Gov)
Mike Kellermann (Gov)
Ellie Powell (Gov)
Gary King (Gov)

Weekly Research Workshop Sponsors

Alberto Abadie, Lee Fleming, Adam Glynn, Guido Imbens, Gary King, Kevin Quinn, Jamie Robins, Don Rubin, Chris Winship

Recent Comments

Recent Entries

Categories

Blogroll

Brad DeLong
Cognitive Daily
Complexity & Social Networks
Developing Intelligence
EconLog
The Education Wonks
Empirical Legal Studies
Free Exchange
Freakonomics
Health Care Economist
Junk Charts
Language Log
Law & Econ Prof Blog
Machine Learning (Theory)
Marginal Revolution
Mixing Memory
Mystery Pollster
New Economist
Political Arithmetik
Political Science Methods
Pure Pedantry
Science & Law Blog
Simon Jackman
Social Science++
Statistical modeling, causal inference, and social science

Archives

Notification

Powered by
Movable Type 3.34


« More on statistics and the death penalty | Main | Duflo on 'Improving School Quality' »

21 November 2007

A case for the case-control method

In his opinion piece in this weekend's New York Times, Henry Louis Gates presents a taste of the research behind his forthcoming book, In Search of Our Roots:

I have been studying the family trees of 20 successful African-Americans, people in fields ranging from entertainment and sports (Oprah Winfrey, the track star Jackie Joyner-Kersee) to space travel and medicine (the astronaut Mae Jemison and Ben Carson, a pediatric neurosurgeon). And I’ve seen an astonishing pattern: 15 of the 20 descend from at least one line of former slaves who managed to obtain property by 1920 — a time when only 25 percent of all African-American families owned property.

The question is, how astonishing is the pattern Gates points out?

Whether we should be impressed that 15 of 20 successful African-Americans had landowning ancestors depends a lot on what we assume about patterns of intermarriage among landowning and non-landowning African-Americans. Let's consider two extreme possibilities. First, assume that landowners never married non-landowners. In that case, one's grandparents would have been either all landowners or all non-landowners (more precisely, all from landowning families or all from non-landowning families). Given that 25% of all African-American families were landowners in 1920, 25% of African Americans of Oprah's generation would have had landowning grandparents (assuming that landowners and non-landowners had equally sized families, and also assuming that African-Americans had only African-American grandparents). The fact that 75% of Gates's successful African-Americans had landowning grandparents would indeed be remarkable, supporting his claim that having landowning ancestors helped them succeed.

At the other extreme, assume that landowners intermarried perfectly freely with non-landowners. In that case, there is an almost 70% chance that a randomly-selected person of Oprah's generation would have at least one landowning grandparent. (To see this: given that 3/4 of all grandparents were not landowners, the probability that a randomly selected person would have no landowning grandparents is (3/4)^4 = 31.6%.) If that were true, we would be quite likely to see as many as 15 people with landowning ancestors in Gates's sample even if landowning had nothing to do with success (p-value = .19). The pattern he observes would thus provide only very weak evidence for his argument about the importance of landowning.

Gates's book appears typical of a genre of case study that focuses on remarkable people (or companies, or countries) in order to determine how they became that way. The problem with these studies (as pointed out in KKV, among many other places) is that they assume too much about the characteristics of unremarkable people (or companies, or countries). In the above example, Gates implicitly assumes that far fewer than 75% of unsuccessful African-Americans had a landowning ancestor in 1920. Instead of relying on this (possibly erroneous) assumption, he could have explicitly compared the family histories of his sample of remarkable African-Americans with those of another sample of unremarkable African-Americans. (This research design is known as "case-control" in epidemiology.) I doubt it would help with book sales to include a few chapters about thoroughly unfamous people, but it would make his arguments more convincing.

Posted by Andy Eggers at November 21, 2007 12:40 AM

Comments

Notification

Enter e-mail address to receive notification of new comments to this entry

Post a comment




Remember Me?

(you may use HTML tags for style)