Social Science Statistics Blog

« Questions about Free Software | Main | The BLOG inference engine »

5 December 2005

Anchoring Vignettes (II)

Sebastian Bauhoff

In my last post I mentioned how differences in expectations and norms could affect self-rated responses in surveys. One fix is to use anchoring vignettes that let the interviewer control the context against which ratings are made.

For example, in a 2002 paper on the use of vignettes in health research, Salomon, Tandon and Murray ask respondents to rank their own difficulty in mobility on a scale from 'no difficulty' to 'extreme difficulty'. Then they let respondents apply the same scale to some hypothetical persons using descriptions like these:

"Paul is an active athlete who runs long distances of 20km twice a week and plays soccer with no problems."

"Mary has no problems walking, running or using her hands, arms, and legs. She jogs 4km twice a week."

Using the difference in how people assess these controlled scenarios, one can adjust the rating of people's own health. Doing this across or within various populations then allows to examine systematic differences across groups. These vignettes have been used in recent World Health Surveys in a number of countries.

King, Murray, Salomon and Tandon introduced the vignettes approach and used the measured differences to correct responses to self-rated questions on political efficacy. The idea is that applying the vignettes to a sub-sample is cheap and sufficient to understand systematic differences in self-reports. Their methods are laid out in the paper, but the results show how much difference the vignettes method can make: instead of suggesting that there is a higher level of political efficacy in China than in Mexico (as self-reports would indicate), the vignette method shows the exact opposite because the Chinese have lower standards for efficacy and thus understand the scale differently.

Intuitively that's what we do all the time: once you talked to enough Europeans and Americans about their (and other peoples') well-being you use your mental model to adjust responses and stop taking the European's minor complaints too seriously. Using this insight in survey-based research can make a huge difference too.

Posted by James Greiner at December 5, 2005 6:41 AM