How should we evaluate survey quality?  What are the tradeoffs that must be considered? Which quality metrics should be reported in published research? What are the challenges and opportunities for improving survey quality?

It is widely recognized that survey research faces methodological challenges which have created general skepticism about survey research as a method among many practitioners and consumers.   At the same time, new technologies have dramatically lowered costs for some types of survey methods (especially opt-in web surveys), resulting in a proliferation of new data.  Unfortunately, a lack of methodological transparency and gaps between the way that survey methodologists understand survey quality and the way survey data are often reported limit the use of many of these new surveys both for the study of survey methodology and substantive inquiry. This conference is designed to bring together experts in survey methodology and important practitioners from the media, government, and major academic surveys to discuss our current understanding of survey quality, the challenges presented in designing and using survey data in light of rapidly emerging technologies, and the opportunities that increasing variation in survey methods may provide for better understanding the nature of survey data and social phenomena.

This important conference will include experts from the academic, government, commercial, and media worlds to discuss this important topic.

Survey Quality 

An Eric C. Mindich Conference

April 17, 2009

Tsai Auditorium (S0101, 1730 Cambridge Street) 

Tentative Schedule

8:45–9:00 Continental Breakfast 

9:00–9:15 Introduction and Welcome (Sunshine Hillygus)

9:15 – 10:30 Overview of Survey Quality 

“Quality Assurance and Quality Control in Surveys” (Lars Lyberg; Statistics Sweden) 

“Adaptive Survey Design Using Bespoke Paradata” (Robert Groves; University of Michigan)

10:30 – 10:45 Coffee Break 

10:45-12:00 Survey Quality Issues in Context

“Developing and Implementing Standards for Surveys within the Federal Statistical Service” (Brian Harris-Kojetin; OMB)

 “Total Survey Error in Comparative Perspective” (Tom W. Smith; NORC) 

12:00-12:45 Lunch
12:45- 2:45 Measuring Quality

“Comparing Various Measures of Survey Accuracy and Summarizing the Findings of Studies Using Each Method” (Jon Krosnick; Stanford University) 

 “Evaluating the Relationship Between Measurement Error Bias and Nonresponse Bias with Implications for Nonresponse Follow-up Operations” (Paul Biemer; University of North Carolina)

 “Ask More” (Steve Ansolabehere; Harvard University)
2:45-3:00 Coffee Break

3:00-5:00 Transparency and Reporting  

“Survey Reporting Standards” (Gary Langer; ABC News) 

 “Transparency and Pre-Election Polls” (Michael Traugott; University of Michigan)

 “Preserving Survey Quality in a Time of Poll Proliferation” (Kathy Frankovic; CBS News) 

  "The Rise and Fall of Truth in Polling" (Phil Meyer; University of North Carolina) 

 5:00-6:00 Reception

Address Information:
Center for Government and International Studies
1737 Cambridge St.
Tsai Auditorium
Cambridge, MA 02138

Sponsored by: Eric C. Mindich Conference on Experimental Social Science