Qualitative Research at the IQSS: Numbers and Narratives Combined

by Amanda Pearson

Qualitative research forms the backbone of our knowledge about human society. From crime and corruption, to race and culture, to violence and well-being, social scientists at IQSS seek to understand who we are, how we behave, and what we believe.

From providing technical support for scholars who want to conduct surveys, to helping researchers improve how they manage and share their data, to building cutting-edge software and tools, “the goal of IQSS is to transform social science research from the art of studying the greatest problems that affect human societies to the science of understanding and solving these problems,” wrote IQSS director Gary King, the Albert J. Weatherhead III University Professor at Harvard University, in his message to the IQSS community.

A recent award-winning book, Forbearance as Redistribution: The Politics of Informal Welfare in Latin America, from IQSS affiliate and associate professor of government Alisha Holland is a prime example of research that confronts pressing social science questions.

Expansive Research Leads to New Insights

Alisha Holland

Alisha Holland

In Forbearance, Holland examines why and when some Latin American governments tolerate legal violations by the poor. Rather than focusing on formal government programs to address the poor’s most basic needs, such as cash assistance or housing, Holland studies informal welfare policies such as squatting and street vending. She finds that when poor people violate the law, sometimes governments decide to look the other way, essentially achieving redistribution by withholding enforcement (or forbearance).

“Many scholars and policymakers assume that a lot of problems in developing countries arise from a lack of state capacity,” Holland said. “For instance, informal economies exist because there are too few police and bureaucrats, or those that exist are too corrupt or lazy to enforce the law. I use the term ‘forbearance’ to refer to intentional decisions not to enforce the law. The goal is to show that there is a complex politics behind law enforcement decisions.”

Forbearance—which won the Giovanni Sartori Book Award (APSA Qualitative and Mixed Methods Section), the Donna Lee Van Cott Best Book Prize (Latin American Studies Association), and the Herbert Jacob Book Prize (Law & Society Association)— grew out of Holland’s doctoral dissertation at Harvard University.

Her book originated from a term paper about welfare states in the developing world she wrote for a class taught by IQSS affiliates Jim Alt and Torben Iversen during her first year in graduate school. That paper wasn’t very good, she said. “I tried to write a formal model of forbearance. It’s quite different than where the project ended up but it shows how long [writing a book] can take.”

But from that humble starting point, Holland spent a lot of time thinking about how she could translate a concept like unemployment insurance to Latin America’s informal workers.

“Many unemployment programs in Latin America are small and only protect workers in the formal sector,” Holland said. “I realized that the key economic cushion for many Latin Americans was the ability to work in the informal sector. So that made me think more about how informality wasn’t really integrated into the ways scholars were thinking about welfare states.”

Holland spent about a year conducting research in three Latin American cities—Santiago (Chile), Lima (Perú), and Bogotá (Colombia)—where she interviewed bureaucrats and politicians to try to understand what enforcement looks like in practice and how politicians make enforcement decisions.

“Especially when studying informal and illegal behaviors, it’s hard to rely on any single data source so I tried to triangulate between different actors and records, blending together statistical data on public attitudes, enforcement actions, interviews and archival data,” Holland explained.

How IQSS Helps Scholars Store, Manage, and Curate Their Qualitative Data

“I like to write books because they have space for many different forms of evidence,” Holland said, including qualitative and quantitative research and analysis. Using multiple methods as Holland has in her book can help scholars understand a given topic or phenomenon more deeply when numbers or narratives alone do not provide a complete picture.

This is especially true for researchers like Holland who are studying topics like

informality, criminal violence, corruption, or civil wars that are hard to measure. “A lot of data sources contain bias in some form or measurement errors and that’s why I like to examine many different kinds of evidence to see the sum of the parts,” Holland said.

To assist other researchers and academics, Holland decided to deposit 69 files related to her book with the Qualitative Data Repository (QDR), a qualitative data repository intended to expand access to information for social science scholars.

QDR is one of 69 institutions across the globe that have installed the Dataverse software, developed at IQSS. Launched in 2014 and based at Syracuse University, QDR is the first U.S. domain repository dedicated to publishing research data arising from qualitative and multimethod social science inquiry. Originally built with digital asset management software Fedora and Islandora, QDR adopted the Dataverse software in 2017.

“QDR is the brainchild of Colin Elman,” professor of political science at Syracuse University and director of its Center for Qualitative and Multi-Method Inquiry, said

Diana Kapiszewski, the Provost's Distinguished Associate Professor in the Department of Government at Georgetown University and the associate director of research at QDR. She, Elman, and several other scholars won a large grant from the National Science Foundation to build QDR.

QDR’s mission is to make the sharing of qualitative data a customary practice in the social sciences. “Scholars produce an extraordinary amount of rich qualitative data,” Kapiszewski said. “And then they use it and throw it all in a box and store it in the garage. What a waste of potential knowledge and resources! We should have a repository that allows us to capture this information so we can build new knowledge.”

For data collection, let’s say you want to teach students how to do an interview. “You could download from QDR a bunch of interview transcripts that scholars conducted as part of their research projects and look at what the interviewers did,” Kapiszewski explained. “How did they sequence the questions? What’s the arc of this interview? There are so many things that students could learn about interviewing from interview transcripts.”

Holland agrees. “I think about what I didn’t learn in graduate school, like the art of conducting an interview and taking notes during an interview,” Holland said. “There are very few graduate courses that prepare you for doing fieldwork. I think it would be helpful for students to be able to see how scholars did interviews and organized their interview notes, much like students learn by playing around with a data file.”

There are some other types of qualitative data that are easier to share. For example, Holland combed through twenty years of newspapers articles and culled and coded articles on street vendors and squatters

“This is publicly available data and I’m just cleaning it to make it usable for those who might be interested in informal economies. The fact that I scanned them all and curated them could be useful for a potential researcher,” said Holland.

Holland hopes that the qualitative data she deposited with QDR could be a starting point for other scholars to build on, perhaps by comparing her data in Latin America to another region of the world.

“The idea of forbearance can apply to many different situations,” Holland said.

Political Scientists Debate Research Transparency

But questions still remain about what information should be curated and made freely available. In recent decades, political scientists have debated the benefits and risks of encouraging scholars to be more open and explicit about the bases of their empirical claims.

That battle originated in the open science movement, and includes attempts to write data access and research transparency standards (DA-RT) into the ethics guide of the American Political Science Association (APSA) and encourage the major journals in the field to make authors abide by those standards (Journal Editors Transparency Statement [JETS]).

Though there’s generally been a push to make information more freely available, these efforts haven’t been universally embraced. “There was huge pushback against these initiatives,” said Kapiszewski, “Journals were key to the underpinning of DA-RT because they had the power to refuse publication unless scholars shared their data or could explain why they couldn’t do so. Something like 1,200 qualitative scholars signed a letter saying ‘You’re not thinking this through.’”

Ethical Considerations of Sharing Data

One thing that political scientists agree on—they have ethical obligations to protect all human participants they study.

This guiding philosophy is inherent to the Henry A. Murray Research Archive, housed at IQSS, which was founded in 1976 and named for the Harvard psychologist known for developing personality theory. That archive now houses 350 datasets. Among its vast holdings are longitudinal studies in disciplines such as criminology and psychology.

Holland and other researchers like her can derive great benefit from access to well-curated data. Without that curation, data can be difficult to find, use, and interpret

Sonia Barbosa

Sonia Barbosa

“We get all the boxes, we clean it, we organize it, we make sure things match, like the codebooks, the numeric files, and we check the paper data inventory for every single box of data and every single subject with every single measure, whether it’s missing or not,” said Sonia Barbosa, Manager of Data Curation for the Harvard Dataverse Repository and Manager of the Murray Archive. “We put subject numbers on everything and store content in acid-free boxes, and then we digitize it,” to provide organized access to the material both online and on site.

To access the Murray Archive and the potentially sensitive data it contains, researchers must fill out an application.

“We reject a lot of applications to use the Murray data if we determine that the researcher does not have the experience to handle the sensitive content or take care of it responsibly,” Barbosa said.

Even with these stringent requirements, scholars might refrain from sharing some of their research to protect the people they’ve studied. “I didn’t share all my data files [with QDR],” Holland said. “I noted there were risks to the politicians who I interviewed in terms of admitting they were engaging in illegal behavior.”

Using Qualitative Data to Make Sense of the World

Despite the challenges and complexity in conducting social science research, its goal is “trying to understand and explain and make sense of the world,” said IQSS affiliate Jennifer Hochschild, the Henry LaBarre Jayne Professor of Government at Harvard University and Professor of African and African American Studies.

Jennifer Hochschild

Jennifer Hochschild

Some scholars attempt to take all this qualitative knowledge and systematize and structure it even further by engaging in causal quantitative analysis. But because ours is a qualitative existence, many human interactions are difficult to quantify.

“We live our own lives qualitatively. We talk to people, we think, we engage in narratives, we speculate, we use our hands to express ourselves,” Hochschild said.

“It’s the qualitative work that allows researchers to really understand what they are studying—knowing how to frame a question, understanding why you would use certain variables, knowing what the results might or might not mean,” Hochschild added.

In the end, “all social science research is qualitative, and some small (growing) fraction is also quantitative,” said IQSS director Gary King, the Albert J. Weatherhead III University Professor at Harvard University.

It’s the narratives behind the numbers that truly teach us about society.