Introduction
Integrating qualitative and quantitative methods in a mixed methods approach enables researchers to answer questions that one approach alone cannot. For example, by using a sequential design to first conduct a survey of program participants and then supplement this data with interviews or focus groups, researchers can gain important context for the survey findings. In this case, qualitative findings can offer an in-depth understanding of an issue and answer a distinct set of questions.
Qualitative inquiry can also reveal gaps and help researchers uncover ideas to improve program design, delivery, outreach, or other important elements that may not be discovered through quantitative methods alone. Using qualitative approaches independently or mixing quantitative and qualitative methods can also provide opportunities for wider involvement from communities that are the focus of research. In turn, this process can enhance equity by helping uplift and center participants’ voices and perspectives, creating space to learn more about program participants’ motivations and experiences.
Purpose
On October 19—20, 2022, the Administration for Children and Families’ Office of Planning, Research, and Evaluation convened a virtual meeting for participants from federal agencies, research firms, academia, and other organizations to discuss qualitative and mixed methods approaches to social policy questions.
This summary document highlights key themes and presentations from the virtual meeting, which addressed the following questions:
-
What role do mixed and qualitative methods play in policymaking? What role could these methods play?
-
How can researchers and evaluators ensure rigor, credibility, and other considerations for high-quality mixed and qualitative methods in their work?
-
How can mixed and qualitative methods promote equity in research and evaluation by incorporating participant voices from a diversity of groups?
-
What are the potential benefits of using mixed and qualitative methods in research across various human services settings? What are the challenges to conducting this type of research?
-
What study designs are appropriate for the research questions that decision makers need answered? What are some examples of mixed methods approaches that could apply in this context?
-
How can researchers ensure that appropriate human subjects protections are applied when using qualitative methods?
-
How does the terminology used in qualitative approaches differ from that of quantitative approaches? How can these terminology differences be reconciled?
-
What are examples of mixed methods approaches that researchers and evaluators have successfully applied in the context of human services evaluation (i.e., how can researchers use qualitative data to inform the appropriateness or applicability of a quantitative measure for a given population)?
-
How can researchers and evaluators integrate qualitative and quantitative data in the context of evidence reviews?
-
How can researchers and evaluators effectively and accurately interpret and communicate the results of qualitative inquiry and mixed methods research?
-
What are the key considerations when implementing mixed methods designs in federally sponsored research?
Key Findings and Highlights
-
Several 2022 Methods Meeting speakers advocated for using the term “qualitative inquiry” rather than “qualitative research.” They explained that inquiry is a term that signals openness to questioning each step along the research path instead of following the more prescriptive step-by-step path associated with research.
-
While quantitative studies test hypotheses and use numbers to relate variables, qualitative inquiry explores general research ideas and analyzes perceptions, beliefs, and understanding of groups of people.
-
Qualitative inquiry can be exploratory, descriptive, and/or explanatory. Exploratory qualitative inquiry investigates a little-understood phenomenon and identifies important variables to generate hypotheses for future research. Descriptive qualitative inquiry documents the phenomenon of interest. Explanatory qualitative inquiry seeks to explain the forces causing the phenomenon in question and explore possible causal networks shaping that phenomenon.
-
Using inclusive and equitable approaches in a mixed methods study requires attention to race, ethnicity, culture, and power dynamics in every phase of the research cycle.
-
Speakers discussed data parties, a participatory technique to include diverse voices in reviewing and interpreting data before drafting a final report or plan of action. Data parties are time-limited events of several hours, where diverse audience members come together to collectively analyze collected data.
-
Benefits of integrating qualitative and quantitative research methods include initiating new understandings, building stronger connections, and gaining a more complete picture of underlying mechanisms. Challenges of integration can include the potential of sacrificing the rigor of individual methods and the lack of documented guidance for researchers on how to successfully integrate qualitative and quantitative methods.
Citation
Martin, Victoria, Hilary Wagner, and Debra Wright. 2023. “Applying Mixed Methods and Qualitative Approaches to Social Policy Questions: Summary of 2022 OPRE Methods Meeting” (OPRE Report 2023-162). Prepared by Insight Policy Research. U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research, and Evaluation.
Glossary
- Qualitative inquiry:
- An alternate term to “qualitative research” that signals openness to questioning each step along the research path instead of following the more prescriptive step-by-step path associated with research.
- Member-checking:
- A process through which findings are shared with participants who engaged in the data collection, enabling researchers to gather input from members of the community at the focus of the research on the themes and findings that emerged from data analysis.
- Arts-based research:
- A data collection methodology that systematically uses artistic processes to explore, understand, and represent the human experience.
- Exploratory qualitative inquiry:
- Investigates a little-understood phenomenon and identifies important variables to generate hypotheses for future research.
- Descriptive qualitative inquiry:
- Documents the phenomenon of interest.
- Explanatory qualitative inquiry:
- Seeks to explain the forces causing the phenomenon in question and explore possible causal networks shaping that phenomenon.
- Convergent parallel design:
- Seeks to compare and converge qualitative and quantitative results for a complete understanding. In this design, the timing is concurrent; qualitative and quantitative data collections are conducted simultaneously.
- Explanatory sequential design:
- Aims to explain quantitative results or develop a qualitative strand. In this design, timing is sequential: The quantitative data collection and analysis come first.
- Exploratory sequential design:
- Intends to generalize qualitative results or develop an instrument or intervention. In this design, qualitative data collection and analysis come first.