Methods Inquiries, 2013-2021

Project Overview

OPRE plays a central role in advancing understanding and disseminating knowledge about research and evaluation methods and tools that are, or could be, used to enhance knowledge about program and policy effectiveness. The purpose of the Methods Inquiries project is to organize meetings that bring together expertise from varying disciplines and policy fields and from academia, government, and the private sector to explore innovations in research design, analytic techniques, and data measurement that could be employed to advance the government’s use of rigorous research methods. These meetings ensure that OPRE‐supported research continues to represent the most scientifically advanced approaches to determining effectiveness and efficiency of ACF programs.

The contractor from 2013 to 2018 is RTI International. The contractor from 2016 to 2021 is Insight Policy Research, in partnership with Applied Engineering Management Corporation.

Visit the OPRE Innovative Methods Meetings website (www.opremethodsmeeting.org) to access the resources and materials from the meetings.

The point of contact is Anna Solmeyer.

  • Causal Validity Considerations for Including High Quality Non-Experimental Evidence in Systematic Reviews

    Published: October 25, 2018

    Federally funded systematic reviews of research evidence play a central role in efforts to base policy decisions on evidence. Historically, evidence reviews have reserved the highest ratings of quality for studies that employ experimental designs, namely randomized control trials (RCTs). However, RCTs are not appropriate for evaluating all intervention programs. To develop an evidence base for those programs, evaluators may need to use non-experimental study designs.

  • Understanding Bayesian Statistics: Frequently Asked Questions and Recommended Resources

    Published: July 3, 2018

    There is a growing understanding that there are some inherent limitations in using p-values to guide decisions about programs and policies. Bayesian methods are emerging as the primary alternative to p-values and offer a number of advantages...

  • Bayesian Methods for Social Policy Research and Evaluation

    Published: July 3, 2018

    Probability (p) values are widely used in social science research and evaluation to guide decisions on program and policy changes. However, they have some inherent limitations, sometimes leading to misuse, misinterpretation, or misinformed decisions. Bayesian methods...

  • Evidence and Equity: Challenges for Research Design

    Published: February 20, 2018

    There is growing emphasis placed on evidence-based interventions, and opportunities to make programmatic decisions based on evidence reflect progress in promoting positive outcomes. However, some populations (e.g., ethnic and cultural minority communities, marginalized groups) may be left behind in efforts to build evidence, if they are more difficult to study. Over time, as evidence builds for the populations...

  • Building Strong Evidence in Challenging Contexts: Alternatives to Traditional Randomized Controlled Trials

    Published: February 20, 2018

    In the fall of 2016, OPRE brought together a diverse group of participants from federal agencies, research firms, foundations, and academia to discuss alternatives to randomized controlled trials and their assumptions, trade-offs, benefits, and challenges.

  • Unpacking the “Black Box” of Programs and Policies: A Conceptual Overview of Mediation Analysis

    Published: March 27, 2017

    Policymakers and practitioners have a growing interest in answering questions beyond simply “does a program work?” They are also interested in learning how programs work. Mediation analysis is one tool that researchers can use to identify elements of an intervention that do, or do not, lead to improved participant outcomes. Researchers can use the results of a mediation analysis to build knowledge to improve programs...

  • Using Aggregate Administrative Data in Social Policy Research

    Published: March 27, 2017

    Policymakers are increasingly interested in using administrative data to address policy-relevant research questions. While researchers generally prefer individual-level administrative data in order to provide maximum flexibility to their analyses, it can be both difficult and costly to obtain. When individual-level data are not available or are too difficult or costly, aggregate administrative data can address many policy-relevant research questions...

  • Issues in Accessing and Using Administrative Data

    Published: March 27, 2017

    Policymakers are increasingly interested in using administrative data to address policy-relevant research questions. In order to make use of administrative data for social policy research, there are multiple issues to consider...

  • Using Administrative Data in Social Policy Research

    Published: August 29, 2016

    Administrative data have the potential to help us answer pressing social policy questions. Government stakeholders and researchers are exploring the promises of using administrative data for research purposes.

    This brief summarizes an Innovative Methods Meeting that was organized by OPRE in the fall of 2015 that considered the potential benefits and pitfalls of using administrative data for research purposes...

  • The Promises and Challenges of Administrative Data in Social Policy Research: Roundtable Discussion

    Published: June 30, 2016

    In this video roundtable, government experts and experienced researchers discuss the opportunities and challenges presented when using administrative data for social policy research. Topics include: tips for planning administrative data research; working with (federal and state) data custodians; negotiating data...

More Reports on this Project