Building Strong Evidence in Challenging Contexts: Alternatives to Traditional Randomized Controlled Trials

February 20, 2018
Topics:
Abuse, Neglect, Adoption & Foster Care, Child Care, Early Head Start, Head Start, Home Visiting, Self-Sufficiency, Welfare & Employment, Strengthening Families, Healthy Marriage & Responsible Fatherhood
Projects:
Methods Inquiries, 2013-2021 | Learn more about this project
Types:
Reports
Building Strong Evidence in Challenging Contexts: Alternatives to Traditional Randomized Controlled Trials Cover
Download report (pdf)
  • File Size 205kb
  • Pages 3
  • Published 2018

Introduction

In the fall of 2016, OPRE brought together a diverse group of participants from federal agencies, research firms, foundations, and academia to discuss alternatives to randomized controlled trials and their assumptions, trade-offs, benefits, and challenges.

Research Questions

  1. 1 When is it appropriate to use alternatives to traditional RCTs?
  2. 2 What are some rigorous alternatives to traditional RCTs for assessing program impacts?
  3. 3 How have these alternatives been employed to strengthen the evidence base for social programs?

Purpose

Randomized controlled trials (RCTs) have long been considered the “gold standard” for evaluating program impacts. However, RCTs are not always appropriate or feasible. In some cases, an RCT design may not align with the program model, research question, or population of interest. In other cases, the necessary resources may not be available to effectively and efficiently carry out an RCT.

Key Findings and Highlights

This brief summarizes the meeting and includes a description of the meeting topic, an overview of each panel, and the meeting agenda.

Methods

Methods highlighted include: single case research designs, randomized roll out designs, interrupted time series designs, double randomized preference trials, kernel matching/optimization, regression discontinuity, instrumental variables, hierarchical Bayesian analysis, and value-added modeling.

Citation

Malin, J. and Deterding, N. (2017). Building Strong Evidence in Challenging Contexts: Alternatives to Traditional Randomized Control Trials, OPRE Report #2017-112, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Last Reviewed: December 18, 2018