Building Strong Evidence in Challenging Contexts: Alternatives to Traditional Randomized Controlled Trials

Publication Date: February 20, 2018
Building Strong Evidence in Challenging Contexts: Alternatives to Traditional Randomized Controlled Trials Cover

Download Report

Download Report PDF (205.29 KB)
  • File Size: 205.29 KB
  • Pages: N/A
  • Published: 2018

Introduction

Research Questions

  1. When is it appropriate to use alternatives to traditional RCTs?
  2. What are some rigorous alternatives to traditional RCTs for assessing program impacts?
  3. How have these alternatives been employed to strengthen the evidence base for social programs?

In the fall of 2016, OPRE brought together a diverse group of participants from federal agencies, research firms, foundations, and academia to discuss alternatives to randomized controlled trials and their assumptions, trade-offs, benefits, and challenges.

Purpose

Randomized controlled trials (RCTs) have long been considered the “gold standard” for evaluating program impacts. However, RCTs are not always appropriate or feasible. In some cases, an RCT design may not align with the program model, research question, or population of interest. In other cases, the necessary resources may not be available to effectively and efficiently carry out an RCT.

Key Findings and Highlights

This brief summarizes the meeting and includes a description of the meeting topic, an overview of each panel, and the meeting agenda.

Methods

Methods highlighted include: single case research designs, randomized roll out designs, interrupted time series designs, double randomized preference trials, kernel matching/optimization, regression discontinuity, instrumental variables, hierarchical Bayesian analysis, and value-added modeling.

Citation

Malin, J. and Deterding, N. (2017). Building Strong Evidence in Challenging Contexts: Alternatives to Traditional Randomized Control Trials, OPRE Report #2017-112, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Current as of: