Causal Validity Considerations for Including High Quality Non-Experimental Evidence in Systematic Reviews

October 25, 2018
Topics:
Methods and Tools
Projects:
Methods Inquiries, 2013-2021 | Learn more about this project
Types:
Reports
Causal Validity Considerations for Including High Quality Non-Experimental Evidence in Systematic Reviews Cover
Download report (pdf)
  • File Size 1mb
  • Pages 7
  • Published 2018

Introduction

Federally funded systematic reviews of research evidence play a central role in efforts to base policy decisions on evidence. Historically, evidence reviews have reserved the highest ratings of quality for studies that employ experimental designs, namely randomized control trials (RCTs). However, RCTs are not appropriate for evaluating all intervention programs. To develop an evidence base for those programs, evaluators may need to use non-experimental study designs.

Purpose

This brief describes the need for non-experimental study designs when a randomized control trial is not appropriate. It also summarizes important considerations for using non-experimental studies.

Key Findings and Highlights

This brief identifies key considerations for assessing the causal validity of non-experimental studies systematically and reliably:

  1. Defines causal validity,
  2. Provides examples of threats to causal validity and methods that can be used to address those threats,
  3. Discusses causal validity ratings in HomVEE and WWC, and
  4. Summarizes key considerations for developing standards to assess the causal validity of non-experimental designs.

Citation

Deke, John (2018). Causal validity considerations for including high quality non-experimental evidence in systematic reviews (OPRE Report 2018-63), Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Last Reviewed: October 24, 2018