Unpacking the “Black Box” of Programs and Policies: A Conceptual Overview of Mediation Analysis

Publication Date: March 27, 2017
Current as of:

Download Report

Download PDF (466.51 KB)
  • File Size: 466.51 KB
  • Pages: N/A
  • Published: 2020

Introduction

Policymakers and practitioners have a growing interest in answering questions beyond simply “does a program work?” They are also interested in learning how programs work. Mediation analysis is one tool that researchers can use to identify elements of an intervention that do, or do not, lead to improved participant outcomes. Researchers can use the results of a mediation analysis to build knowledge to improve programs.

Purpose

This brief has four primary purposes:

  1. To describe what mediation analysis is, and provide an example.
  2. To explain how mediation analysis can fit into a variety of study designs.
  3. To describe the analytical tools available for conducting mediation analysis.
  4. To recommend best practices for designing and conducting mediation analysis.

Key Findings and Highlights

Researchers can use mediation analysis in the early stages of research to identify potential intervention targets, or in a program evaluation to improve understanding of the underlying mechanisms of an intervention.

There are different ways to conduct a mediation analysis. These different techniques can assess:

  • longitudinal mediation effects;
  • latent trajectories of change, and;
  • how a program may work differently for different populations.

Recent methodological advances, such as instrumental variable approaches and propensity scoring, may improve the challenge of addressing causal inference in mediation.

Recommendations

Keep the following best practices in mind when designing or conducting a mediation analysis:

  1. Carefully consider what should be measured and seek input from both methodologists and practitioners.
  2. Clearly describe the theory of how the program causes changes in participant outcomes.
  3. Develop an analysis plan that appropriately accounts for the data structure.
  4. Conduct a power analysis for the mediation analyses.
  5. Use an approach that estimates the sampling distribution of the mediated effect.

Citation

Morgan-Lopez, A., & Bir, A. (2017). Unpacking the “black box” of programs and policies: A conceptual overview of mediation analysis, OPRE Report #2017-01, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.