Linking Process Indicators to Outcomes in Evaluations of Home Visiting Programs

Publication Date: April 9, 2021
Cover image for Linking Process Indicators to Outcomes in Evaluations of Home Visiting Programs

Download Report

Download Report PDF (523.45 KB)
  • File Size: 523.45 KB
  • Pages: 12
  • Published: 2021

Introduction

Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program awardees may conduct well-designed evaluations to contribute knowledge to the field of home visiting and inform data-driven policy. MIECHV also supports research and evaluation activities that build knowledge around the implementation and effectiveness of home visiting services and that examine how programs can improve short-term and long-term outcomes for families.

Process indicators reflect the actions taken by program staff and the results of those actions. They can help identify the mechanisms of change or active ingredients for families—i.e., how the program achieves outcomes or why it does not.

Purpose

This evaluation brief defines measures of home visiting services called process indicators, describes how process indicators link to short- and long-term outcomes in home visiting evaluations, and provides an example illustrating the role of process indicators in evaluations.

Key Findings and Highlights

Identifying linkages between process indicators and outcomes enables programs to understand which program activities help all families or different types of families. Sharing evaluation findings allows other awardees to replicate and confirm the impact of process indicators on short- or long-term outcomes, verify common measures to include in future evaluations, and provide greater evidence for what works in home visiting. Key takeaways include the following:

  • Evaluations that link process indicators to outcomes have the potential to be useful tools for awardees assessing home visiting implementation. Demonstrating linkages between process indicators and outcomes can show whether an intervention is on track to achieve desired long-term outcomes.
  • When specifying process indicators, it is important to specify indicators that are supported by program theory and that connect activities and outputs to desired outcomes.
  • Evidence produced from evaluations linking process indicators and outcomes may provide support for the relationships identified or proposed in the program’s logic model and theory of change and demonstrate that program components can successfully achieve desired outcomes.
  • Evaluations that focus on the linkages between process indicators and outcomes may help awardees understand what works and for whom. This approach can be used in multiple evaluation designs, including studies to identify the active ingredients of a home visiting program, rapid cycle evaluations, and pay for outcomes research.

Citation

Bartko, T., Higman, S., & Thomson, A. (2021). Linking process indicators to outcomes in evaluations of home visiting programs. OPRE Report # 2021-54. Office of Planning, Research, and Evaluation, U.S. Department of Health and Human Services. Produced by James Bell Associates.

Glossary

Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program:
Administered by the Health Resources and Services Administration in partnership with the Administration for Children and Families, the MIECHV Program was established in 2010 to support voluntary, evidence-based home visiting for at-risk pregnant women and parents with children up to kindergarten entry. The program provides grants to states, US territories, and tribes, which conduct needs assessments to identify eligible at-risk communities and serve priority populations.
Active ingredients:
Process indicators can help to identify key program activities or “active ingredients” that lead to change. In some interventions, multiple active ingredients may be needed to produce changes in outcomes.
Rapid cycle evaluation (RCE):
RCE approaches can aid awardees in understanding what program activities work for whom by assessing the effectiveness of program components and providing timely and ongoing feedback to program staff to improve program effectiveness. For example, an RCE approach may be used to assess how a process indicator is differentially affecting outcomes in one subgroup may allow for a change in services without affecting the evaluation design.
Pay for outcomes (PFO):
PFO focuses on demonstrating success by using a payment model that promotes innovative financing for social initiatives and connects funding to outcomes. Awardees pursuing a PFO initiative may evaluate the linkages between process indicators and outcomes to demonstrate a quicker return on investment than measuring outcomes alone.