Road Maps for Change: Youth-Serving HMRE Grant Recipients’ Rapid Cycle Learning in the SIMR Project

Publication Date: May 11, 2023
Cover page of Road Maps for Change: Youth-Serving HMRE Grant Recipients' Rapid Cycle Learning in the SIMR Project

Download Brief

Download PDF (1,007.00 KB)
  • File Size: 1,007.00 KB
  • Pages: 13
  • Published: 2023

Introduction

Healthy marriage and relationship education (HMRE) services are designed to help participants build and sustain strong families. HMRE services for youth between the ages of 14 and 24 focus on preparing participants for positive, healthy relationships in adulthood and educating them about the social and emotional aspects of relationships (Alamillo et al. 2021; Simpson et al. 2018). Studies have generally found positive impacts on short-term outcomes related to youths’ relationship attitudes and beliefs. To date, however, little evidence has emerged on the effects HMRE services for youth have on longer-term outcomes (Alamillo et al. 2021; Simpson et al. 2018). To achieve their intended effects, HMRE service providers might need support to address key implementation challenges related to recruitment, retention, and content engagement.

The Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF), with funding from the Office of Family Assistance (OFA), contracted with Mathematica and its partner, Public Strategies, to conduct the Strengthening the Implementation of Marriage and Relationship Programs (SIMR) project. This project aims to identify key implementation challenges facing HMRE grant recipients and, in close collaboration with HMRE grant recipients and their staff, develop and test strategies to address those challenges using rapid cycle learning techniques.  This brief summarizes lessons and insights from the project, focusing on the five youth-serving HMRE grant recipients that participated in SIMR.

Purpose

In the SIMR project, Mathematica and its partner, Public Strategies, collaborated with 10 HMRE grant recipients—five youth-serving grant recipients and five adult-serving grant recipients—to conduct iterative rapid cycle testing aimed at strengthening their services. SIMR focused on common implementation challenges related to recruitment, retention, and content engagement.

SIMR had two main goals:

  1. to improve the service delivery of these grant recipients
  2. to develop lessons for the broader HMRE field about promising practices for addressing common implementation challenges

This brief presents five infographics—one for each of the youth-serving grant recipients who participated in SIMR—called “Road Maps for Change.” These infographics present the strategies that each grant recipient focused on in SIMR, how they changed over the course of rapid cycle learning, and lessons and insights for the field.  

Key Findings and Highlights

In SIMR, each grant recipient developed and tested improvement strategies tailored to their specific needs, service populations, and individual contexts, using an approach to program improvement and rapid cycle learning known as Learn, Innovate, Improve (LI2). Through their work with the SIMR team, grant recipients:

  • Addressed pressing implementation challenges: One grant recipient focused on improving recruitment, while four others focused on topics related to improving content engagement.
  • Increased their capacity to collect and use data to inform decision-making: Through rapid cycle learning, grant recipient staff administered feedback surveys to participants, tracked recruitment data, and analyzed social media analytics. They then reviewed these data with the SIMR team and developed insights to refine their improvement strategies.
  • Developed skills for identifying and responding to emerging implementation challenges: At the end of each learning cycle, the SIMR team met with grant recipients to review and interpret data and determine next steps. When new challenges emerged, grant recipients were able to pivot to address them in later learning cycles.
  • Developed tools and strategies to support strong implementation through the rest of the grant period: Grant recipients developed promising tools and strategies to support facilitators, enhance case management, recruit youth from rural areas, and encourage peer and staff-participant relationships. At the end of SIMR, the grant recipients planned to continue using these tools and strategies.

Methods

Grant recipients in SIMR tested strategies using a rapid cycle learning approach. Rapid cycle learning is a method for quickly and iteratively testing strategies to strengthen programming. It often involves successive cycles—referred to as learning cycles—to pilot strategies, collect feedback from staff and participants on how these strategies are working, and gather data to demonstrate whether the strategies are supporting improvement. Based on what grant recipients learn, staff can refine and test strategies again in a subsequent learning cycle. The length of a learning cycle is dependent upon the strategy being tested and the program setting.

HMRE grant recipients and other human services providers may be familiar with continuous quality improvement (CQI). Programs can adopt rapid cycle learning as part of their CQI process to test programmatic changes. CQI is often used to support internal improvements; whereas, rapid cycle learning can be used either for internal improvement or in partnership with researchers to build evidence for the broader field.

Recommendations

Through their collaboration as part of SIMR rapid cycle learning, the SIMR team and the five youth-serving grant recipients that participated generated insights and lessons to inform strong service delivery that are relevant to other HMRE grant recipients. The tools and strategies that grant recipients developed provide starting points for other organizations that want to strengthen their own HMRE services. Findings from SIMR suggest that youth-serving grant recipients could consider strategies that:

  • Provide supports and tools for facilitators to successfully lead HMRE workshops, such as strategies to manage sources of stress and co-facilitate lessons.
  • Look for innovative ways to reinforce workshop content, such as by engaging participants with curriculum content on social media and helping youth set and make progress on goals through case management.
  • Prioritize relationship-building to engage participants, by establishing a safe and supportive classroom environment and using technology in intentional, innovative ways.

Citation

Baumgartner, S., L. Mattox, D. Friend, M. Ezzo, and A. Jordan. “Road Maps for Change: Youth-Serving HMRE Grant Recipients’ Rapid Cycle Learning in the SIMR Project.” OPRE Report #2023-066. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2023.

Glossary

HMRE:
Healthy Marriage and Relationship Education