Iterating to Improve: Lessons from Rapid Cycle Learning with Healthy Marriage and Relationship Education Grant Recipients
- Introduction
- What are HMRE Services?
- What is SIMR?
- Meet the grant recipients
- Tip 1: Orient your team toward improvement
- Tip 2: Find and focus on your team’s most important goal
- Tip 3: Make space for innovation
- Tip 4: Collect and use data to inform improvements
- Tip 5: Overcome the fear of change
- Beyond SIMR
- Resources and references
Introduction

Healthy marriage and relationship education (HMRE) services are designed to help participants build and sustain strong families. However, HMRE practitioners often face common implementation challenges that prevent them from delivering services as intended. These challenges include enrolling enough participants, motivating participants to attend workshops and supplemental services, and developing engaging content so participants learn and incorporate healthy relationship skills into their lives.
When faced with these complex challenges, getting unstuck can feel difficult. Rapid cycle learning, or small-scale, iterative testing, is a method that can help practitioners to improve. In the Strengthening the Implementation of Marriage and Relationship Programs (SIMR) project, 10 HMRE grant recipients partnered with researchers and technical assistance staff to test and refine improvement strategies using a rapid cycle learning approach. The grant recipients represented a range of youth and adult service providers, serving various populations, located in rural and urban areas, and ranging from smaller community organizations to programs within large universities. Read more about the grant recipients on the Meet the Grant Recipients tab.
This digital essay presents the experiences and lessons learned of the 10 grant recipients through narrative and video. Key lessons learned are summarized below and detailed across “Tips” tabs in this essay.
One grant recipient’s SIMR journey
Montefiore Medical Center’s Supporting Healthy Relationships (SHR) program has served couples in the Bronx for nearly 20 years. SHR provides evidence-based HMRE services. When the COVID-19 pandemic began, Montefiore faced one of its toughest challenges yet, as their experienced staff struggled to lead engaging virtual workshops. On SIMR, staff at Montefiore partnered with a team of researchers and technical assistance staff to tackle their key implementation challenges using rapid cycle learning. Montefiore’s team developed and tested innovative, participant-centered improvements to their virtual services.
Click below to hear staff from Montefiore describe their challenges and opportunities to improve.
Staff from Montefiore Medical Center discuss how they partnered with SIMR to improve virtual HMRE services
What is rapid cycle learning?
Rapid cycle learning is a process for programs to develop and test targeted improvement strategies to address programmatic challenges, with the goals of strengthening their own services and generating lessons for the field. In the SIMR project, HMRE grant recipients spent time upfront prioritizing a challenge and identifying its root causes, before developing and testing improvement strategies. See the What is SIMR? tab for more information on the approach used in SIMR. HMRE grant recipients and other human services providers may be familiar with continuous quality improvement (CQI). CQI can employ short, iterative rapid learning cycles to test improvement strategies, such as those that the grant recipients in SIMR used.
Key Lessons Learned
This digital essay features lessons learned from 10 grant recipients that participated in rapid cycle learning in SIMR. Key lessons about engaging in rapid cycle learning included:
- Orient your team toward improvement, as a program improvement mindset helps providers identify their challenges and areas where they could enhance services
- Find a common goal to motivate the team
- Make space for innovation to bring staff into the improvement process
- Collect data and feedback to assess strategies and make evidence-informed decisions
- Overcome the fear of change to move from planning to action
Though SIMR grant recipients engaged in rapid cycle learning in partnership with researchers, their lessons learned are practices they intend to sustain on their own. Other providers can implement these tips themselves to help improve their services in meaningful ways.
Navigating the digital essay
Read, watch, and listen to this digital essay to learn from HMRE grant recipients about their experience with rapid cycle learning and their advice on how you can begin your journey toward program improvement. The essay contains video interviews in both English and Spanish from nine of the participating grant recipients. All videos include English captions. To turn off these captions or switch them to Spanish, please use the "cc" button at the bottom of each video.
- For background on SIMR: Click on the next three tabs to the left to learn more about HMRE services, SIMR, and the grant recipients who participated in the project.
- For lessons learned about rapid cycle learning: Explore the tabs labeled “Tips” to hear from grant recipients about several key ingredients for improving their services using rapid cycle learning, refined through their participation in SIMR.
- For additional SIMR reports and resources: More information about SIMR project findings can be found on ACF's SIMR project site page.


What are HMRE Services?
Healthy Marriage and Relationship Education (HMRE) services aim to help participants form and maintain healthy relationships. The federal government is making a long-standing investment to provide these services across the United States. HMRE grant recipients, funded by the Office of Family Assistance (OFA) within the Administration for Children and Families (ACF), Department of Health and Human Services have been serving youth, adults, and families since 2005. This digital report features videos from staff at several HMRE grant recipients discussing the need for HMRE services for youth and adults (for more information please visit the grant program page).
Click below to hear SIMR grant recipients speak about the importance of HMRE services for youth and adults.
Marlin Young from More Than Conquerors, Inc., discusses why HMRE services are meaningful for high school students
Staff from Montefiore Medical Center, Anthem Strong Families, and The RIDGE Project discuss the importance of HMRE services for adults
Explore the sections below to learn more about HMRE services for adults and for youth.
HMRE services for adults
HMRE services for adults typically serve two populations: (1) adult couples and (2) individual adults. Some HMRE services are designed for a couple’s joint participation. Other services are designed for individual adults to participate in regardless of their relationship status.
What do adult HMRE participants learn? Providers teach skills such as effective communication and conflict management, how to recognize signs of healthy and unhealthy relationships, and strategies to deepen connection and intimacy.1,2,3 Typically, HMRE services include a core educational workshop, along with supplemental services like case management and parenting workshops. Some providers also integrate job and career advancement activities to promote families’ economic stability. See ACF’s Pathways to Outcomes report for more information on adult HMRE services.
HMRE services for youth
Young people’s early relationship experiences can establish a foundation for the success of their future relationships.4 HMRE services developed for youth aim to prepare participants between the ages of 14 and 24 for positive relationships in adulthood and educate them about the social and emotional aspects of relationships.5,6 Providers focus on topics such as the signs of healthy and unhealthy relationships, teen dating violence, and effective communication and conflict management skills.7,8 Youth-serving HMRE services are most often delivered in high schools, but services are also offered in after-school or community-based settings.9,10 See this ACF brief for more information on youth HMRE services.


What is SIMR?
Strengthening the Implementation of Marriage and Relationship Programs (SIMR)’s two main goals were (1) to improve service delivery of the 10 participating grant recipients, using rapid cycle learning methods to test strategies for addressing common implementation challenges; and (2) to develop lessons for the broader HMRE field about promising practices.
SIMR grant recipients tackled three broad implementation challenges:

Recruiting participants to enroll in services

Motivating participants to attend primary HMRE workshops and supplemental services (like case management)

Developing engaging content so participants understand and absorb key takeaways.
Addressing these challenges is vital to ensuring that HMRE providers serve participants effectively.
Want to learn more about promising strategies HMRE providers could adapt and test in their context? See Strategies for Addressing Common Implementation Challenges in Healthy Marriage and Relationship Education Programs.
Learn more about the LI2 framework that guided SIMR
Learn, Innovate, Improve (LI2) is a program improvement framework that incorporates rapid cycle learning as well as important upfront steps to set up learning cycles, including understanding the problem and developing innovative strategies to address it.11 The LI2 framework guided the work in SIMR. LI2 can be used to guide continuous quality improvement efforts or used within research partnerships to build evidence for the field. In SIMR, researchers and practitioners used LI2 to partner to unpack challenges and identify barriers to addressing those challenges (Learn); create innovative program improvement strategies that are participant-centered, informed by evidence, and sustainable (Innovate); and use rapid cycle learning methods to test and refine strategies (Improve). Throughout each learning cycle, the SIMR team and grant recipients met to discuss progress, identify challenges and barriers, monitor data, and fine tune the strategy as needed.
Click below to learn more about LI2 and the importance of having an improvement process.
Scott Baumgartner, a senior researcher at Mathematica, explains the phases of LI2 and Phillippia Faust, a senior adviser at More Than Conquerors, Inc., discusses the importance of integrating an improvement process into program operations
Learn more about each phase in the LI2 framework below.
In the Learn phase, grant recipients worked to identify and specify challenges; engage their staff in identifying root causes contributing to the specific implementation challenge they were addressing; and understand barriers to addressing the challenge. The Learn phase helps teams to get on the same page about the motivation for change and provides a foundation for the Innovate phase, during which grant recipients identified improvement strategies.
In the Innovate phase, grant recipients drew on the multiple perspectives of their staff and the research team to develop strategies to address key root causes identified during the Learn phase. Teams gathered input and improvement ideas from staff by using human-centered design activities (see Tip 3: Make space for innovation). In addition, staff participated in trainings on research-informed best practices and models, which grant recipients then operationalized to fit their unique context and meet the needs of their service population.
In the Improve phase, the SIMR team and grant recipient staff collaborated to iteratively test and refine strategies using rapid cycle learning. Grant recipients and the SIMR team used several sources of data to carry out the strategy and assess its success. Data sources included:
- Quantitative data
- Surveys
- Observations
- Administrative data from nFORM, OFA’s management information used by HMRE grant recipients, and the grant recipients’ own internal data systems
- Qualitative data
- Interviews
- Focus groups
- Human-centered design activities
- Regular debriefs with staff


Meet the grant recipients
The ten HMRE grant recipients that participated in SIMR serve a range of populations and operate in a mix of urban and rural locations across the country. Their experiences reflect a range of implementation challenges that are relevant to the broader HMRE field.
Locations of HMRE grant recipients in SIMR
Click on the videos below to hear adult and youth providers share their thoughts about HMRE services.
Staff from The RIDGE Project and Anthem Strong Families discuss unique aspects of their HMRE services for adults
FRAMEWorks grant recipients serving adult couples or individuals
Anthem Strong Families (Anthem)

- Location: Dallas, Texas
- Service population: Women regardless of their parental or relationship status and men without children or romantic partners; focuses on those with low incomes; serves English and Spanish speakers
- Strategies developed and tested: Develop, enhance, and maintain recruitment partnerships and create and refine new Instagram posts to recruit participants
Family Service Agency of Santa Barbara County (FSA-SB)

- Location: Santa Barbara, California
- Service population: Primarily Spanish-speaking couples and individuals
- Strategies developed and tested: Develop, strengthen, and maintain recruitment partnerships; enhance direct outreach and marketing strategies; and adopt a motivation-driven coaching approach with staff and participants
Gateway Community Services (Gateway)

- Location: West Liberty, Kentucky
- Service population: Families and adult individuals in rural Kentucky
- Strategies developed and tested: Develop, strengthen, and maintain recruitment partnerships and enhance virtual facilitation, including a toolkit with virtual resources to promote participant engagement during classes
Montefiore Medical Center (Montefiore)

- Location: Bronx, New York
- Service population: Primarily Latino and Black couples who have low-incomes and are parenting or expecting a child
- Strategies developed and tested: Develop engaging virtual services by training staff on virtual facilitation skills and technology best practices; pilot strategies to enhance the virtual workshop experience, including a Zoom technology tip sheet, a streamlined virtual curriculum, and additional virtual skill coaching sessions
The RIDGE Project (RIDGE)

- Location: McClure, Ohio
- Service population: Men and women who are incarcerated and close to reentry
- Strategies developed and tested: Use motivational interviewing to enhance case management and participant—staff interactions
Staff from More Than Conquerors, Inc., and University of Denver’s MotherWise program share about the skills youth gain in their HMRE workshops
Ready4LIFE grant recipients serving youth (ages 14 to 24)
Auburn University Youth Relationship Education Program (Auburn)

- Location: Auburn, Alabama
- Service population: Youth in high schools across Alabama
- Strategies developed and tested: Support HMRE facilitators in identifying and managing sources of stress using a strategy called Take Note, Tag It, Tune It (T3)
More Than Conquerors, Inc. (MTCI)

- Location: Conyers, Georgia
- Service population: Youth in high schools in suburban Atlanta
- Strategies developed and tested: Conduct an enhanced case management approach with youth in high schools, drawing on motivation-driven goal setting to support engagement with the core curriculum
Texas A&M AgriLife Extension (Texas A&M AgriLife)

- Location: College Station, Texas
- Service population: Youth ages 18 to 24 in the Brazos Valley of Texas
- Strategies developed and tested: Recruit youth in rural areas by using Instagram to increase awareness in the community and engage potential participants; conduct county-specific recruitment efforts, including direct recruiting and seeking out new partners
University of Denver’s MotherWise program

- Location: Denver, Colorado
- Service population: Pregnant and parenting young women ages 14 to 24, most of whom are Hispanic
- Strategies developed and tested: Use technology to build community and engage participants in virtual services, including a strategy to encourage camera use among participants; a social media platform called the Nest to build community among participants; and prerecorded videos to supplement live delivery
Youth & Family Services (YFS)

- Location: Rapid City, South Dakota
- Service population: High-school-age youth in a variety of settings, both in and out of school
- Strategies developed and tested: Use an enhanced co-facilitation approach involving planning and debriefing strategies and learn about and use co-regulation strategies to foster warm relationship and a welcoming environment
Strategies grant recipients tested in SIMR
"It has been, I think, eye-opening for me the progress we’ve made—just in a few short months, having someone who’s actually dedicated to going out and recruiting for us and even finding new partnerships and making new connections."
-Laurie Naumann, project director, Texas A&M AgriLife
Grant recipients implemented and tested a range of different strategies to address challenges. Grant recipients focused on common recruitment challenges and tested various best practices related to referral partner development, direct outreach, print marketing, and social media. Among their strategies were bringing on dedicated recruiters and building out social media recruitment strategies. Grant recipients focused on testing retention and engagement strategies that use motivation-based practices and enhance virtual services to foster greater engagement with participants.
Click below to hear staff from University of Denver’s MotherWise program and Montefiore share practices they tested to enhance virtual services. Then, watch a Public Strategies trainer and a facilitator from YFS describing a classroom teacher expectation-setting tool they used to support partnerships between teachers and facilitators in the classroom. For more info on the specific strategies used by each grant recipient, see the findings reports from SIMR for FRAMEWorks grant recipients and READY4Life grant recipients.
Staff from University of Denver's MotherWise and Montefiore describe strategies they tested to translate strong in-person practices to a virtual environment
Note: The Nest is an internal social media platform that allowed participants to communicate and build community in a virtual environment.
Want to learn more about what SIMR grant recipients learned to strengthen virtual practices? See Tips for Delivering Engaging Virtual Workshop Sessions in Healthy Marriage and Relationship Education Programs.
Scott Roby, a master trainer at Public Strategies, and Ben Fichter, a YFS facilitator, share about a classroom teacher expectation-setting strategy
Want to learn more about YFS’s co-facilitation approach? See Strategies to Support Co-Facilitation in Classroom Sessions.
Tip 1: Orient your team toward improvement
Navigating the digital essay
Explore the “Tip” tabs, where grant recipients share their thinking on what they found to be the key ingredients of success for addressing their implementation challenges using rapid cycle learning.
No matter how experienced or established they are, grant recipients will inevitably run into challenges. Rapid cycle learning is a process that can support service providers in working through challenges. Getting the most out of the process, however, may require grant recipients to rethink or enhance their existing operations.
“We didn’t want to say, ’Well, that worked great for 5 years, let’s do it again.”
-Cynthia MacDuff, senior grants manager, FSA-SB
An improvement mindset means always looking for opportunities to enhance services, as even strong providers can get better. Staff participating in SIMR spoke about their desire to tackle lingering challenges, such as recruiting Spanish-speaking males or improving outreach in rural communities. Others shared a desire to continue delivering high-quality services under new circumstances—for instance, being funded to serve a new population or translating in-person services to a virtual environment in response to the COVID-19 pandemic. Adopting a mindset of continuous improvement means reexamining practices that might have gone stale and demonstrating an openness to trying something different.
Click below to hear staff emphasize the importance of questioning why things are done the way they are.
Staff at Family Service Agency of Santa Barbara and University of Denver’s MotherWise program emphasize the need to continually examine practices to maintain quality services
“We were underestimating how some of these things can help to support the very human connection that we need to make.”-Traci Maynigo, project director, Montefiore
An improvement mindset helped grant recipients develop solutions by propelling them to be more open to new or different ideas. Grant recipients shared that changing a long-standing practice can feel unnerving but is worth the risk. More Than Conquerors, Inc., and FSA-SB, for example, enhanced their case management approach by shifting from a more traditional assessment and referral process to a motivation-based, goal-attainment approach.
Want to learn more about more about this strategy? See Adopting a Motivation-Driven Approach to Case Management in HMRE Services.
Not every change needs to be transformational. Being open to change also means not overlooking small tweaks. Rather than waiting for a single perfect solution, service providers may find more success making incremental changes to strengthen processes over time. Many teams in SIMR tested small tweaks to address challenges—for instance, refinements to pitch emails or a Zoom guide to help participants navigate virtual workshops.
Want to learn more about what SIMR grant recipients learned to strengthen virtual practices? See Tips for Delivering Engaging Virtual Workshop Sessions in Healthy Marriage and Relationship Education Programs.
Click below to hear staff speak to the importance of being open to new ways of working, highlighting the need for shifting mindsets to innovate, be open to change, and appreciate the value of making small changes.
Staff at MTCI, FSA-SB, Texas A&M AgriLife, and YFS discuss being open to trying a new approach
Note: MTCI and FSA-SB both tested motivation-driven practices as strategies, encouraging participants to identify and work on their own goals.
Staff at MotherWise and Montefiore share how small refinements to practices made a big difference
Note: Montefiore tested a range of strategies to enhance virtual facilitation, including a Zoom guide to help participants navigate the technology.
Tip 2: Find and focus on your team’s most important goal
Grant recipients started their work in SIMR by identifying an implementation challenge and a goal related to the challenge, developed strategies to address the challenge, and then tested and refined strategies to support improvement. For instance, Auburn and YFS identified challenges related to content engagement and chose to focus on supporting facilitators to deliver engaging lessons. Several teams identified challenges with recruiting specific populations— for example, rural youth or Spanish-speaking men—and strove to enhance recruitment processes to boost enrollment of those populations.
Want to learn more about what SIMR grant recipients learned to strengthen their recruitment efforts? See Tips for Leveraging Partnerships to Improve Recruitment for Healthy Marriage and Relationship Education Programs and Four Tips to Recruit Spanish-Speaking Men into Healthy Marriage and Relationship Education Programs.
Setting Goals
Identifying a clear and motivating goal at the start of an improvement effort is an important step for maintaining disciplined focus across staff and seeing it through to the end. Goal setting within an organization can motivate staff and encourage them to become active participants in change.12
Narrowing the focus of improvement to just the most important goal, can help staff commit to supporting improvement while still having the capacity to attend to daily work.13
To develop a motivating goal, SIMR grant recipients involved a range of staff and used structured activities to ensure the staff had input in identifying a key challenge and determining the goal. Having a common goal, staff said, gave teams a motivating purpose to come back to during parts of the rapid cycle learning process that felt less clear. For example, digging into challenges and brainstorming strategies involve “diverging before converging” -in other words, putting everything on the table before narrowing the focus to a specific challenge or solution.14
Click below to hear staff speak about how having a clearly defined and shared goal provides a compass for navigating a successful rapid cycle learning process. For example, members of grant recipient teams share how having a clear objective helped support collaborative and disciplined brainstorming.
Staff at MTCI, University of Denver’s MotherWise program, Texas A&M AgriLife, and FSA-SB share how purpose-driven improvement efforts support collaboration and motivation to persist through barriers
A true co-creative process must recognize the value of diverse forms of expertise, particularly from those who are affected by the changes being implemented, such as grant recipient staff and participants.15 Grant recipients in SIMR saw value in bringing diverse voices into the process of developing strategies that support progress toward goals. These diverse perspectives included a high-level, strategic view from leadership; an evidence-informed perspective from the research team; on-the-ground insights from frontline staff; and personal experiences from participants and partners. Drawing on multiple perspectives, teams developed evidence-informed strategies that proved feasible to implement within the service provider's context and hold promise for supporting improvement.
“It was something that we built together, and everyone collaborated. In the beginning there were no bad ideas, all the ideas were welcomed.”
-Yerson Torres, facilitator, Montefiore
Click below to hear leaders from YFS and Texas A&M AgriLife emphasize the value of multiple perspectives when developing solutions.
Leaders at YFS and Texas A&M AgriLife share examples of what collaborative strategy development looked like within their teams
Tip 3: Make space for innovation
“Getting everything done in a day is important, but also, having time in a day to think about, ’Am I doing this the best way possible?’ is important, too.”
-Mary Minges, clinical coordinator, Montefiore
Keeping up with the day-to-day tasks of running an HMRE service provider can feel all-consuming. With everything else on your agency's collective plate, how can staff fit in conversations on how to do things better? While understanding the competing pressures on time, grant recipients found that a key takeaway from engaging in a rapid cycle learning process was a deeper appreciation for “team time”—that is, spending time to be intentional in their plans for program improvement.
For grant recipients in SIMR, prioritizing improvement meant allocating dedicated and structured time to focus on challenges, develop solutions, and monitor progress. This was a key lesson learned that staff intended to retain after their involvement with SIMR ended.
Who was on SIMR teams?
- Project leadership
- Facilitators
- Case managers/family advocates
- Partners
- SIMR team researchers/TA staff
Most grant recipients held regular program improvement meetings as part of SIMR. They found that carving out space to discuss improvement efforts supported greater intentionality in responding to challenges and determining next steps. Regularly scheduled meetings also helped to foster a sense of staff accountability to make progress toward goals and this helped teams avoid getting distracted by other priorities that arose.
Want to learn more about strategies for supporting staff in HMRE service providers? See Four Tips for Supporting Staff Providing Healthy Relationship and Marriage Education Services.
Click below to listen next to the voices of staff from four teams share the value of having dedicated time devoted to their improvement efforts.
Staff at Montefiore, FSA-SB, University of Denver’s MotherWise program, and Gateway share the value of making time for reflection
Staff shared that an important part of what made “team time” so valuable in SIMR was the opportunity it provided to hear from all staff, allowing teams to draw on multiple perspectives to shape improvement decisions. Creating space where all staff felt comfortable contributing ideas required project leaders to ensure staff saw themselves as “experts of the implementation team,” according to one director. Grant recipients also valued hearing from all voices through structured human-centered design activities.
More on human-centered design activities
Human-centered design empowers a range of people, such as staff, partners, and participants, to get involved in innovation. Because of this, it often yields strategies that are more feasible for staff to implement and more effective.
Several human-centered design activities are useful for rapid cycle learning. For instance, staff at Montefiore used Round Robin, a strategy development activity where team members build upon each other’s ideas to make them a little better with each iteration. According to one project director, the activity allowed everyone on the team to bring their unique perspectives to collectively develop ideas for what the grant recipient could do differently.
Read more about human-centered design.
Click below to hear what other leadership and staff involved in SIMR shared about what it takes to empower staff and how frontline staff members felt about participating in planning conversations.
Staff at FSA-SB, Montefiore, University of Denver’s MotherWise, and Texas A&M AgriLife share what it takes to empower staff to participate in improvement conversations
Want to learn more about skill coaching? The above video references Love Bites, which was a new approach to skills coaching that Montefiore tested and refined through SIMR. To learn more, see Tips for Providing Skill Coaching to Reinforce Workshop Content in Healthy Marriage and Relationship Education Programs.
Tip 4: Collect and use data to inform improvements
Gathering data on how strategies are working helps practitioners know how to refine and improve their approach.16 Grant recipients that participated in SIMR observed the value of gathering and using feedback from frontline staff, participants, and partner organizations. In addition to this qualitative information, SIMR grant recipients found that collecting quantitative data to monitor their progress helped them continuously refine their improvement strategies.
Monitoring overall success of improvement efforts
To assess the overall success of their improvement efforts, grant recipients set overarching goals and many used data from nFORM to monitor progress. nFORM, or the Information, Family Outcomes, Reporting, and Management system , is a data system used by Healthy Marriage and Responsible Fatherhood grant recipients to collect, store, and analyze data.
Monitoring success over the course of the project helped grant recipients to determine whether their efforts appeared to be supporting success. For example, recruitment teams monitored whether their strategies appeared to support increased enrollments of key populations by examining baseline numbers and tracking improvement over time.
Grant recipients shared the value in collecting new data—data outside existing administrative data systems—to learn about their challenges and identify strategies for improvement. They developed processes to track data that allowed them to obtain real-time feedback on their strategies, enabling them to learn what was working and what needed to be refined. For example, grant recipients working to establish new partnerships with organizations in their community used a partner tracking sheet to monitor relevant data, such as outreach attempts, in order to assess their progress. Staff shared that this tracker helped them be more strategic with partner outreach.
Want to learn more about using data to guide rapid cycle learning? See Using Data to Guide Program Improvement in Healthy Marriage and Relationship Education Services.
Click to listen to staff from Texas A&M AgriLife share how tracking data to learn about improvement strategies—in their case, social media analytics to assess an Instagram strategy—helped inform the team’s recruitment outreach approach. Staff from University of Denver’s MotherWise program also share how social media analytics helped the team be strategic in how they promoted The Nest, a phone app they used to connect with participants.
Staff at Texas A&M AgriLife and University of Denver’s MotherWise program talk about the value of data
SIMR grant recipients appreciated that rapid cycle learning included opportunities to gather feedback from participants and partners. Grant recipients used various methods to hear from those involved with their services, such as interviews, focus groups, surveys, and human-centered design activities. Participants and partners shared useful feedback on how grant recipients could improve their services and refine their strategies.
For example, Anthem held a focus group with younger participants in HMRE services to test a new Instagram recruitment campaign. The goal of the focus group was to get youth's perspectives on how they use social media, what posts they commonly interact with, and how to improve the images and captions of their draft posts. The team refined their social media plan and posts based on what they learned from participants. Other grant recipients used surveys with partners or exit tickets from youth—short one-to-two-question surveys administered at the end of a workshop—to gather feedback.
“I honestly don’t think [the feedback youth shared] is something I would have been able to pick up on my own… without the exit ticket [survey].”
-Emma Spies, facilitator, YFS
Click below to hear staff from three grant recipients share what they learned from gathering feedback from participants and partners.
Staff at YFS, Montefiore, and Gateway share key insights from participants and partners
Tip 5: Overcome the fear of change
Rapid cycle learning emphasizes starting small and reframing failure as a learning opportunity; grant recipients shared that this framing gave them permission to be innovative and creative and pushed them to try out new strategies. Staff shared that getting too comfortable with existing practices or fearing to disrupt a previously successful strategy often caused them to play it safe and avoid risks.
Grant recipients appreciated how rapid cycle learning helped them cultivate a mindset of “failing forward,” where missing the mark with an initial strategy can provide new insights on how to solve an existing problem. Using a rapid cycle learning approach propelled grant recipients to try some of the ideas they had discussed in the past but never launched. For example, prior to SIMR, Texas A&M AgriLife discussed expanding their use of social media to try to recruit more participants into their HMRE services. Starting small using a rapid cycle learning process gave them the push to get started and launch an even broader recruitment revamp effort than had been previously planned.
“It can be hard not to over analyze. And I think that’s the beauty behind rapid cycle, is that it says, ’Hey, don’t get into the paralysis of analysis. Take some time, put your thought into it, but then let’s get something activated.'”
-Chris Liotta, outreach specialist, Texas A&M AgriLife
Click below to hear staff at Texas A&M AgriLife, who tested variety of strategies to improve recruitment in rural counties, discuss how rapid cycle learning helped them overcome planning paralysis.
Staff at Texas A&M AgriLife share how rapid cycle learning helped them move from planning to action
Rapid cycle learning can help staff move from a mindset whereby a strategy either works or does not work to a mindset of how to make it work through small refinements. Grant recipients shared that rapid cycle learning helped them identify specific ways to adjust and tweak their strategies to inform improvements. For example, staff at FSA-SB shared that if certain strategies didn’t work in the past, they typically would abandon the approach and seek out other strategies. Rapid cycle learning gave them a structure and process for making incremental changes to existing strategies to strengthen them.
“Through SIMR, we learned to adjust”
-Cynthia MacDuff, senior grants manager, FSA-SB.
Click below to hear staff speak to the value of adjusting to improve.
Staff at FSA-SB, MTCI, and University of Denver’s MotherWise program share an appreciation for refining strategies over time
Beyond SIMR
Using rapid cycle learning won’t end for the grant recipients who participated in SIMR with the end of the SIMR project. Rapid cycle learning can be adopted as part of a CQI process that happens outside of a research partnership. Teams planned to retain key elements from their work in SIMR. For example, many grant recipients in SIMR developed teams of staff to guide implementation of improvement strategies during the project and these teams planned to continue meeting post-SIMR. Teams also expressed an appreciation for and desire to continue using data to inform their improvement work. Other service providers could use the experience of the grants recipients that participated in SIMR to inform their own efforts to improve their services through rapid cycle learning.
Click below to listen to what leaders of three grant recipients say about their plans to keep improving their services through rapid cycle learning going forward and hear from a senior project manager on the SIMR team about what grant recipients overall may take from the experience.
Leaders at Texas A&M AgriLife, FSA-SB, and MTCI discuss sustaining an improvement process
Lauren Mattox, a senior project manager at Public Strategies, explains how SIMR gave grant recipients a model for addressing future challenges
Using rapid cycle learning offers an approach for other providers to try out and adapt strategies tested in SIMR in their contexts. By drawing on tips for engaging with rapid cycle learning through testing strategies, providers can continue to learn about how to address key challenges related to recruitment, retention, and engagement.
Click below to listen to Daniel Friend’s call to action to tailor strategies in SIMR and test them using rapid cycle learning and, later, rigorous evaluations.
Daniel Friend, a senior researcher at Mathematica, talks about what’s next for the field
Mathematica and its partner Public Strategies conducted the SIMR project under a federal contact overseen by ACF’s Office of Planning, Research, and Evaluation with funding from ACF’s Office of Family Assistance. For more details on the SIMR project design, grant recipients’ early implementation challenges, and strategies tested in SIMR, see ACF's SIMR project site page.
Resources and references
Glossary of terms
- Continuous quality improvement (CQI): CQI includes long-term and ongoing processes for programs to identify performance challenges, develop changes to address those challenges, and monitor the success of those changes over time. Programs can adopt rapid cycle learning as part of their CQI process to test programmatic changes. CQI is often used to support internal improvements.
- Healthy marriage and relationship education (HMRE): A federal government grant program serving youth, adults, and families. HMRE grant recipients are funded by the Office of Family Assistance (OFA) within the Administration for Children and Families (ACF).
- Learn, Innovate, Improve (LI2): A framework that guided the work in SIMR. LI2 is an evidence-driven program improvement approach that providers can use to guide internal CQI efforts to improve programs or to build evidence for the field through research partnerships.
- Rapid cycle learning: A method for quickly testing strategies to strengthen programs. It often involves successive cycles to pilot strategies, where teams collect feedback on how they’re working as well as data to demonstrate whether they’re supporting improvement. Next steps focus on refining strategies and testing them again based on what is learned. Rapid cycle learning can be used for internal improvement or in partnership with researchers to build evidence for the broader field.
- Strengthening the Implementation of Marriage and Relationship Education Programs (SIMR): A project funded by ACF. SIMR’s two main goals were to improve service delivery of the 10 participating grant recipients by testing strategies to address common implementation challenges, and to develop lessons for the broader HMRE field about promising practices.
Acknowledgements
Developing this report was a collaborative effort. Many people worked on the SIMR project and contributed to the success of this report. First, we thank the SIMR project teams from Mathematica and Public Strategies (PS). In addition to the digital essay authors, the following team members led the technical assistance and rapid cycle learning with each site: Delara Aharpour, Alex Bauer, Sheila Cavallo, Mark Ezzo, Camila Fernandez, Aly Frei (PS), Daisy Gonzalez, Nick Gunzelman (PS), Avery Hennigar, Azaliah Israel (PS), Alisha Jordan (PS), Mollie Lovera (PS), Lauren Mattox (PS), Hannah McInerney, Adilia McManus (PS), Caroline O’Callahan, Grace Guerrero Ramirez, Scott Roby (PS), Shaun Stevenson, Veronica Murphy Sotelo, and Armando Yanez.
We also recognize the following staff members for their contributions creating the videos and report content. Sheila Cavallo, Mark Ezzo, Caroline O’Callahan, Katelyn Olsen, and Veronica Murphy Sotelo served on the digital report video review team. Pamela Holcomb served as the report’s quality assurance reviewer. Robert Wood is the SIMR project director and Quinn Moore is the SIMR principal investigator. Tara Merry and George Hoffman led the web development effort. Alejandra Hernandez, Grazia Maroso Mieren, Yvonne Marki and Laura Sarnoski created the report’s graphics and Donna Verdier edited the text. Rich Clement shot the videos and photos displayed throughout this digital report. Jaime Carreon and Adam Cole also provided photography.
We thank OPRE and OFA for guiding the SIMR project. We are deeply grateful for the guidance and feedback from OPRE Project Officers Samantha Illangasekare, Rebecca Hjelm, and Shirley Adelstein and Project Monitor Meghan Heffron. We also thank the OFA federal program specialists who provided insight and feedback throughout SIMR and participated in meetings with grant recipients and the SIMR team: John Allen, Millicent Crawford, Tanya Howell, Toya Joyner, Jackie Proctor, and Barbara Spoor.
We also thank the grant recipient staff who hosted us for site visits and participated in interviews. In particular, we’d like to thank the leadership at each grant recipient: Cosette Bowles and Charles Dillon (Anthem Strong Families); Francesca Adler-Baeder, Vanessa Finnegan, and Charles Jackson (Auburn Youth Relationship Education Program); Alison Espinola, Cynthia MacDuff, and Kendra Webster (Family Service Agency of Santa Barbara); Leah Kohr (Gateway Community Services); Traci Maynigo and Moshe Moeller (Montefiore Medical Center); Phillippia Faust, Jamila Hudson Ford, Viola January, and Tina Thomas (More Than Conquerors, Inc.); Jessie Purcel and Galena Rhoades (University of Denver MotherWise); Lakshmi Mahadevan and Laurie Naumann (Texas A&M AgriLife); Catherine and Ron Tijerina (The RIDGE Project); Sharon Oney and Mary Ann Slanina (Youth and Family Services).
Suggested citation
Buonaspina, A, R.D. Piatt, H. Gordon, K. Hunter, D. Friend, S. Baumgartner, and R. Wood. “Lessons from Rapid Cycle Learning with Healthy Marriage and Relationship Education Grant recipients.” OPRE Report #2023-054. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, March 2023.
Disclaimer
This report is in the public domain. Permission to reproduce is not necessary.
Mathematica wrote this brief under contract with the Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services (#HHSP233201500035I/75P00119F37045). The views expressed in this publication do not necessarily reflect the views or policies of OPRE, ACF, or the U.S. Department of Health and Human Services.
Endnotes
1Hsueh, J., D. P. Alderson, E. Lundquist, C. Michalopoulos, D. Gubits, D. Fein, and V. Knox. “The Supporting Healthy Marriage Evaluation: Early Impacts on Low-Income Families.” OPRE Report #2012-11. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, 2012.
2Stanley, S.M., R.G. Carlson, G.K. Rhoades, H.J. Markman, L.L. Ritchie, and A.J. Hawkins. “Best Practices in Relationship Education Focused on Intimate Relationships.” Family Relations, vol. 69, no. 3, July 2020, pp. 497—519.
3Wadsworth, M.E., and H.J. Markman. “Where’s the Action? Understanding What Works and Why in Relationship Education.” Behavior Therapy, vol. 43, no. 1, March 2012, pp. 99—112.
4Simpson, D. M., N. D. Leonhardt, and A. J. Hawkins. “Learning About Love: A Meta-Analytic Study of Individually-Oriented Relationship Education Programs for Adolescents and Emerging Adults.” Journal of Youth and Adolescence, vol. 47, 2018, pp. 477—489. https://doi.org/10.1007/s10964-017-0725-1 .
5Scott, M., E. Karberg, I. Huz, and M. Oster. “Healthy Marriage and Relationship Education Programs for Youth: An In-Depth Study of Federally Funded Programs.” OPRE Report #2017-74. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2017.
6Scott, M. E., and I. Huz. “An Overview of Healthy Marriage and Relationship Education Curricula.” Bethesda, MD: Marriage Strengthening Research and Dissemination Center, June 2020.
7Wu, A.Y., Q. Moore, and R. Wood. “Healthy Marriage and Relationship Education with Integrated Economic Stability Services: The Impacts of Empowering Families.” OPRE Report #2021-224. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2021.
8Baumgartner, S. and H. Zaveri. “Implementation of Two Versions of Relationship Smarts Plus in Georgia,.” OPRE Report # 2018-121. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2018.
9Friend, D. and Paulsell, D. “Research to Practice Brief: Developing Strong Recruitment Practices for Healthy Marriage and Relationship Education (HMRE) Programs.” OPRE Report #2020-78. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2020.
10Fixsen, D.L., S.F. Naoom, K.A. Blase, R.M. Friedman, and F. Wallace. Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network (FMHI Publication #231), 2005.
11Derr, M., A. Person, and J. McCay. "Learn, Innovate, Improve: Enhancing Programs and Improving Lives." OPRE Report #2017-108. Washington, DC: Mathematica, 2017.
12University of Minnesota. Organizational Behavior. Minneapolis, MN: University of Minnesota Libraries Publishing, 2017. Available at: https://open.lib.umn.edu/organizationalbehavior/
13McChesney, C., S. Covey, & J. Huling. The 4 Disciplines of Execution: Achieving Your Wildly Important Goals. New York: Free Press, 2012.
14Derr, M., A. Person, and J. McCay. "Learn, Innovate, Improve: Enhancing Programs and Improving Lives."
15Nicholas, G., J. Foote, K. Kainz, K., G. Midgley, K. Prager, & C. Zurbriggen. Towards a Heart and Soul for Co-Creative Research Practice: A Systemic Approach." Evidence & Policy, vol 15, no. 3 (2019), pp 353-370. Available at https://bristoluniversitypressdigital.com/view/journals/evp/15/3/article-p353.xml
16Derr, M., A. Person, and J. McCay. "Learn, Innovate, Improve: Enhancing Programs and Improving Lives."