Iterating to Improve: Lessons from Rapid Cycle Learning with Healthy Marriage and Relationship Education Grant Recipients

Publication Date: June 20, 2023
 

Iterating to Improve: Lessons from Rapid Cycle Learning with Healthy Marriage and Relationship Education Grant Recipients

Introduction

logo-simr

Healthy marriage and relationship education (HMRE) services are designed to help participants build and sustain strong families. However, HMRE practitioners often face common implementation challenges that prevent them from delivering services as intended. These challenges include enrolling enough participants, motivating participants to attend workshops and supplemental services, and developing engaging content so participants learn and incorporate healthy relationship skills into their lives.

When faced with these complex challenges, getting unstuck can feel difficult. Rapid cycle learning, or small-scale, iterative testing, is a method that can help practitioners to improve. In the Strengthening the Implementation of Marriage and Relationship Programs (SIMR) project, 10 HMRE grant recipients partnered with researchers and technical assistance staff to test and refine improvement strategies using a rapid cycle learning approach. The grant recipients represented a range of youth and adult service providers, serving  various populations, located in rural and urban areas, and ranging from smaller community organizations to programs within large universities. Read more about the grant recipients on the Meet the Grant Recipients tab.

This digital essay presents the experiences and lessons learned of the 10 grant recipients through narrative and video. Key lessons learned are summarized below and detailed across “Tips” tabs in this essay.

One grant recipient’s SIMR journey

Montefiore Medical Center’s Supporting Healthy Relationships (SHR) program has served couples in the Bronx for nearly 20 years. SHR provides evidence-based HMRE services. When the COVID-19 pandemic began, Montefiore faced one of its toughest challenges yet, as their experienced staff struggled to lead engaging virtual workshops. On SIMR, staff at Montefiore partnered with a team of researchers and technical assistance staff to tackle their key implementation challenges using rapid cycle learning. Montefiore’s team developed and tested innovative, participant-centered improvements to their virtual services. 

videoicon5 Click below to hear staff from Montefiore describe their challenges and opportunities to improve.

Staff from Montefiore Medical Center discuss how they partnered with SIMR to improve virtual HMRE services

 

 What is rapid cycle learning?

Rapid cycle learning is a process for programs to develop and test targeted improvement strategies to address programmatic challenges, with the goals of strengthening their own services and generating lessons for the field. In the SIMR project, HMRE grant recipients spent time upfront prioritizing a challenge and identifying its root causes, before developing and testing improvement strategies. See the What is SIMR? tab for more information on the approach used in SIMR. HMRE grant recipients and other human services providers may be familiar with continuous quality improvement (CQI). CQI can employ short, iterative rapid learning cycles to test improvement strategies, such as those that the grant recipients in SIMR used.      

 

Key Lessons Learned

This digital essay features lessons learned from 10 grant recipients that participated in rapid cycle learning in SIMR. Key lessons about engaging in rapid cycle learning included:

  • Orient your team toward improvement, as a program improvement mindset helps providers identify their challenges and areas where they could enhance services
  • Find a common goal to motivate the team
  • Make space for innovation to bring staff into the improvement process
  • Collect data and feedback to assess strategies and make evidence-informed decisions
  • Overcome the fear of change to move from planning to action

Though SIMR grant recipients engaged in rapid cycle learning in partnership with researchers, their lessons learned are practices they intend to sustain on their own.  Other providers can implement these tips themselves to help improve their services in meaningful ways. 

 

Navigating the digital essay

Read, watch, and listen to this digital essay to learn from HMRE grant recipients about their experience with rapid cycle learning and their advice on how you can begin your journey toward program improvement. The essay contains video interviews in both English and Spanish from nine of the participating grant recipients. All videos include English captions. To turn off these captions or switch them to Spanish, please use the "cc" button at the bottom of each video.

  • For background on SIMR: Click on the next three tabs to the left to learn more about HMRE services, SIMR, and the grant recipients who participated in the project.
  • For lessons learned about rapid cycle learning: Explore the tabs labeled “Tips” to hear from grant recipients about several key ingredients for improving their services using rapid cycle learning, refined through their participation in SIMR.
  • For additional SIMR reports and resources: More information about SIMR project findings can be found on ACF's SIMR project site page.
placeholder
placeholder

 

 

 

What are HMRE Services?

Healthy Marriage and Relationship Education (HMRE) services aim to help participants form and maintain healthy relationships. The federal government is making a long-standing investment to provide these services across the United States. HMRE grant recipients, funded by the Office of Family Assistance (OFA) within the Administration for Children and Families (ACF), Department of Health and Human Services have been serving youth, adults, and families since 2005. This digital report features videos from staff at several HMRE grant recipients discussing the need for HMRE services for youth and adults (for more information please visit the grant program page). 

Click below to hear SIMR grant recipients speak about the importance of HMRE services for youth and adults.

Marlin Young from More Than Conquerors, Inc., discusses why HMRE services are meaningful for high school students


Staff from Montefiore Medical Center, Anthem Strong Families, and The RIDGE Project discuss the importance of HMRE services for adults


Explore the sections below to learn more about HMRE services for adults and for youth.

HMRE services for adults

HMRE services for adults typically serve two populations: (1) adult couples and (2) individual adults. Some HMRE services are designed for a couple’s joint participation. Other services are designed for individual adults to participate in regardless of their relationship status.

What do adult HMRE participants learn? Providers teach skills such as effective communication and conflict management, how to recognize signs of healthy and unhealthy relationships, and strategies to deepen connection and intimacy.1,2,3 Typically, HMRE services include a core educational workshop, along with supplemental services like case management and parenting workshops. Some providers also integrate job and career advancement activities to promote families’ economic stability. See ACF’s Pathways to Outcomes report for more information on adult HMRE services.

HMRE services for youth 

Young people’s early relationship experiences can establish a foundation for the success of their future relationships.4 HMRE services developed for youth aim to prepare participants between the ages of 14 and 24 for positive relationships in adulthood and educate them about the social and emotional aspects of relationships.5,6 Providers focus on topics such as the signs of healthy and unhealthy relationships, teen dating violence, and effective communication and conflict management skills.7,8 Youth-serving HMRE services are most often delivered in high schools, but services are also offered in after-school or community-based settings.9,10 See this ACF brief for more information on youth HMRE services.

 

placeholder
placeholder

What is SIMR?

Photo of a Montefiore Medical Center staff member engaging in a human-centered design activity at a SIMR site visit

Strengthening the Implementation of Marriage and Relationship Programs (SIMR)’s two main goals were (1) to improve service delivery of the 10 participating grant recipients, using rapid cycle learning methods to test strategies for addressing common implementation challenges; and (2) to develop lessons for the broader HMRE field about promising practices.

SIMR grant recipients tackled three broad implementation challenges:

Icon depicting a person standing next to a graph with bars of increasing height and a star at the top of the last bar. Used to indicate enrollment challenges.

Recruiting participants to enroll in services

Icon depicting a clipboard with person icons and circles or xes next to the names. Used to indicate retention challenges.

Motivating participants to attend primary HMRE workshops and supplemental services (like case management)

Icon depicting three people, with the person on the left raising their right hand. Used to indicate engagement challenges.

Developing engaging content so participants understand and absorb key takeaways.

Addressing these challenges is vital to ensuring that HMRE providers serve participants effectively.

Icon with dark blue ellipses enclosed in a circle.Want to learn more about promising strategies HMRE providers could adapt and test in their context? See Strategies for Addressing Common Implementation Challenges in Healthy Marriage and Relationship Education Programs.  

Learn more about the LI2 framework that guided SIMR

Infographic for the L-I-squared approach with a circle that has arrows to indicate a continuous process. LEARN is at the top with a binoculars icon. INNOVATE is on the right with a light bulb icon. IMPROVE is on the left with a graph icon.

Learn, Innovate, Improve (LI2) is a program improvement framework that incorporates rapid cycle learning as well as important upfront steps to set up learning cycles, including understanding the problem and developing innovative strategies to address it.11 The LI2 framework guided the work in SIMR. LI2 can be used to guide continuous quality improvement efforts or used within research partnerships to build evidence for the field. In SIMR, researchers and practitioners used LI2 to partner to unpack challenges and identify barriers to addressing those challenges (Learn); create innovative program improvement strategies that are participant-centered, informed by evidence, and sustainable (Innovate); and use rapid cycle learning methods to test and refine strategies (Improve). Throughout each learning cycle, the SIMR team and grant recipients met to discuss progress, identify challenges and barriers, monitor data, and fine tune the strategy as needed.

placeholderClick below to learn more about LI2 and the importance of having an improvement process.

Scott Baumgartner, a senior researcher at Mathematica, explains the phases of LI2 and Phillippia Faust, a senior adviser at More Than Conquerors, Inc., discusses the importance of integrating an improvement process into program operations

Learn more about each phase in the LI2 framework below.

In the Learn phase, grant recipients worked to identify and specify challenges; engage their staff in identifying root causes contributing to the specific implementation challenge they were addressing; and understand barriers to addressing the challenge. The Learn phase helps teams to get on the same page about the motivation for change and provides a foundation for the Innovate phase, during which grant recipients identified improvement strategies.

In the Innovate phase, grant recipients drew on the multiple perspectives of their staff and the research team to develop strategies to address key root causes identified during the Learn phase. Teams gathered input and improvement ideas from staff by using human-centered design activities (see Tip 3: Make space for innovation). In addition, staff participated in trainings on research-informed best practices and models, which grant recipients then operationalized to fit their unique context and meet the needs of their service population.

In the Improve phase, the SIMR team and grant recipient staff collaborated to iteratively test and refine strategies using rapid cycle learning. Grant recipients and the SIMR team used several sources of data to carry out the strategy and assess its success. Data sources included:

  • Quantitative data
    • Surveys
    • Observations
    • Administrative data from nFORM, OFA’s management information used by HMRE grant recipients, and the grant recipients’ own internal data systems
  • Qualitative data
    • Interviews
    • Focus groups
    • Human-centered design activities
    • Regular debriefs with staff

 

Photo of a Texas A&M Agrilife Extension staff member engaging in a human-centered design activity
Photo of a More than Conquerors, Inc. staff member engaging in a human-centered design activity

Tip 1: Orient your team toward improvement

Lightbulb icon shown in tip boxesNavigating the digital essay

Explore the “Tip” tabs, where grant recipients share their thinking on what they found to be the key ingredients of success for addressing their implementation challenges using rapid cycle learning.

No matter how experienced or established they are, grant recipients will inevitably run into challenges. Rapid cycle learning is a process that can support service providers in working through challenges. Getting the most out of the process, however, may require grant recipients to rethink or enhance their existing operations.

“We didn’t want to say, ’Well, that worked great for 5 years, let’s do it again.”

-Cynthia MacDuff, senior grants manager, FSA-SB

An improvement mindset means always looking for opportunities to enhance services, as even strong providers can get better. Staff participating in SIMR spoke about their desire to tackle lingering challenges, such as recruiting Spanish-speaking males or improving outreach in rural communities. Others shared a desire to continue delivering high-quality services under new circumstances—for instance, being funded to serve a new population or translating in-person services to a virtual environment in response to the COVID-19 pandemic. Adopting a mindset of continuous improvement means reexamining practices that might have gone stale and demonstrating an openness to trying something different.

Icon indicating a video is being presentedClick below to hear staff emphasize the importance of questioning why things are done the way they are.
 

Staff at Family Service Agency of Santa Barbara and University of Denver’s MotherWise program emphasize the need to continually examine practices to maintain quality services 

 

“We were underestimating how some of these things can help to support the very human connection that we need to make.”

-Traci Maynigo, project director, Montefiore

An improvement mindset helped grant recipients develop solutions by propelling them to be more open to new or different ideas. Grant recipients shared that changing a long-standing practice can feel unnerving but is worth the risk. More Than Conquerors, Inc., and FSA-SB, for example, enhanced their case management approach by shifting from a more traditional assessment and referral process to a motivation-based, goal-attainment approach.

placeholderWant to learn more about more about this strategy? See Adopting a Motivation-Driven Approach to Case Management in HMRE Services.

Not every change needs to be transformational. Being open to change also means not overlooking small tweaks. Rather than waiting for a single perfect solution, service providers may find more success making incremental changes to strengthen processes over time. Many teams in SIMR tested small tweaks to address challenges—for instance, refinements to pitch emails or a Zoom guide to help participants navigate virtual workshops.

placeholderWant to learn more about what SIMR grant recipients learned to strengthen virtual practices? See Tips for Delivering Engaging Virtual Workshop Sessions in Healthy Marriage and Relationship Education Programs.

Icon indicating a video is being presentedClick below to hear staff speak to the importance of being open to new ways of working, highlighting the need for shifting mindsets to innovate, be open to change, and appreciate the value of making small changes.

Staff at MTCI, FSA-SB, Texas A&M AgriLife, and YFS discuss being open to trying a new approach

Note: MTCI and FSA-SB both tested motivation-driven practices as strategies, encouraging participants to identify and work on their own goals.

Staff at MotherWise and Montefiore share how small refinements to practices made a big difference

Note: Montefiore tested a range of strategies to enhance virtual facilitation, including a Zoom guide to help participants navigate the technology.

Tip 2: Find and focus on your team’s most important goal

Grant recipients started their work in SIMR by identifying an implementation challenge and a goal related to the challenge, developed strategies to address the challenge, and then tested and refined strategies to support improvement. For instance, Auburn and YFS identified challenges related to content engagement and chose to focus on supporting facilitators to deliver engaging lessons. Several teams identified challenges with recruiting specific populations— for example, rural youth or Spanish-speaking men—and strove to enhance recruitment processes to boost enrollment of those populations.

placeholderWant to learn more about what SIMR grant recipients learned to strengthen their recruitment efforts? See Tips for Leveraging Partnerships to Improve Recruitment for Healthy Marriage and Relationship Education Programs and Four Tips to Recruit Spanish-Speaking Men into Healthy Marriage and Relationship Education Programs.

Lightbulb icon shown in tip boxesSetting Goals

Identifying a clear and motivating goal at the start of an improvement effort is an important step for maintaining disciplined focus across staff and seeing it through to the end. Goal setting within an organization can motivate staff and encourage them to become active participants in change.12

Narrowing the focus of improvement to just the most important goal, can help staff commit to supporting improvement while still having the capacity to attend to daily work.13

 

To develop a motivating goal, SIMR grant recipients involved a range of staff and used structured activities to ensure the staff had input in identifying a key challenge and determining the goal. Having a common goal, staff said, gave teams a motivating purpose to come back to during parts of the rapid cycle learning process that felt less clear. For example, digging into challenges and brainstorming strategies involve “diverging before converging” -in other words, putting everything on the table before narrowing the focus to a specific challenge or solution.14

Icon indicating a video is being presentedClick below to hear staff speak about how having a clearly defined and shared goal provides a compass for navigating a successful rapid cycle learning process. For example, members of grant recipient teams share how having a clear objective helped support collaborative and disciplined brainstorming.

Staff at MTCI, University of Denver’s MotherWise program, Texas A&M AgriLife, and FSA-SB share how purpose-driven improvement efforts support collaboration and motivation to persist through barriers    

A true co-creative process must recognize the value of diverse forms of expertise, particularly from those who are affected by the changes being implemented, such as grant recipient staff and participants.15 Grant recipients in SIMR saw value in bringing diverse voices into the process of developing strategies that support progress toward goals. These diverse perspectives included a high-level, strategic view from leadership; an evidence-informed perspective from the research team; on-the-ground insights from frontline staff; and personal experiences from participants and partners. Drawing on multiple perspectives, teams developed evidence-informed strategies that proved feasible to implement within the service provider's context and hold promise for supporting improvement.

“It was something that we built together, and everyone collaborated. In the beginning there were no bad ideas, all the ideas were welcomed.”

-Yerson Torres, facilitator, Montefiore

Icon indicating a video is being presentedClick below to hear leaders from YFS and Texas A&M AgriLife emphasize the value of multiple perspectives when developing solutions. 

Leaders at YFS and Texas A&M AgriLife share examples of what collaborative strategy development looked like within their teams

Tip 3: Make space for innovation

“Getting everything done in a day is important, but also, having time in a day to think about, ’Am I doing this the best way possible?’ is important, too.”

-Mary Minges, clinical coordinator, Montefiore

Keeping up with the day-to-day tasks of running an HMRE service provider can feel all-consuming. With everything else on your agency's collective plate, how can staff fit in conversations on how to do things better? While understanding the competing pressures on time, grant recipients found that a key takeaway from engaging in a rapid cycle learning process was a deeper appreciation for “team time”—that is, spending time to be intentional in their plans for program improvement.

For grant recipients in SIMR, prioritizing improvement meant allocating dedicated and structured time to focus on challenges, develop solutions, and monitor progress. This was a key lesson learned that staff intended to retain after their involvement with SIMR ended.

Who was on SIMR teams?

  • Project leadership
  • Facilitators
  • Case managers/family advocates
  • Partners
  • SIMR team researchers/TA staff

Most grant recipients held regular program improvement meetings as part of SIMR. They found that carving out space to discuss improvement efforts supported greater intentionality in responding to challenges and determining next steps. Regularly scheduled meetings also helped to foster a sense of staff accountability to make progress toward goals and this helped teams avoid getting distracted by other priorities that arose.

 

 

Icon with dark blue ellipses enclosed in a circle.Want to learn more about strategies for supporting staff in HMRE service providers? See Four Tips for Supporting Staff Providing Healthy Relationship and Marriage Education Services

Icon indicating a video is being presentedClick below to listen next to the voices of staff from four teams share the value of having dedicated time devoted to their improvement efforts.

Staff at Montefiore, FSA-SB, University of Denver’s MotherWise program, and Gateway share the value of making time for reflection

Staff shared that an important part of what made “team time” so valuable in SIMR was the opportunity it provided to hear from all staff, allowing teams to draw on multiple perspectives to shape improvement decisions. Creating space where all staff felt comfortable contributing ideas required project leaders to ensure staff saw themselves as “experts of the implementation team,” according to one director. Grant recipients also valued hearing from all voices through structured human-centered design activities.

placeholderMore on human-centered design activities

Human-centered design empowers a range of people, such as staff, partners, and participants, to get involved in innovation. Because of this, it often yields strategies that are more feasible for staff to implement and more effective.

Several human-centered design activities are useful for rapid cycle learning. For instance, staff at Montefiore used Round Robin, a strategy development activity where team members build upon each other’s ideas to make them a little better with each iteration. According to one project director, the activity allowed everyone on the team to bring their unique perspectives to collectively develop ideas for what the grant recipient could do differently.

Read more about human-centered design.

 

Icon indicating a video is being presentedClick below to hear what other leadership and staff involved in SIMR shared about what it takes to empower staff and how frontline staff members felt about participating in planning conversations.

Staff at FSA-SB, Montefiore, University of Denver’s MotherWise, and Texas A&M AgriLife share what it takes to empower staff to participate in improvement conversations

placeholderWant to learn more about skill coaching? The above video references Love Bites, which was a new approach to skills coaching that Montefiore tested and refined through SIMR. To learn more, see Tips for Providing Skill Coaching to Reinforce Workshop Content in Healthy Marriage and Relationship Education Programs

 

Tip 4: Collect and use data to inform improvements

Gathering data on how strategies are working helps practitioners know how to refine and improve their approach.16 Grant recipients that participated in SIMR observed the value of gathering and using feedback from frontline staff, participants, and partner organizations. In addition to this qualitative information, SIMR grant recipients found that collecting quantitative data to monitor their progress helped them continuously refine their improvement strategies.

placeholderMonitoring overall success of improvement efforts

To assess the overall success of their improvement efforts, grant recipients set overarching goals and many used data from nFORM to monitor progress. nFORM, or the Information, Family Outcomes, Reporting, and Management system , is a data system used by Healthy Marriage and Responsible Fatherhood grant recipients to collect, store, and analyze data.

Monitoring success over the course of the project helped grant recipients to determine whether their efforts appeared to be supporting success. For example, recruitment teams monitored whether their strategies appeared to support increased enrollments of key populations by examining baseline numbers and tracking improvement over time.

Grant recipients shared the value in collecting new data—data outside existing administrative data systems—to learn about their challenges and identify strategies for improvement. They developed processes to track data that allowed them to obtain real-time feedback on their strategies, enabling them to learn what was working and what needed to be refined. For example, grant recipients working to establish new partnerships with organizations in their community used a partner tracking sheet to monitor relevant data, such as outreach attempts, in order to assess their progress. Staff shared that this tracker helped them be more strategic with partner outreach.

placeholderWant to learn more about using data to guide rapid cycle learning? See Using Data to Guide Program Improvement in Healthy Marriage and Relationship Education Services.

Icon indicating a video is being presentedClick to listen to staff from Texas A&M AgriLife share how tracking data to learn about improvement strategies—in their case, social media analytics to assess an Instagram strategy—helped inform the team’s recruitment outreach approach. Staff from University of Denver’s MotherWise program also share how social media analytics helped the team be strategic in how they promoted The Nest, a phone app they used to connect with participants.

Staff at Texas A&M AgriLife and University of Denver’s MotherWise program talk about the value of data

SIMR grant recipients appreciated that rapid cycle learning included opportunities to gather feedback from participants and partners. Grant recipients used various methods to hear from those involved with their services, such as interviews, focus groups, surveys, and human-centered design activities. Participants and partners shared useful feedback on how grant recipients could improve their services and refine their strategies.

For example, Anthem held a focus group with younger participants in HMRE services to test a new Instagram recruitment campaign. The goal of the focus group was to get youth's perspectives on how they use social media, what posts they commonly interact with, and how to improve the images and captions of their draft posts. The team refined their social media plan and posts based on what they learned from participants. Other grant recipients used surveys with partners or exit tickets from youth—short one-to-two-question surveys administered at the end of a workshop—to gather feedback.

“I honestly don’t think [the feedback youth shared] is something I would have been able to pick up on my own… without the exit ticket [survey].”

-Emma Spies, facilitator, YFS

Icon indicating a video is being presentedClick below to hear staff from three grant recipients share what they learned from gathering feedback from participants and partners.

Staff at YFS, Montefiore, and Gateway share key insights from participants and partners

Tip 5: Overcome the fear of change

Rapid cycle learning emphasizes starting small and reframing failure as a learning opportunity; grant recipients shared that this framing gave them permission to be innovative and creative and pushed them to try out new strategies. Staff shared that getting too comfortable with existing practices or fearing to disrupt a previously successful strategy often caused them to play it safe and avoid risks.

Grant recipients appreciated how rapid cycle learning helped them cultivate a mindset of “failing forward,” where missing the mark with an initial strategy can provide new insights on how to solve an existing problem. Using a rapid cycle learning approach propelled grant recipients to try some of the ideas they had discussed in the past but never launched. For example, prior to SIMR, Texas A&M AgriLife discussed expanding their use of social media to try to recruit more participants into their HMRE services. Starting small using a rapid cycle learning process gave them the push to get started and launch an even broader recruitment revamp effort than had been previously planned.

“It can be hard not to over analyze. And I think that’s the beauty behind rapid cycle, is that it says, ’Hey, don’t get into the paralysis of analysis. Take some time, put your thought into it, but then let’s get something activated.'”

-Chris Liotta, outreach specialist, Texas A&M AgriLife

Icon indicating a video is being presentedClick below to hear staff at Texas A&M AgriLife, who tested variety of strategies to improve recruitment in rural counties, discuss how rapid cycle learning helped them overcome planning paralysis.

Staff at Texas A&M AgriLife share how rapid cycle learning helped them move from planning to action

Rapid cycle learning can help staff move from a mindset whereby a strategy either works or does not work to a mindset of how to make it work through small refinements. Grant recipients shared that rapid cycle learning helped them identify specific ways to adjust and tweak their strategies to inform improvements. For example, staff at FSA-SB shared that if certain strategies didn’t work in the past, they typically would abandon the approach and seek out other strategies. Rapid cycle learning gave them a structure and process for making incremental changes to existing strategies to strengthen them.

“Through SIMR, we learned to adjust”

-Cynthia MacDuff, senior grants manager, FSA-SB.

Icon indicating a video is being presentedClick below to hear staff speak to the value of adjusting to improve.


Staff at FSA-SB, MTCI, and University of Denver’s MotherWise program share an appreciation for refining strategies over time

 

Beyond SIMR

Using rapid cycle learning won’t end for the grant recipients who participated in SIMR with the end of the SIMR project. Rapid cycle learning can be adopted as part of a CQI process that happens outside of a research partnership. Teams planned to retain key elements from their work in SIMR. For example, many grant recipients in SIMR developed teams of staff to guide implementation of improvement strategies during the project and these teams planned to continue meeting post-SIMR. Teams also expressed an appreciation for and desire to continue using data to inform their improvement work. Other service providers could use the experience of the grants recipients that participated in SIMR to inform their own efforts to improve their services through rapid cycle learning. 

Icon indicating a video is being presentedClick below to listen to what leaders of three grant recipients say about their plans to keep improving their services through rapid cycle learning going forward and hear from a senior project manager on the SIMR team about what grant recipients overall may take from the experience.

Leaders at Texas A&M AgriLife, FSA-SB, and MTCI discuss sustaining an improvement process

Lauren Mattox, a senior project manager at Public Strategies, explains how SIMR gave grant recipients a model for addressing future challenges

Using rapid cycle learning offers an approach for other providers to try out and adapt strategies tested in SIMR in their contexts. By drawing on tips for engaging with rapid cycle learning through testing strategies, providers can continue to learn about how to address key challenges related to recruitment, retention, and engagement.

Icon indicating a video is being presentedClick below to listen to Daniel Friend’s call to action to tailor strategies in SIMR and test them using rapid cycle learning and, later, rigorous evaluations.

Daniel Friend, a senior researcher at Mathematica, talks about what’s next for the field

Decorative icon with ellipses enclosed in a circleMathematica and its partner Public Strategies conducted the SIMR project under a federal contact overseen by ACF’s Office of Planning, Research, and Evaluation with funding from ACF’s Office of Family Assistance. For more details on the SIMR project design, grant recipients’ early implementation challenges, and strategies tested in SIMR, see ACF's SIMR project site page.

Resources and references

Glossary of terms

  • Continuous quality improvement (CQI): CQI includes long-term and ongoing processes for programs to identify performance challenges, develop changes to address those challenges, and monitor the success of those changes over time. Programs can adopt rapid cycle learning as part of their CQI process to test programmatic changes. CQI is often used to support internal improvements.        
  • Healthy marriage and relationship education (HMRE): A federal government grant program serving youth, adults, and families. HMRE grant recipients are funded by the Office of Family Assistance (OFA) within the Administration for Children and Families (ACF).
  • Learn, Innovate, Improve (LI2): A framework that guided the work in SIMR. LI2 is an evidence-driven program improvement approach that providers can use to guide internal CQI efforts to improve programs or to build evidence for the field through research partnerships.
  • Rapid cycle learning: A method for quickly testing strategies to strengthen programs. It often involves successive cycles to pilot strategies, where teams collect feedback on how they’re working as well as data to demonstrate whether they’re supporting improvement. Next steps focus on refining strategies and testing them again based on what is learned. Rapid cycle learning can be used for internal improvement or in partnership with researchers to build evidence for the broader field. 
  • Strengthening the Implementation of Marriage and Relationship Education Programs (SIMR): A project funded by ACF. SIMR’s two main goals were to improve service delivery of the 10 participating grant recipients by testing strategies to address common implementation challenges, and to develop lessons for the broader HMRE field about promising practices.

Acknowledgements

Developing this report was a collaborative effort. Many people worked on the SIMR project and contributed to the success of this report. First, we thank the SIMR project teams from Mathematica and Public Strategies (PS). In addition to the digital essay authors, the following team members led the technical assistance and rapid cycle learning with each site: Delara Aharpour, Alex Bauer, Sheila Cavallo, Mark Ezzo, Camila Fernandez, Aly Frei (PS), Daisy Gonzalez, Nick Gunzelman (PS), Avery Hennigar, Azaliah Israel (PS), Alisha Jordan (PS), Mollie Lovera (PS), Lauren Mattox (PS), Hannah McInerney, Adilia McManus (PS), Caroline O’Callahan, Grace Guerrero Ramirez, Scott Roby (PS), Shaun Stevenson, Veronica Murphy Sotelo, and Armando Yanez.

We also recognize the following staff members for their contributions creating the videos and report content. Sheila Cavallo, Mark Ezzo, Caroline O’Callahan, Katelyn Olsen, and Veronica Murphy Sotelo served on the digital report video review team. Pamela Holcomb served as the report’s quality assurance reviewer. Robert Wood is the SIMR project director and Quinn Moore is the SIMR principal investigator. Tara Merry and George Hoffman led the web development effort. Alejandra Hernandez, Grazia Maroso Mieren, Yvonne Marki and Laura Sarnoski created the report’s graphics and Donna Verdier edited the text. Rich Clement shot the videos and photos displayed throughout this digital report. Jaime Carreon and Adam Cole also provided photography.  

We thank OPRE and OFA for guiding the SIMR project. We are deeply grateful for the guidance and feedback from OPRE Project Officers Samantha Illangasekare, Rebecca Hjelm, and Shirley Adelstein and Project Monitor Meghan Heffron. We also thank the OFA federal program specialists who provided insight and feedback throughout SIMR and participated in meetings with grant recipients and the SIMR team: John Allen, Millicent Crawford, Tanya Howell, Toya Joyner, Jackie Proctor, and Barbara Spoor.

We also thank the grant recipient staff who hosted us for site visits and participated in interviews. In particular, we’d like to thank the leadership at each grant recipient: Cosette Bowles and Charles Dillon (Anthem Strong Families); Francesca Adler-Baeder, Vanessa Finnegan, and Charles Jackson (Auburn Youth Relationship Education Program); Alison Espinola, Cynthia MacDuff, and Kendra Webster (Family Service Agency of Santa Barbara); Leah Kohr (Gateway Community Services); Traci Maynigo and Moshe Moeller (Montefiore Medical Center); Phillippia Faust, Jamila Hudson Ford, Viola January, and Tina Thomas (More Than Conquerors, Inc.); Jessie Purcel and Galena Rhoades (University of Denver MotherWise); Lakshmi Mahadevan and Laurie Naumann (Texas A&M AgriLife); Catherine and Ron Tijerina (The RIDGE Project); Sharon Oney and Mary Ann Slanina (Youth and Family Services).

Suggested citation

Buonaspina, A, R.D. Piatt, H. Gordon, K. Hunter, D. Friend, S. Baumgartner, and R. Wood. “Lessons from Rapid Cycle Learning with Healthy Marriage and Relationship Education Grant recipients.” OPRE Report #2023-054. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, March 2023.

Disclaimer

This report is in the public domain. Permission to reproduce is not necessary.

Mathematica wrote this brief under contract with the Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services (#HHSP233201500035I/75P00119F37045). The views expressed in this publication do not necessarily reflect the views or policies of OPRE, ACF, or the U.S. Department of Health and Human Services.

Endnotes

1Hsueh, J., D. P. Alderson, E. Lundquist, C. Michalopoulos, D. Gubits, D. Fein, and V. Knox. “The Supporting Healthy Marriage Evaluation: Early Impacts on Low-Income Families.” OPRE Report #2012-11. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, 2012. 

2Stanley, S.M., R.G. Carlson, G.K. Rhoades, H.J. Markman, L.L. Ritchie, and A.J. Hawkins. “Best Practices in Relationship Education Focused on Intimate Relationships.” Family Relations, vol. 69, no. 3, July 2020, pp. 497—519.

3Wadsworth, M.E., and H.J. Markman. “Where’s the Action? Understanding What Works and Why in Relationship Education.” Behavior Therapy, vol. 43, no. 1, March 2012, pp. 99—112.

4Simpson, D. M., N. D. Leonhardt, and A. J. Hawkins. “Learning About Love: A Meta-Analytic Study of Individually-Oriented Relationship Education Programs for Adolescents and Emerging Adults.” Journal of Youth and Adolescence, vol. 47, 2018, pp. 477—489. https://doi.org/10.1007/s10964-017-0725-1

5Scott, M., E. Karberg, I. Huz, and M. Oster. “Healthy Marriage and Relationship Education Programs for Youth: An In-Depth Study of Federally Funded Programs.” OPRE Report #2017-74. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2017.

6Scott, M. E., and I. Huz. “An Overview of Healthy Marriage and Relationship Education Curricula.” Bethesda, MD: Marriage Strengthening Research and Dissemination Center, June 2020.

7Wu, A.Y., Q. Moore, and R. Wood. “Healthy Marriage and Relationship Education with Integrated Economic Stability Services: The Impacts of Empowering Families.” OPRE Report #2021-224. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2021.

8Baumgartner, S. and H. Zaveri. “Implementation of Two Versions of Relationship Smarts Plus in Georgia,.” OPRE Report # 2018-121. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2018.

9Friend, D. and Paulsell, D. “Research to Practice Brief: Developing Strong Recruitment Practices for Healthy Marriage and Relationship Education (HMRE) Programs.” OPRE Report #2020-78. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2020.

10Fixsen, D.L., S.F. Naoom, K.A. Blase, R.M. Friedman, and F. Wallace. Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network (FMHI Publication #231), 2005.

11Derr, M., A. Person, and J. McCay. "Learn, Innovate, Improve: Enhancing Programs and Improving Lives." OPRE Report #2017-108. Washington, DC: Mathematica, 2017.

12University of Minnesota. Organizational Behavior. Minneapolis, MN: University of Minnesota Libraries Publishing, 2017. Available at: https://open.lib.umn.edu/organizationalbehavior/

13McChesney, C., S. Covey, & J. Huling. The 4 Disciplines of Execution: Achieving Your Wildly Important Goals. New York: Free Press, 2012.

14Derr, M., A. Person, and J. McCay. "Learn, Innovate, Improve: Enhancing Programs and Improving Lives."

15Nicholas, G., J. Foote, K. Kainz, K., G. Midgley, K. Prager, & C. Zurbriggen. Towards a Heart and Soul for Co-Creative Research Practice: A Systemic Approach." Evidence & Policy, vol 15, no. 3 (2019), pp 353-370. Available at https://bristoluniversitypressdigital.com/view/journals/evp/15/3/article-p353.xml

16Derr, M., A. Person, and J. McCay. "Learn, Innovate, Improve: Enhancing Programs and Improving Lives."