Introduction
Two-generation initiatives intentionally combine intensive, high quality adult-focused services with intensive, high quality child-focused programs to improve outcomes for children, primary caregivers, and families. The goal of integrating services for primary caregivers and their children is to achieve better outcomes than those accomplished by serving each generation in isolation (Chase-Lansdale and Brooks-Gunn 2014; Sama-Miller et al. 2017). Research suggests that to effectively support families, these services should be high quality, intensive, and intentionally aligned (Sama-Miller et al. 2017).
In the Next Steps for Rigorous Research on Two-Generation Approaches (NS2G) project, technical assistance (TA) providers partnered with four two-generation initiatives to conduct formative evaluations using rapid cycle learning. These evaluations aimed to help the initiatives iteratively test strategies to strengthen the quality, intensity, and intentionality of the services they offer to primary caregivers and their children.
This project is part of a portfolio of research focused on coordinated services to support children and families. Projects within this research portfolio address the intentional coordination of two or more services. These projects span OPRE’s program-specific research portfolios, including child care, Head Start, home visiting, child welfare, and welfare and family self-sufficiency. More information about OPRE’s Coordinated Services projects can be found at Coordinated Services Research and Evaluation Portfolio.
References:
Chase-Lansdale, P.L., and J. Brooks-Gunn. “Two-Generation Programs in the Twenty-First Century.” Future of Children, vol. 24, no. 1, 2014, pp. 13—39.
Sama-Miller, E., C. Ross, T.E. Sommer, S. Baumgartner, L. Roberts, and P.L. Chase-Lansdale. “Exploration of Integrated Approaches to Supporting Child Development and Improving Family Economic Security.” OPRE Report #2017-84. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2017.
Purpose
This brief is the third in a series of three briefs that aims to support future evaluations in the field of two-generation approaches. This third brief highlights the four initiatives’ experiences using rapid cycle learning to implement, test, and refine strategies to strengthen service delivery and program implementation. This brief is intended for practitioners who provide two-generation services and seek to strengthen their initiative using formative evaluation, as well as their evaluation partners.
Key Findings and Highlights
Each initiative engaged in rapid cycle learning as a way to quickly and iteratively test strategies to strengthen service design. To support these efforts, each initiative conducted at least two improvement cycles. During successive improvement cycles, initiative staff tried out their strategies and collected data on strategy implementation, including data from their data systems and feedback from staff. NS2G TA providers met regularly (approximately monthly) with initiative staff to check on progress and troubleshoot issues as they emerged and assisted initiative staff with data collection. At the end of each improvement cycle, initiative staff reviewed data with the NS2G TA providers and determined adjustments to the strategy before beginning another round of pilot testing with a revised strategy.
Four key lessons about conducting rapid cycle learning in two-generation initiatives emerged from the experiences of the four initiatives participating in NS2G:
- Two-generation initiatives lengthened their iterative improvement cycles to learn more about their improvement strategies or adjusted their planned approach as needed. Initiatives need to be flexible when engaging in rapid cycle learning to account for the often small number of families served by two-generation initiatives. Being flexible better enables initiatives to gather the feedback they need to make informed decisions.
- The rapid cycle learning approach was motivating. The narrow scope used by the rapid cycle learning process on NS2G made change faster paced and more manageable than undertaking large changes all at once. The short time-frame boosted motivation and encouraged future learning.
- The strategies that initiatives tested intentionally integrated supports for staff. While strategies varied across the four initiatives, they all included some form of staff supports. In this way, each initiative recognized the need to bolster staff capacity to improve the staff experience, and ultimately improve service delivery and the family experience.
- Regular, structured communication helped service providers coordinate their efforts. Initiative staff reported that when their initiatives’ strategies built in time for communication and coordination between staff, they experienced increased levels of comfort in referrals across departments, better coordination, and more capacity to support service delivery.
Methods
The findings in this brief emerged from rapid cycle learning with four two-generation initiatives. Rapid cycle learning is a method for quickly and iteratively testing strategies to strengthen service design and implementation. It often involves improvement cycles, which are successive cycles to pilot strategies, collect feedback from staff and participants on how these strategies are working, and gather data to demonstrate whether the strategies are supporting improvement. Based on the feedback collected and what initiative staff learn, they can revise their strategies and test again in a subsequent improvement cycle. This iterative testing and learning process builds staff capacity and initiative readiness for future evaluation by cultivating a learning mindset and strengthening service delivery.
Citation
Bauer, Alex, Nickie Fung, and Scott Baumgartner. “Using Rapid Cycle Learning to Build Momentum for Change in Two-Generation Service Delivery.” OPRE Report #2023-196. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2023.
Glossary
- NS2G:
- Next Steps for Rigorous Research on Two-Generation Approaches.
- Formative evaluation (also called process or implementation evaluation):
- A type of evaluation that is intended to strengthen the implementation of an intervention. Formative evaluation is important for understanding what services a program offers, the level of participation by clients and their satisfaction with services, challenges to participating, and ideas for improving the program (Rossi et al. 2003; Smith et al. 2009). This type of evaluation enables practitioners to define the core components of the initiative, develop a logic model, understand participant satisfaction with services, identify barriers to participation in services and areas for improvement in the model, and test strategies to see whether they strengthen the model.
- Learn, Innovate, Improve (LI2):
- A structured learning process for and an approach to program improvement that helps practitioners unpack program challenges, develop evidence-informed solutions, and use analytic methods to gather data to assess the success of a solution (Derr 2022).
- Rapid cycle learning:
- A method for quickly and iteratively testing strategies to strengthen service design and implementation. This iterative testing and learning process builds staff capacity and initiative readiness for future evaluation by cultivating a learning mindset and strengthening service delivery.
- Improvement cycle:
- Part of the rapid cycle learning process in which successive cycles are used to pilot strategies, collect feedback from staff and participants on how these strategies are working, and gather data to demonstrate whether the strategies are supporting improvement. Based on the feedback collected and what initiative staff learn, they can revise their strategies and test again in a subsequent improvement cycle.
- References:
- Derr, M. “Learn, Innovate, Improve: A Practice Guide for Enhancing Programs and Improving Lives.” Washington, DC: Mathematica, 2022. Rossi, P.H., M.W. Lipsey, and H.E. Freeman. Evaluation: A Systematic Approach. Thousand Oaks, CA: Sage Publications, 2003. Smith, C., T.J. Devaney, T. Akiva, and S.A. Sugar. “Quality and Accountability in the Out-of-School Time Sector.” New Directions for Youth Development, vol. 2009, no. 121, 2009, pp. 109–127.