Introduction
Two-generation initiatives intentionally combine intensive, high quality adult-focused services with intensive, high quality child-focused programs (such as Head Start or early childhood education) to improve outcomes for children, primary caregivers, and families. The goal of integrating services for primary caregivers and their children is to achieve better outcomes than those accomplished by serving each generation in isolation (Chase-Lansdale and Brooks-Gunn 2014; Sama-Miller et al. 2017). Although interest in two-generation initiatives has grown, the field lacks commonly accepted definitions and measures of relevant characteristics, processes, and outcomes. One relevant concept for two-generation researchers and practitioners is mutual reinforcement. Mutual reinforcement occurs when service providers align and build on each other’s efforts to serve whole families by adopting a shared vision and working toward common or compatible goals. This page contains the following preliminary resources to quantify a two-generation initiative’s level of mutual reinforcement for two-generation research and service delivery:
- A brief that describes the initial development of the measurement tool: The Two-Generation Mutual Reinforcement Measurement Tool: Development and Pilot Study Findings (PDF)
- Appendix A: Subscales and Scoring (PDF)
- Appendix B: Technical Approach for the Pilot Study (PDF)
- Appendix C: Two-Generation Mutual Reinforcement Measurement Tool: Hardcopy PDF (PDF)
- Appendix D: Two-Generation Mutual Reinforcement Measurement Tool: Electronic Version with Automated Scoring (link in Appendix section)
For an overview of the Two-Generation Mutual Reinforcement Measurement Tool, see this short video:
For a demonstration of the electronic version with automated scoring, see this video:
These resources were developed through the Next Steps for Rigorous Research on Two-Generation Approaches (NS2G) project. The NS2G project is part of a portfolio of research focused on coordinated services to support children and families. Projects within this research portfolio address the intentional coordination of two or more services. These projects span OPRE’s program-specific research portfolios, including child care, Head Start, home visiting, child welfare, and welfare and family self-sufficiency. More information about OPRE’s Coordinated Services projects can be found at https://www.acf.hhs.gov/opre/coordinated-services-research-and-evaluation-portfolio .
Purpose
The two-generation mutual reinforcement measurement tool is a first step to quantify a two-generation initiative’s level of mutual reinforcement. A valid and reliable measure of mutual reinforcement could provide actionable information specific to two-generation initiatives that will strengthen their service delivery, and therefore, child and family outcomes.
The Two-Generation Mutual Reinforcement Measurement Tool is intended for research and evaluation purposes, both formative and summative. To use this tool in a formative way, two-generation initiative staff and partners from the same initiative can work together to answer the questions, generate a mutual reinforcement score using the electronic version of the tool, identify areas for strengthening their initiative, and track changes over time as they work on those areas. The NS2G team designed the tool so that higher scores indicate stronger levels of mutual reinforcement. However, this tool and its scoring method are preliminary and not yet validated. This means the NS2G team has not yet taken the mathematical steps to determine the extent to which the tool are measures what it is intended to measure (validity) or the extent to which the tool can consistently reproduce the same results (reliability).While the NS2G team assumes that improving scores coincide with strengthening mutual reinforcement, the NS2G team does not yet know what scores translate to emergent, strong, or exemplary levels of mutual reinforcement. Once the tool and its scoring method are validated, two-generation initiatives and their research partners may be able to use this tool in a summative way (for example, in an impact evaluation).
Key Findings and Highlights
This brief and the accompanying appendix describe the approach to the initial development and piloting of the measurement tool, how the tool is scored, the results of initial testing, and next steps for continued testing and development.
Two-generation initiative staff and partners who complete the tool will be asked to:
-
Consider their partners who contribute to two-generation programs and services
-
Determine the extent to which the partners involved in the two-generation initiative have aligned their visions, goals, missions, and theories of change
-
Identify how many programs or services the two-generation initiative offers to each generation and to both generations (for example, child-focused with parent elements, parent-focused with child elements, whole family)
Then, the measurement tool will ask them to assess the degree to which:
-
Programs and services in the two-generation initiative are of high quality and appropriate intensity
-
Partners align and streamline the implementation of these programs and services
-
Partners capitalize on their relative strengths or areas of expertise in providing programming and services
-
Partners collect data about the two-generation initiative and, if so, (1) what types of data, (2) the extent of data sharing and linking across partners, and (3) the quality and usage of data
Key findings of the small pilot study are that the measurement tool:
-
Would be useful for participating two-generation initiative staff. All respondents participating in the pilot thought the tool would be useful in their work, would stimulate ideas for strengthening their initiative, and would help them improve coordination with their partners.
-
Is best suited for group administration. It takes each person approximately 30 minutes to complete the measurement tool on their own, but two-generation initiative staff in the pilot were more confident when they spent additional time discussing the questions with their colleagues to finalize their answers.
-
Would be more useful if initiatives could calculate their own mutual reinforcement score. Respondents reported that they could envision themselves using the tool to track mutual reinforcement over time and facilitate conversations within their organization and among partners, but they needed to be able to score their responses independently. Based on this input, the NS2G team developed the electronic, Excel-based version of the tool that automates scoring for users.
Methods
NS2G researchers refined the measurement tool over three waves of data collection. Eight two-generation initiative staff participated in the pilot study (May through July 2022) through cognitive interviews and survey debriefs. The goals of the pilot study were to:
-
Determine whether it would be feasible for staff and partners from two-generation initiatives to find time to respond to questions pertaining to mutual reinforcement
-
Test the survey for comprehension and ease of use
-
Understand whether respondents would find a mutual reinforcement measurement tool useful in their efforts to serve families
-
Refine the measurement tool
Appendix B contains more information on how the NS2G team recruited participants for the pilot study, how they conducted it, and how the approach to scoring changed over time.
Appendix
| File Type | File Name | File Size | XLSX | Appendix D: Two Generation Mutual Reinforcement Measurement Tool (Excel) | 168.49 KB |
|---|
Citation
Conroy, Kara, Sarah Brunskill, and Amanda Carrillo-Perez (2023). “The Two-Generation Mutual Reinforcement Measurement Tool: Development and Pilot Study Findings.” OPRE Report #2023-149. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- NS2G:
- Next Steps for Rigorous Research on Two-Generation Approaches
- Two-generation initiatives:
- Initiatives that intentionally combine intensive, high quality adult-focused services with intensive, high quality child-focused programs to improve outcomes for children, primary caregivers, and families.
- Mutual reinforcement:
- Service providers aligning and building on each other’s efforts to serve whole families by adopting a shared vision and working toward common or compatible goals.
- Cognitive interview:
- An in-depth interview used to understand the thought process respondents use to answer survey questions. Typically, the respondents do not see the questions in advance and are asked to think aloud as they see the question and formulate their answer. This type of interview aims to find out how respondents understand and interpret questions and arrive at their answers. Researchers use this technique to adjust question wording or item structure, thus making items easier to answer and strengthening the potential validity of the responses.
- Survey debrief:
- An interview after the respondents have completed the survey. This method also identifies how respondents understand the terms and phrases used in individual questions and their overall understanding. This technique helps researchers assess survey length and uncover any issues with the flow of the questions.
- Validity:
- The extent to which a tool is measuring what it set out to measure. (Not calculated for this tool.)
- Reliability:
- The extent to which the tool can consistently reproduce the same results. (Not calculated for this tool).