Introduction
To expand the evidence base on interventions to prevent homelessness among youth and young adults who have been involved in the child welfare system, the Administration for Children and Families (ACF) of the U.S. Department of Health and Human Services launched the Youth At-Risk of Homelessness (YARH) multiphase grant program. This program specifies three primary populations: (1) adolescents who enter foster care from ages 14 to 17; (2) young adults aging out of foster care; and (3) homeless youth and young adults, up to age 21, with foster care histories.
In 2019, ACF contracted with Mathematica for the third phase of YARH (2019 — 2028, known as YARH-3), which provides information to the field on how to better serve youth and young adults through a rigorous summative evaluation. ACF’s goal for YARH-3 is to produce evidence about interventions intended to prevent homelessness and improve key outcomes among youth and young adults who have been involved in the child welfare system. The summative evaluation conducted under YARH-3 will examine the effect of Colorado’s Pathways to Success (Pathways) comprehensive service model through implementation and impact studies. The Pathways service model offers intensive, coach-like case management for youth and young adults with foster care histories at age 14 or older.
The YARH team, in consultation with ACF, is releasing a series of analytic plans for the summative evaluation. This is the first — or foundational — analytic plan that describes the evaluation design in detail and presents the analytic methods to date. Additional analytic plans will be released that focus on the methods for a specific product. Future analytic plans will be brief and reference this foundational analytic plan.
Purpose
This analytic plan serves as the foundation for a summative evaluation of the Colorado Pathways to Success comprehensive service model that Mathematica and ACF are conducting as part of YARH-3 in partnership with the Colorado Department of Human Services and the Center for Policy Research. This plan describes the implementation study analytic approach, the impact study analytic approach, and the research questions guiding the studies and the data sources that will inform analyses. This is one of a series of public documents describing and documenting the summative evaluation.
Key Findings and Highlights
The YARH-3 summative evaluation includes an implementation study and an impact study. The implementation study will use multiple methods to collect, analyze, and report on comprehensive data to address the study’s two broad objectives, supporting interpretation of Pathway’s impacts on outcomes and assessing Pathways implementation. Mathematica will collect data from a range of stakeholders using interviews and administrative data. Mathematica will take a structured approach, guided by the conceptual framework, to analyze the data across Pathways sites.
The impact study will be the first rigorous impact evaluation of Colorado’s Pathways comprehensive service model. The impact study will conduct traditional inferential tests. It will examine evidence of program effectiveness on a large number of policy-relevant outcomes, including stable housing, education, employment, permanent connections to caring adults, and social-emotional well-being. It will explore the effectiveness of Pathways in short- and long-term follow-up periods and estimate the extent to which the program is more or less effective for key subgroups. Finally, the study originally proposed to explore linking features of program implementation (for example, dosage, quality, or adherence of the program delivery) to youth and young adult outcomes. However, this last analysis might no longer be feasible, given the limited variability in youth and young adult Pathways experiences and projections for the number of youth and young adults expected to enroll in Pathways by the end of the study.
Citation
Fung, Nickie, Robert Lynn-Green, Megan McCormick, Elizabeth Mugo, and M.C. Bradley (2023). “Youth At-Risk of Homelessness: Foundational Analytic Plan for the Summative Study Evaluation of Pathways to Success.” OPRE Report # 2023-059, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- Coach-like case management:
- Involves the youth and young adults in every aspect of their development. The model emphasizes coaching practices to engage youth and young adults, and a youth-driven approach to help identify their goals, connect them with existing services, and promote positive outcomes.
- YARH:
- Youth At Risk of Homelessness, acronym used to represent the initiative funded by Administration for Children & Families to support communities in addressing homelessness among youth and young adults with child welfare involvement