Introduction
Research Questions
- What grantee and workshop characteristics predict participation in HMRF primary workshops?
- What participant demographics predict participation in HMRF primary workshops?
- What interactions exist between grantee and workshop characteristics and client demographics that predict participation in HMRF primary workshops?
Since 2006, the Office of Family Assistance (OFA), within the Administration for Children and Families (ACF), U.S. Department of Health and Human Services, has awarded and overseen federal funding for four cohorts of healthy marriage (HM) and responsible fatherhood (RF) grant programs (2006-2011, 2011-2015, 2015-2020, and 2020-2025). OFA works with the Office of Planning, Research, and Evaluation (OPRE), also within ACF, to research how to best serve families through these grants.
HMRF programs are generally required to offer a primary workshop aimed at improving healthy relationship and parenting knowledge and skills. RF programs also offer employment and economic stability opportunities. Yet HMRF programs often face challenges with consistent participant attendance in their primary workshops. To understand more about what factors might predict greater participation in primary workshops, ACF asked Mathematica to produce a special topic report (STR) to explore this topic in depth. The resources produced as part of this STR share how the analysis was conducted and highlight key findings of what was learned.
This work was part of the Building Usage, Improvement, and Learning with Data in HMRF Programs (BUILD HMRF), led by OPRE in collaboration with OFA. ACF has contracted with Mathematica to conduct the BUILD HMRF project.
Purpose
This research-to-practice brief describes findings from the STR about what workshop- and participant-level factors might predict greater participation in HMRF primary workshops. Appendix A of the brief presents scenarios illustrating predictors of workshop completion; Appendix B is a technical appendix. Mathematica also shared findings from the STR in a webinar with grantees. These products share information with HMRF practitioners, local evaluators, and others about the predictive methods used, key findings, potential uses of the results, and next steps, including considerations for ensuring that results are used equitably to help all HMRF program participants.
Key Findings and Highlights
Overall, the results from this study highlight the importance of context when determining how to best structure a workshop series. The study found that there are numerous ways that participant and workshop characteristics interact to predict the likelihood that a participant will complete a workshop series. Across grantees, the model predicted that workshops with fewer, longer sessions would have higher completion rates for grantees’ typical participants. While the performance and predictive power of the models indicate that they can be useful to HMRF practitioners and staff involved in program design and improvement, these types of models can be difficult to interpret. To help illustrate these findings, we created a range of visual scenarios (which are presented in Appendix A) for participants typically served by grantees in the models. Each scenario highlighted the set of optimal workshop characteristics that predict the highest probability of the typical participant completing the workshop, as well as the set of suboptimal workshop characteristics that predict the lowest probability of the typical participant completing the workshop.
Methods
This study used data collected from the 2015 cohort of HMRF grantees, focusing on grantees that served HM adult couples (21 grantees, approximately 10,000 partners served as couples), HM adult individuals (22 grantees, approximately 7,900 individuals), and RF community individuals (17 grantees, approximately 5,700 individuals).
To answer our research questions, we used a specific type of tree-based predictive algorithm called random forest modeling. Tree-based algorithms are data driven and make predictions by using a decision rule in the form of a series of yes-or-no questions that are applied to a set of predictor variables. Using this method, we fit three models, one for each population, that used participant and workshop characteristics to predict participant completion of the first primary workshop they attended.
Citation
Friend, Daniel, Derekh Cornwell, Arielle Marks-Anglin, Avery Hennigar, James Troxel, and Grace Roemer. “Predicting Participation in Healthy Marriage and Responsible Fatherhood Programs.” OPRE Report #2023-125. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2023.