Download ReportDownload Report PDF (2,152.82 KB)
- File Size: 2,152.82 KB
- Pages: 43
- Published: 2021
This report — part of the first phase of the Expanding Evidence on Replicable Recovery and Reunification Interventions for Families (R3) project — describes features of select interventions that use recovery coaches in the child welfare system, characterizes their current stage of readiness for replication and further evaluation, and informs a long-term effort by the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services (HHS) to build high-quality evidence on recovery coaching interventions for families involved with the child welfare system. Section 8082 of the 2018 Substance Use-Disorder Prevention that Promotes Opioid Recovery and Treatment for Patients and Communities (SUPPORT) Act (Pub. Law 115-271) called for HHS to replicate a promising family recovery and reunification intervention that uses recovery coaches, and conduct a three-part evaluation: a pilot study, impact study, and implementation study. As a first step, the R3 research team conducted a systematic scan and identified nine eligible interventions to consider for replication and further evaluation. Using a framework it designed, the R3 research team assessed the readiness of the nine interventions for replication and further rigorous evaluation.
Parental substance misuse is one of the most common reasons families become involved in the child welfare system. Though successful completion of substance use disorder (SUD) treatment is but one of many factors considered in court decisions to reunify children with their parents, those who engage in and complete treatment are more likely to reunify with their children. Recovery coaching emerged in select child welfare systems in the late 1990s and grew throughout the first decade of the 2000s. In the child welfare system, recovery coaches work with parents with SUD who either have had or are at risk of having a child removed from the home. Coaches aim to increase access to and engagement in treatment and other services to support parents’ recovery, coordinate with child welfare agency staff, and ensure treatment and recovery progress is incorporated into plans to either maintain children with their families of origin or place them with other permanent families.
We are just beginning to understand the effectiveness of recovery coaching as strategy for attaining the goals of parental recovery and family reunification. With the field in the early stages of building evidence about this strategy, ACF launched a long-term effort in 2019 to learn more about effective family recovery and reunification interventions that use recovery coaches and help inform policymakers’ and practitioners’ decisions about how to best spend limited resources. This report shares the results of the initial step in ACF’s overall effort to move recovery coaching interventions in child welfare to the next level of evidence.
Key Findings and Highlights
Readiness Factors that Support Implementation
Of the nine recovery coaching interventions assessed by the research team, Oregon Parent Mentor Program and Sobriety Treatment and Recovery Teams (START) received the highest ratings for potential replicability and scalability.
- Both interventions currently operate in multiple jurisdictions, have some technical assistance and training infrastructure, and have well-documented procedures and detailed manuals.
Most did not have a comprehensive manual or a cohesive set of materials for implementing the intervention.
- The recovery coaching interventions that lacked strong documentation may be theoretically sound and worth further development and evaluation. However, among these nine interventions, those that were well defined and had a foundational set of materials and other implementation supports were better positioned for the next stage of evidence building.
Most of the nine recovery coaching interventions had clearly articulated logic models or theories of change and demonstrated high applicability to child welfare.
- Six of the interventions demonstrated high applicability to child welfare, primarily because they were designed with that system in mind.
- Three interventions, Massachusetts FRESH Start, Recovery Specialist Volunteer Program (RSVP), and Summit Co. STARS, were no longer operating at the time of this report.
Readiness Factors Related to the Strength of the Available Evidence
Together with the potential for replication and scaling, the strength of the available evidence is an important consideration in determining readiness. Strength of the available evidence is based on both the quality of studies’ design/analysis and the direction of findings.
There was limited evidence on which to assess this factor. Thus, most of the recovery coaching interventions had “low” readiness in terms of the quality and findings of prior evaluations.
Three interventions showed some promise based on the strength of their available evidence:
- The Family Recovery and Reunification Program was the only one of the nine interventions with prior favorable findings from an experimental study in any of the primary outcome domains (i.e., permanency, child safety, parental substance use, and SUD treatment compliance).
- Oregon Parent Mentor Program was the only intervention besides Family Recovery and Reunification Program to be previously evaluated with an experimental study design. That evaluation suggested neutral findings (i.e., no statistically significant effect on primary outcomes) from a small sample that did not rule out potentially larger effects. We consider these results to be inconclusive due to the limitations of the design.
- The only other intervention supported by a mix of neutral and favorable findings (and no unfavorable findings) was START, which used a quasi-experimental study design.
The other six interventions were limited by evaluations with small samples, unclear matching procedures and non-equivalent study groups, and evaluation designs not intended to isolate the effect of the recovery coaching component from that of other service components. These challenges made it difficult to draw conclusions about the strength of the evidence on those interventions.
The research team conducted a systematic scan for recovery coaching interventions and assessed the identified interventions on their readiness for future replication and evaluation. The scan identified 1,594 potentially eligible interventions. Of those, we confirmed that nine interventions met eligibility criteria to be considered for replication and evaluation. Using a pre-specified rubric, we assessed the readiness of these nine interventions on six factors: the clarity of its underlying logic, potential for replication, potential for scaling up, applicability to the child welfare services environment, the quality of prior evaluation design and analysis, and the direction of prior evaluation findings.
The aims, methods, and results of the scan and readiness assessment process are summarized in the Recovery Coaching Interventions for Families Involved with the Child Welfare System: Moving Toward Evidence-Based Practices report. These appendices to the report provide supplemental technical detail.
The appendices are organized into three sections. Appendix A provides supplemental details about the literature scan and readiness assessment process described in Section 3 of the report. This detail includes eligibility criteria for the scan, detail on the search parameters used in the scan, and the disposition of specific interventions and programs resulting from the scan. Appendix A concludes with an explanation of the readiness assessment criteria used to rate the design and analysis quality of the evidence on each intervention.
Appendix B contains descriptive profiles of the nine interventions that were included in the readiness assessment described in Section 4 of the report. The profiles summarize the readiness assessment results for each intervention and describe their characteristics. Last, Appendix C contains a list of evaluations reviewed for the assessment of evidence, organized by intervention.
|File Type||File Name||File Size|
|Recovery Coaching Interventions for Families Involved with the Child Welfare System: Technical Appendices||1,784.51 KB|
Francis, Kimberly, Jessica Thornton Walker, Jill Hamadyk, and Sandra Jo Wilson (2021). Recovery Coaching Interventions for Families Involved with the Child Welfare System: Moving Toward Evidence-Based Practices, OPRE Report 2021-53. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
- Substance Use Disorder