Download ReportDownload Report PDF (1,349.75 KB)
- File Size: 1,349.75 KB
- Pages: N/A
- Published: 2020
The Strengthening Relationship Education and Marriage Services (STREAMS) evaluation is a random assignment impact study and in-depth process study of five healthy marriage and relationship education (HMRE) grantees funded by the Office of Family Assistance (OFA) within the Administration for Children and Families (ACF). To maximize its contributions to the evidence base and to inform future program and evaluation design, STREAMS is examining the full range of populations served by HMRE programs, including individual adults, adult couples, and youth in high school. Each STREAMS site functions as a separate study within the larger evaluation and each site addresses a distinct research question.
This process study report presents findings on the implementation of the University of Florida’s (UF) ELEVATE program in six counties across the state. UF delivered ELEVATE through its Institute of Food and Agricultural Sciences (IFAS) county-based Cooperative Extension Service (Extension). ELEVATE is a popular curriculum for adult couples of all ages who may or may not be married. It was developed as part of the Alabama Healthy Marriage and Relationship Education Initiative for delivery through Extension and other organizations. The curriculum has two primary goals: (1) to teach couples practical strategies and tools to maintain a healthy relationship and (2) to develop mindfulness practices that help couples regulate their physiological responses to conflict and stress.
ACF’s Office of Planning, Research, and Evaluation (OPRE) with funding from OFA contracted with Mathematica and its partner Public Strategies to rigorously evaluate a text-messaging intervention designed to boost attendance in ELEVATE. Due to the focus of the impact study on couples’ attendance, in this report we emphasize the strategies UF used to recruit couples and encourage their attendance and participation. We also highlight other aspects of implementation, including strategies for delivering a statewide program through Extension and characteristics of the counties in which ELEVATE operated.
This process study examines (1) the context for implementing ELEVATE; (2) procedures for supporting ELEVATE implementation through Extension; (3) UF’s processes for recruiting and enrolling couples; and (4) couples’ participation and engagement in the program.
Key Findings and Highlights
Findings from this process study will provide context and help interpret the text-messaging intervention findings. Key findings are:
- Implementing a statewide program through Extension required system building and cultural shifts. ELEVATE differed from typical Extension programs, which were often one or two days long with no specific enrollment targets or performance monitoring requirements. Overseeing a longer program that involved meeting enrollment targets, entering data into nFORM, and meeting specific performance goals required a different approach to supervision and program management. UF took several steps to support county Extension agents and increase their involvement in program decision making. For example, project leaders convened a monthly meeting with agents and looked for opportunities for agents to publish information and make presentations on UF’s HMRE program. At the time of the site visit, UF also planned to hire a dedicated staff member to provide onsite support to each county on a rotating basis. UF’s data manager provided extensive support to ensure that UF stayed on top of its reporting requirements despite the geographic distance between sites.
- UF used data and technology to adapt its recruitment and enrollment strategies as needed. UF carefully monitored data on recruitment and enrollment and used this data to inform its strategies. Project leaders worked with the STREAMS evaluation team to set enrollment targets for each county and the program as a whole. UF then divided these targets among the program instructors and held each instructor accountable for enrolling a certain number of couples in each workshop. The data manager tracked progress toward these targets and shared this information regularly with project leadership and frontline staff, which allowed everyone on the project team to work together to improve enrollment as needed. UF also leveraged technology to boost enrollment. In response to low enrollment in the program’s early months, UF decided to advertise on social media, in addition to other methods like distributing flyers and attending community events. By the end of 2017, UF was meeting its enrollment targets. Over half of the participants reported learning about ELEVATE through an Internet ad, including those placed on social media.
- ELEVATE engaged couples with highly relevant content and flexible options to make up missed sessions. Overall, program participants were very satisfied with ELEVATE. Participants reported that the program’s content was highly relevant to their lives. They especially appreciated learning about personal triggers, the ways that stress and anger can interfere with their ability to communicate effectively, and practical skills they could use for resolving conflicts. Recognizing that changing work schedules and other conflicts made it difficult for some couples to attend all five workshop sessions, UF offered two options for couples to make up missed sessions—in-person and online. The make-up options greatly improved program attendance, boosting the percentage of couples who attended all five sessions from 44 to 63 percent.
This report is based on analysis of data from the following four sources, collected to document the implementation of ELEVATE in the first two years of the program (January 2017 – December 2018).
- Semi-structured interviews, focus groups, and observations. We conducted a three-day site visit in September 2018. We interviewed 12 staff, observed two sessions of ELEVATE, and held two focus groups with couples who had attended the workshop.
- nFORM attendance and service data. nFORM is the client management system that OFA provided to UF and other 2015 grantees. UF staff entered workshop attendance and other data about service delivery in the system. After each ELEVATE workshop session, UF program instructors completed a short assessment about their ability to deliver that day’s content and engage participants.
- Applicant characteristic and entrance survey data. Participants completed an applicant characteristics survey and entrance survey at the first session of ELEVATE. The surveys gathered information on participant demographics and relationship characteristics.
- Staff survey. Seven program staff who facilitated ELEVATE completed a web-based survey in September 2018. This survey included questions about their work roles and experiences; feelings toward the program; and impressions of the quality of their supervision, of training opportunities, and of the organizational climate.
Alamillo, Julia, Diane Paulsell, and Daniel Friend. (2020). ELEVATE: Implementation of a healthy marriage and relationship education program by a statewide cooperative extension service. OPRE Report # 2020-99. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.