Text Message Reminders and Their Impact on Attendance at Healthy Marriage and Relationship Education Workshops

This report describes the impacts of text message reminders on couples’ continuing attendance and initial attendance at UF’s ELEVATE group workshop sessions. These impacts were estimated by comparing the attendance outcomes of more than 1,700 couples that enrolled in the study as part of the ELEVATE program. To estimate impacts, we randomly assigned each couple to receive a different type of text message reminder or to a control group that did not receive reminders. In addition to describing these impacts, the report also summarizes earlier evidence on the effectiveness of text message reminders, describes the process we used to develop different text message reminders, and documents the study design and methods. An earlier report provided more detailed information on the design and implementation of the ELEVATE program during the first two years of the impact study.  Mathematica and Public Strategies conducted this study as part of the Strengthening Relationship Education and Marriage Services (STREAMS) evaluation for ACF.

• We found no evidence to suggest that any one of the text message reminders was more effective than others (or sending no reminders) in increasing continued session attendance. Among couples who attended the first workshop session, attendance rates were high for the rest of the workshop sessions after the first one and were similar across all research groups. Most couples attended three additional sessions after the first one and missed no more than one session over the course of the workshop.
• We found that simple text message reminders that provided only the date and time of the first session increased couples’ initial session attendance by about seven percent points. Among couples who registered for a workshop, many did not attend the first workshop session. Only 58 percent of couples who did not receive reminders attended the first session. However, the attendance rate at the first session was higher (65 percent) for couples who received simple reminders. The simple reminders also outperformed two enhanced reminders we had designed for this study that included motivational messages, graphics, and peer testimonials.

Study enrollment occurred over a three-year period from January 2017 to February 2020. For the first two years of the study, couples were enrolled at the first ELEVATE workshop session and randomly assigned to either a control group that received no reminders or one of several study groups that each received a different type of text message reminder. We compared couples’ attendance outcomes across groups to estimate impacts on couples’ continuing session attendance. For the last year of the study, the timing of study enrollment changed to happen when couples first registered for the workshop. We used attendance data from this period of the study to estimate the impacts of different text message reminders on couples’ initial session attendance.

Throughout the study, both members of the couple had to provide consent to the STREAMS evaluation and provide valid phone numbers to receive text messages, and neither member could have previously attended an HMRE workshop offered by UF. To estimate impacts more efficiently than would be possible with a traditional randomization procedure, we used a Bayesian adaptive design to randomly assign couples to research groups. This design involved conducting interim analyses of the study outcome data to make mid-course changes to the random assignment procedures and inform how future study participants were randomly assigned.

Our results suggest that text message reminders can be effective in increasing attendance at HMRE group workshop sessions but that providers must pinpoint the source of the leak in the attendance pipeline and then target reminders accordingly. As this study unfolded, we learned that (1) among couples that attended the first session, continued attendance was high at remaining sessions and (2) text message reminders did not further increase attendance. When we interviewed program staff and participants to assess why some couples missed a session after the first one, we found that the reason usually involved something outside the scope of a text message reminder. We also learned that the largest drop in attendance occurred between workshop registration and the first session. In response, we shifted the study’s focus to improving initial attendance among workshop registrants and found that simple text message reminders increased the initial attendance rate. Our experience highlights the importance of looking at attendance patterns to identify the source of couples’ difficulties with initial or regular attendance and then directing the reminder messages accordingly.

The findings from this study also suggest that simple reminders may work better for HMRE program participants as compared to messages with enhanced content such as motivational messages, graphics, or peer testimonials. This study does not allow us to conclude why the simple reminder was most effective at increasing initial attendance, but we can suggest a few possibilities. Given the personal nature of romantic relationships, it might be difficult to design enhanced content that captures couples’ many possible motivations for attending HMRE group workshop sessions. Capturing motivations might be particularly tough for programs that serve diverse populations, as ELEVATE did. It is possible that some HMRE providers know their clients well enough or serve a narrow enough population that they could craft carefully worded text message reminders that resonate with everyone. For most HMRE programs, however, our findings from ELEVATE suggest that it may be advisable to keep the content simple.

Future research should explore whether and how these findings generalize to other populations, programs, and methods for sending text message reminders. ELEVATE offered five workshop sessions to a demographically diverse mix of couples from six counties in different parts of Florida. Although a five-session workshop is about average in length for recent HMRE grantees, some programs offer much longer workshops or additional content and services. In addition, as compared to clients served by other federally funded HMRE programs, the ELEVATE participants had higher education levels, lower levels of economic insecurity, and one fewer child, on average. These factors could limit the generalizability of our study findings to other programs and contexts. We sent the text messages using an automated software system. Although the system was easy to use and efficient, it did not allow for interactive dialogue, which might have led some couples to perceive the messages as impersonal. Future research should seek to build rigorous evidence on the impacts and cost-effectiveness of more personalized approaches for sending text message reminders.

Patnaik, Ankita, Jonathan E. Gellar, Rebecca Dunn and Brian Goesling (2021). Text Message Reminders and Their Impact on Attendance at Healthy Marriage and Relationship Education Workshops, OPRE Report #2022-49, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.