Table of Contents | Previous | Next |
Summary of Meta-analytic Findings
Relationship Satisfaction
A statistically significant average effect size for relationship satisfaction indicates that these relationship programs are effective in improving couple satisfaction overall. Looking at differences in effect sizes across characteristics of programs reveals a more complex picture. When program types (therapy, pre-marital, enrichment, education/communication skills, and counseling) are compared to each other, the only two that differ are therapy and pre-marital programs (see Table 2). However, when comparing the effect size for a particular program type to the average effect size of all other program types combined, each program type with the exception of counseling differs from the average of the others. Similarly, when comparing programs on characteristics like total hours or the number of sessions, differences were not detected when comparing individual subgroups to each other, yet differences often emerged when comparing a particular subgroup to the average of the others. For example, one notable pattern is that more sessions, although not necessarily longer sessions, may be more effective at improving couple satisfaction. Yet given these differences only occur when statistical power is increased by comparing the subgroup to the average of all other groups, rather than to individual subgroups, these results should be considered preliminary. More evidence will be needed to detect subgroup differences with confidence.
When only two subgroups are compared, differences can be stated with more confidence because the subgroup sample sizes are larger. For example, studies of programs with treatment group sizes of one couple indicate a greater impact on couples than studies of programs with larger treatment group sizes. And relationship programs appear to be more effective at improving relationship satisfaction for distressed couples than for non-distressed couples.
Relationship Communication
A statistically significant overall average effect size for relationship communication indicates that relationship programs are effective in improving couple communication. Some differences between and among types of programs exist. For example, enrichment programs are less effective at improving couple communication than education and communication skills programs (see Table 3). And the effect size of enrichment programs is significantly less than the average effect size of all other program types combined. The effect size for programs lasting between 1 and 9 weeks is less than the combined effect size of weekend programs and programs lasting longer than 9 weeks, but is not different from these two subgroups when compared individually. Thus, the difference in effect size for programs lasting 1 to 9 weeks should be interpreted with caution, since it only emerges when statistical power is increased by comparing one group to the average of all other groups, rather than comparing subgroups to each other.
Quality Assessment
The studies in this sample tend to be in the upper range in terms of quality, given the rigorous criteria for inclusion. The review’s inclusion criteria required that study authors demonstrate that they either created a treatment and control group through pure or matched random assignment or through a high-quality quasi-experimental process, by which researchers create a control group using advanced statistical analysis. The review also mandates that studies must demonstrate a retention rate from random assignment to post-test analysis of at least 60 percent.
Still, within that range, there is variation with respect to study quality. There are several quality issues and their potential role in introducing a bias in the review or a study that are worth examination.
Random Assignment— All studies demonstrated that they implemented random assignment of some form, but they varied in their methods. Four studies used matched random assignment, and the rest used a pure random assigned strategy. The studies that used a matching design first measured couples on demographic characteristics and/or levels of relationship distress before randomly assigning them to groups. Couples that were similar on these measures were placed into dyads or triads before random assignment to treatment and control groups. The reviewers considered this methodology sufficient to meet the requirements of a high quality study. The rest of the studies used a regular random assignment strategy.
Verifying random assignment— Some studies employed checks to make sure random assignment led to treatment and control groups that appeared similar on measurable characteristics. They checked to see if there were statistically significant differences between the treatment and control groups created in the study. Assuming random assignment was effective, there should be no detectable differences between the two groups. However, because these studies tend to have small sample sizes, it is likely that random assignment will not always produce identical samples. Twenty-one studies checked for statistically significant differences between the treatment and control groups and found none. Five studies detected and noted some differences but did not control for them in their results. The other 12 studies did not report that they checked for differences. It is possible that there are other similarities or differences between the groups that may exist but are not measured by the studies’ authors or differences that simply cannot be measured. The effect sizes for the treatment and control groups were very different at pre-test for many of the studies, suggesting that random assignment was not sufficient for some studies to create identical groups within their small samples.
Attrition— Studies still varied in attrition rates although the baseline retention rate was set at a minimum of 60 percent for inclusion in the review. From the time of random assignment to post-test, 13 studies demonstrated a 100 percent retention rate, 5 studies retained 95-99.9 percent of their sample, 9 studies retained 90-94.9 percent , 6 studies retained 80-89.9 percent , and 6 studies retained 70-79.9 percent . Reviewers dropped 4 studies because they either could not demonstrate a retention rate of at least 60 percent or they specifically reported having a retention rate lower than 60 percent.
Treatment of attrition— Studies also varied in their methods to deal with attrition at post-test. Some studies attempted to follow an “intent-to-treat” evaluation model by considering couples that dropped out of the program at any point during the study as part of their original treatment group. These studies attempted to follow up with such couples in order to include their results in later stages of the evaluation. Of the 26 studies that demonstrated some attrition, there were three studies that attempted this “intent to treat” model. However, for these studies it was unclear as to whether the evaluators were actually able to include some or all of the data for these couples. Of the remaining 25 studies demonstrating attrition, 20 demonstrated in their reports that they did not follow couples that dropped from the program at any point, and in 2 studies it was unclear whether or not researchers followed couples that dropped out of the intervention.
Conflict of interest— There were two types of conflict of interest in these studies. The first is that in some of these studies, the evaluators studied programs that they created. In other studies, evaluators acted as therapists during treatment. In both of these cases, there is a possible bias on the part of the evaluator to inflate results.
In 11 of the 39 studies, the evaluator studied an intervention that he developed. Of these 11 studies, 4 evaluators also acted as a therapist, while the others either did not or the information was not available.
The evaluator acted as either the sole therapist or one of the therapists in 13 of the studies. Three studies specifically mentioned the evaluator was not involved directly in the treatment as a therapist. In 22 studies, the information could not be confirmed either way.
Data Collection— Another possible quality issue arose in the method used for data collection for these studies. Reviewers obtained information on the site and method used for pre-test and post-test data collection for each study. It is assumed that results that are taken onsite and/or by the treatment provider would be biased in the upward direction for the treatment group because the clients might feel pressured to provide a positive evaluation of the program. Clients were evaluated onsite in 15 studies in the pre-test and 16 of the studies in the post-test. In 4 of the studies the therapist or evaluator proctored the client assessments for both the pre-test and post-test measurement periods. Four of the studies had their clients fill out their assessment forms at home for the pre-test, and one study did this for post-test. Aside from this information, all other studies did not report on these items.
Results presentation— Another quality issue relates to the presentation of results. As mentioned before, most studies in this review presented results as means and standard deviations of scale measures. However, there are some studies that did not present this information. These studies either presented effect sizes or data from specific significance tests that were done by authors. A bias may result if these authors presented the results from their study that were statistically significant and positive and did not present results that might not have been so.
Allocation concealment— None of the studies addressed whether or not the individual(s) who implemented random assignment was aware of which group each couple was assigned.
Selection bias— Selection bias appears not to be a problem because all of the studies used random assignment to create their treatment and control groups. Many studies tested for differences between treatment and control groups at pre-test.
Sample sizes— Treatment and control group sample sizes are fairly small for this set of studies. These studies will likely have a greater error variance. The average sample size (treatment and control groups combined) is 34 couples.
Control group contamination— Control group contamination appears not to be a problem for any of these studies because the control groups for all of these studies were held on a wait-list while the treatment groups received treatment.
| Table of Contents | Previous | Next |