Table of Contents | Previous | Next |
Meta-analysis: Outcomes and Analysis Methods
Effect Size Creation
Reviewers created an effect size for satisfaction, communication, or both for each study. They analyze the two effect sizes separately. Of 39 studies, 28 measured relationship satisfaction, and 13 measured communication. Reviewers created a type of effect size called the standardized gain score difference. To create this effect size, they took the gain score (post-test score minus pre-test score) for the control group, subtracted it from the treatment group gain score, and divided the treatment/control difference by the pooled pre-test standard deviation. 2
Interpreting the Effect Size Statistic
A positive effect size indicates that the treatment group had a larger increase than the control group between the pre- and post- intervention periods in the measure observed. For example, in one study that measures couples’ satisfaction with the DAS and has an effect size of .5, the control group gained .94 DAS points (93.06 to 94) and the treatment group gained 12.5 DAS points (88.1 to 100.63). In a similar study with a much larger effect size of 1.5, the control group gained 1.2 DAS points (66.9 to 68.1) and the treatment group gained 19.8 DAS points (76.6 to 96.4). A widely used convention for appraising effect sizes was established by Cohen (1977, 1988), who reports that the standardized mean difference effect sizes fall into the following ranges over a wide range of behavioral science research: small (ES<= .20), medium (ES=.50), and large (ES>=.80).
Weighting Strategy
The effect sizes are weighted by the inverse of the study variance for each study. The smaller the variance in a study’s sample, the more precise the study is likely to be, and thus the larger weight it receives in the overall effect size. Reviewers calculate the mean effect sizes using these weights. They chose a random effects model weighting strategy because the studies failed the test for homogeneity. This indicates that the variability among the effect sizes is greater than what is likely to have resulted from subject-level sampling error alone. However, the alternative weighting strategy— a fixed-effects model— produces identical mean effect sizes.
Results
The reviewers created an effect size (standardized mean gain difference) for both relationship satisfaction and communication.
Relationship Satisfaction
The average effect size for relationship satisfaction is .68. This finding is based upon a sample size of 28 studies because only 28 of the 39 studies in this review measured relationship satisfaction. This effect size is statistically significant, meaning that the impact of the intervention on the treatment group is different from the impact on the control group. Reviewers used an ANOVA for all tests of statistical significance. Average effect sizes for satisfaction are available in Table 2.
Relationship Communication
The average effect size for the total sample of 39 studies for relationship communication is .26 (N=13 studies). This effect size is statistically significant. Average effect sizes for communication are available in Table 3.
Group differences
The reviewers present the effect sizes for certain groups below to examine how the effect size varies by characteristic in the studies (see Tables 2 and 3). In cases where the sample sizes of studies do not add up to the total studies available for each measure (satisfaction and communication), there is missing information on the specific characteristics (program type, hours in program, etc.) from each study left out of the analysis.
Program Type— Satisfaction
The mean effect size for studies of therapy programs is .86 (N=15), pre-marital preparation is .08 (N=1), enrichment is .23 (N=2), education / communication skills is .58 (N=8), and counseling is .94 (N=2).
Program Type— Communication
The mean effect size for studies of therapy programs is .38 (N=5), pre-marital preparation is .11 (N=2), enrichment is -.43 (N=2), and education / communication skills is .51 (N=4). There were no studies of counseling programs that measured relationship communication.
Total hours in program— Satisfaction
The mean effect size for studies of programs lasting fewer than 10 hours is 1.17 (N=2), between 10 and 15 hours is .71 (N=13), and over 15 hours is .59 (N=10).
Total hours in program— Communication
The mean effect size for studies of programs lasting between 10 and 15 hours is .12 (N=6) and .37 for studies over 15 hours (N=7). No evaluations of programs lasting fewer than 10 hours measured communication.
Number of sessions— Satisfaction
The average effect size for programs with fewer than 5 sessions is .06, between 5 and 10 sessions is .66, and 12 sessions or more is .98.
Number of sessions— Communication
The average effect size for programs with fewer than 5 sessions is .36, between 5 and 10 sessions is .11, and 12 sessions or more is .45.
Session length— Satisfaction
The average effect size for programs that have hour-long sessions is .88, for 2- to 4-hour long sessions it is .47, and for programs with sessions lasting 8 hours is .15.
Session length— Communication
The average effect size for programs that have hour-long sessions is .20, for 2- to 4-hour long sessions it is .18, and for programs with sessions lasting 8 hours is .49.
Number of weeks— Satisfaction
The average effect size for weekend-style programs is .15, for programs lasting between 1 and 10 weeks is .55 and .90 for those between 10 and 15 weeks.
Number of weeks— Communication
The average effect size for weekend-style programs is .49, for programs lasting between 1 and 10 weeks is -.04 and .45 for those between 10 and 15 weeks.
Level of couple distress— Satisfaction
The mean effect size for studies with a distressed-couple sample is .94 (N=18) and .43 for a non-distressed sample (N=7).
Level of couple distress— Communication
The mean effect size for studies with a distressed-couple sample is .39 (N=6) and -.12 for a non-distressed sample (N=5).
Number of couples receiving treatment in each session— Satisfaction
In this section of the report, the reviewers consider “treatment group size” to be the number of couples that are present during a therapy / education session in the programs. The mean effect size for studies of programs with a treatment group size of more than one couple is .40 (N=7) and .80 for one couple (N=19).
Number of couples receiving treatment in each session— Communication
The mean effect size for studies of programs with a treatment group size of more than one couple is .29 (N=6) and .18 for one couple (N=6).
Attrition Rates
The reviewers examine average effect sizes for studies that exhibit high retention (80 percent and higher). All of the studies that examine communications skills already fit this description. For programs measuring satisfaction, there are 24 that exhibit retention rates of 80 percent or higher and 4 that do not (they have between 60 percent and 79 percent retention rates). The high retention studies have an average effect size of .65, while those with lower retention have an average effect of .99.
2 Many studies presented data for men and women separately. When this occurred, the reviewers combined the two groups and created a single effect size based on the combined results. All studies that reported data at the individual level as opposed to the couple level provided information on both members of the couple. The reviewers did not find any cases where data were only given for one member of the couple. [back]
| Table of Contents | Previous | Next |