Analyzing Data on Paired Partners in Program Evaluation: Strategies to Overcome Common Challenges

Publication Date: March 3, 2021
Cover image for Analyzing Data on Paired Partners in Program Evaluation

Download Report

Download Report PDF (856.29 KB)
  • File Size: 856.29 KB
  • Pages: 8
  • Published: 2021

Introduction

Many healthy marriage and responsible fatherhood (HMRF) programs serve paired partners simultaneously, such as couples seeking to improve their relationship or coparents raising a child together. For example, this is often the case when the focus of the HMRF program is to strengthen the quality of the couple’s relationship, and not to change individuals' attitudes, behavior, and decision making about relationships in general. To understand how effective such pair-centric programs are in changing relationships, it is typically necessary to collect data from both partners. This brief describes challenges that evaluators face in correctly analyzing data from paired partners and offers some strategies researchers can consider for addressing them.

Purpose

This brief provides guidance to researchers interested in evaluating programs serving paired partners. It provides strategies for addressing four challenges to correctly analyzing data from pairs:

  1. Statistical tests that don’t adjust for the interdependence of partners’ outcomes will overstate how likely it is that an impact is statistically different than zero; as a result, an evaluation can erroneously conclude that a program had an impact on an outcome.
  2. Analyzing partners’ data separately can give a false sense of the differences between partners’ outcomes or any gender differences in the results if no formal test for these differences is conducted.
  3. Impact estimates for some outcomes can be biased if a program affects the number and type of pairs for whom these outcomes can be observed.
  4. When some follow-up data are missing for one partner in a pair, dropping that pair (and others with the same issue) from the analysis can yield biased results.  

Key Findings and Highlights

Recommended strategies for evaluations of programs serving paired partners include:

  • Account for the interdependency of partners’ outcomes by (1) constructing and analyzing outcome measures at the pair level, or (2) using statistical models that account for the interdependency, such as repeated-measures analysis of variance, multilevel modeling, or structural equation modeling.
  • Think carefully about partner-specific effects and only test for them with forethought and justification.
  • Select outcome measures that are defined for all people rather than those that exclude some people.
  • Collect and use as much data as possible by maximizing survey response rates and selecting an analytic approach that can handle pairs in which only one member is a respondent, such as multilevel modeling or imputation.

Researchers should carefully weigh the strengths and limitations of these strategies in the context of their evaluation. A thoughtful plan for collecting and analyzing data from both partners of participating pairs can improve our understanding of whether and how programs serving pairs are changing the lives of participants.

Methods

The brief includes:

  • An introduction to challenges to evaluating programs serving paired partners
  • An analysis of the implications of four challenges for assessing program impacts for paired partners (see Purpose)
  • Recommended strategies for addressing each of these four challenges (see Key findings and highlights).