Introduction
Rapid cycle evaluation (RCE) approaches are attracting growing attention as a way to efficiently assess the implementation of interventions and to inform program improvement. RCE is an umbrella term that is used and defined broadly. Many methods may be suitable to use within the RCE framework. All involve iterative data collection to answer specific questions quickly, often to support program improvement.
RCE approaches use interim data in iterative and formative ways to track progress and improve programs along the way. Programs can assess, modify, and improve interventions in real time while maintaining the integrity of the study, and then use the final findings to deliver precision approaches that improve outcomes for children and families after the study concludes.
This work is part of the Design Options for Home Visiting Evaluation (DOHVE) project, led by OPRE in collaboration with HRSA. ACF has partnered with JBA to conduct the DOHVE project.
Purpose
RCE may help Maternal, Infant, and Early Childhood Home Visiting (MIECHV) awardees test program changes quickly and rigorously. The purpose of this brief is to introduce MIECHV awardees to RCE and its potential use in their programs.
Highlights
This brief—
- Defines RCE
- Compares RCE with traditional research designs
- Describes methods that can be used in RCE
- Provides an example of how RCE can address a research question
- Outlines steps to prepare for RCE
- Lists potential benefits of RCE
- Provides additional resources
Citation
Atukpawu-Tipton, G., & Poes, M. (2020). Rapid cycle evaluation at a glance. OPRE Report # 2020-152. Office of Planning, Research, and Evaluation, U.S. Department of Health and Human Services. Produced by James Bell Associates.
Glossary
- Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program:
- Administered by the Health Resources and Services Administration in partnership with the Administration for Children and Families, the MIECHV Program was established in 2010 to support voluntary, evidence-based home visiting for at-risk pregnant women and parents with children up to kindergarten entry. The program provides grants to states, US territories, and tribes, which conduct needs assessments to identify eligible at-risk communities and serve priority populations.
- Rapid cycle evaluation (RCE):
- RCE is not a specific design or method. Rather, it is an approach to evaluation that relies on innovative designs and methods. Common hallmarks of RCE approaches include rapid cycle analysis and formative process improvement.
- Rapid cycle analysis:
- Quickly assesses the effectiveness of program components and facilitates ongoing feedback to staff to support program improvement.
- Formative process improvement:
- Allows for a careful review of interim data and modifications to the study design for optimal implementation to achieve desired outcomes.
- Continuous quality improvement (CQI):
- Uses cycles of planning, testing, refining, and retesting (sometimes referred to as Plan-Do-Study-Act cycles) to examine improvements to a system or process. Data are collected continuously, and changes are assessed over time.
- Idiographic controlled trials (ICTs):
- Quasi-experimental designs that compare results during and after an intervention to a baseline period. ICTs use statistical techniques that allow for the inclusion of certain variables and the assessment of effect size.
- Bayesian adaptive trials:
- Combine prior information (i.e., information from previous trials, scientific research, expert opinion) with emerging study data to make timely and efficient inferences about participant outcomes.