Introduction
Research Questions
- What are rapid learning methods?
- What should be considered when designing a rapid learning approach?
- How can federal agencies promote the effective use of rapid learning methods?
On October 25 and 26, 2018, OPRE brought together a diverse group of participants from Federal agencies, research firms, academia, and other organizations for a meeting titled, Rapid Learning Methods for Testing and Evaluating Change in Social Programs . This brief is based on a presentation at the meeting.
Purpose
This brief and the accompanying presentation provide an orientation to rapid learning methods, including: (1) a definition of rapid learning methods, (2) a guiding framework of questions to design an optimal rapid learning approach, and (3) suggested steps that federal agencies can take to promote the effective use of rapid learning methods.
Key Findings and Highlights
Core traits of rapid learning methods include:
- Start with program objectives
- Identify a strategic change to test
- Hypothesize potential improvements
- Determine the appropriate measurement
- Analyze the outcomes and make decisions
Core questions to consider when designing a rapid learning approach include:
- What am I trying to understand?
- What outcomes do I want to change?
- How will I use the results? Who else will use them?
- What are the organization’s priorities and where does this issue fit?
- How confident do I need to be in the results?
- How hard is it to implement the innovation?
- How much will-building and engagement of program delivery staff do I need?
- What data are available?
- How soon do I need to know the results?
- What time is needed to observe an impact? Is there a near-term proxy for that impact?
Recommendations
To promote the effective use of rapid learning methods, federal agencies can:
- Establish expectations around evaluation and learning.
- Combine rapid learning with other forms of program evaluation.
- Write evaluation procurements that foster rapid learning.
- Facilitate trainings on rapid learning approaches.
- Create venues to share results from rapid learning.
Citation
Cody, S., & Arbour, M. (2019). Rapid learning: Methods to examine and improve social programs (OPRE Report 2019-86). Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.