Rapid Learning: Methods for Testing and Evaluating Change in Social Service Programs

On October 25 and 26, 2018, OPRE brought together a diverse group of participants from Federal agencies, research firms, academia, and other organizations to discuss topics related to rapid learning methods.

This brief summarizes the meeting, which explored the following topics:

• Important considerations when selecting and implementing rapid learning methods
• Examples of rapid learning method applications in the field
• Ways rapid learning methods help answer practice- or policy-relevant questions
• Best practices for building a culture of learning within programs and organizations

• Rapid learning methods can provide stakeholders with timely evidence needed to inform ongoing program decisions
• However, stakeholders may be unsure which rapid learning methods are most appropriate for a program’s specific challenges and how best to apply the methods
• Important considerations for selecting a rapid learning method and implementing and testing program changes include:
  • Letting the context determine the methods for rapid improvement
  • Remembering that factors that drive the timeline are context-specific
  • Continuing to learn and improve to respond to shifts in the program environment
• Example applications of rapid learning methods in the field include: developing and using rapid cycle evaluation toolkits and frameworks to test the effectiveness of an intervention, using a continuous quality improvement approach to inform practice, and conducting experimental tests of improvement strategies

• When employing rapid learning methods to help organizations answer practice- or policy-relevant questions, it may be important for researchers to:
  • Customize their tools and strategies with each new partner organization
  • Assess programs’ readiness and commitment
  • Work closely with decision-makers and authority figures within an organization to ensure their ongoing support
  • Use technology to make data more accessible across various partners and teams
  • Help team members to understand data and incorporate it into their decision-making
  • Align their work with decision-making timelines and pressure points
  • Carefully balance feasibility and speed with rigor
• Best practices for building a lasting culture of learning within programs and organizations include:
  • Addressing topics that truly matter to program participants
  • Investing time to coach and build skills among program and organization staff
  • Developing a culture of “all teach, all learn,” in which participants’ voices and expertise are acknowledged and sought out
  • Seeking and fostering leadership support so leaders will in turn continue to support their teams

Holzwart, R., & Wagner, H. (2019). Rapid learning: Methods for testing and evaluating change in social service programs (OPRE Report 2019-57). Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.