Introduction
A road test is an iterative, rapid prototyping approach used to refine the design and implementation of a program strategy or intervention. This analytic piloting process involves multiple cycles of gathering formative feedback, adjusting the design, and strengthening the implementation of a strategy prior to scaling it up. By using this accessible approach to vetting programmatic changes, human services practitioners can clarify and strengthen the linkages between a program strategy and its anticipated outcomes and more precisely identify the necessary conditions for successful implementation. The road test is part of the Learn, Innovate, Improve (LI2) process for using and producing evidence in the course of program change.
Pilot testing is a common practice in human services programs (such as workforce development and employment services, safety net programs, child welfare services, early childhood education programs, and healthy family programs, among others), yet programs can often do more to maximize learning from the experience of trying something new. In cases where a strategy or intervention is rolled out without intentional and incremental refinement, the program change might ultimately be abandoned due to complications or perceived ineffectiveness, resulting in wasted energy and resources.
Purpose
This brief serves to explain the road test process within the context of a larger systematic and evidence-informed framework for program improvement, provide practical guidance for using this approach in human services programs, and describe concrete examples of road tests.
Key Findings and Highlights
- A road test is a component of the broader LI2 process and relies upon a thoughtful diagnosis and design process. When coupled with this iterative and rapid prototyping, LI2 can offer programs a new systematic – yet flexible – way of improving their services. A road test falls within the Improve phase of LI2.
- Road tests typically involve small numbers of program staff and clients who try out a new strategy or intervention over a short period of time (about four to six weeks) and provide structured feedback about their experience. This feedback is used to generate concrete recommendations for refining or revising the strategy.
- Each road test typically involves two or three feedback periods, known as learning cycles. Between each feedback period, the strategy or intervention is revised in preparation for further vetting in the subsequent period.
- A road test should be anchored in a detailed plan, called a road map, which specifies the causal linkages between strategies, targeted changes, outcomes, and expected contextual influences.
- A road test is not meant to be an in-depth research study; therefore, collecting and analyzing information as a part of this process should not be cumbersome or lengthy. Programs can conduct road tests on their own, but should consider partnering with external researchers if they do not have strong internal design and analysis capacity.
Related Documents
Using a “Road Test” to Improve Human Services Programs: A Primer.pdf (PDF)
| File Type | File Name | File Size | Using a “Road Test” to Improve Human Services Programs: A Primer.pdf | 91.26 KB |
|---|