Introduction
One approach to program improvement is to use data to help you learn about your program, identify areas for improvement, and set goals. Potential sources of data include your management information system, observations of your program, and feedback from staff members. Another source of data for program improvement is impact evaluations, like the B3 study, that compare the experiences and outcomes of program participants with those of similar populations who do not participate.
Data can help to answer questions about whom you are serving, what the content of each service is, and how many services clients receive. What you learn may help you identify an improvement strategy to apply to your program. Then you can monitor changes and continue reviewing data to learn about how the strategy is working. This infographic presents examples of how programs in the B3 study used data from different sources to gain insights. Based on these data, the next steps would be for the programs to identify specific opportunities to incorporate new strategies and to continue to monitor the data for improvement.
Purpose
This infographic provides a framework to help organizations think about how the data they may already be collecting or could collect can help answer questions about their program or identify areas for improvement. It also provides examples of the ways data can be used for program improvement drawn from the context of the Building Bridges and Bonds (B3) study, a rigorous evaluation of new program approaches to support low-income fathers in working toward economic stability and improved relationships with their children.
Key Findings and Highlights
- Data can help an organization understand three major elements of its program: who is participating in the program, what services are delivered, and how much service is received by participants.
- From the data it collects, an organization can answer questions it has about its program.
Methods
The data presented in this infographic come from the B3 process study and baseline survey. The process study systematically documented the implementation of the Cognitive Behavioral Intervention for Justice-Involved Individuals Seeking Employment (CBI-Emp) and the Just Beginning (JB) interventions. It included a series of semi-structured interviews with staff members, focus groups with fathers, and program observations, as well as information from nFORM, the management information system used by federally funded fatherhood programs. The B3 baseline survey was administered at intake to fathers who enrolled in the study.
These data represent interim findings. Findings collected from the B3 baseline survey for CBI-Emp include fathers randomly assigned as of January 28, 2018. Findings collected from the B3 baseline survey for JB include fathers who were randomly assigned as of February 9, 2018. Findings collected from nFORM include fathers who were randomly assigned as of October 1, 2018.
Citation
Brennan, Emily, Michelle Manno, and Samantha Steimle. (2019). “Using Data to Understand Your Program.” OPRE Report 2019-90. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.