Evaluation and Monitoring 101

The purpose of this brief is to spread awareness about OPRE’s work, highlighting one way that OPRE supports ACF’s capacity to build and use evidence. The information in this brief may be useful for ACF staff who are considering participating in the training, ACF program leadership who are interested in sending staff to the training, and/or other agencies who are contemplating launching their own such training to build capacity for learning and evaluation.

The Evaluation and Monitoring 101 training has evolved over nearly a decade to meet the changing needs of Administration staff and address a growing federal interest in evidence-based decision making. In the April 2022 training, learners participated in seven interactive sessions over the course of two days, guided by skilled facilitators from a range of OPRE divisions. Session topics included:

1. Introduction to OPRE
2. Logic models and research questions
3. Performance measurement
4. Continuous quality improvement
5. Process and implementation evaluations
6. Outcome and impact evaluations
7. Planning and implementing program assessment approaches

The training rooted all content in a framework that distinguishes between two categories of program assessment: (1) Performance monitoring and improvement, which refers to ongoing monitoring of program performance, and (2) Program evaluation, which refers to systematic studies to assess how well programs are working. Each category of program assessment is associated with different methods that can help staff answer different types of questions to advance the ACF mission.

OPRE is committed to incorporating participant feedback to continuously improve the Evaluation and Monitoring 101 training and collects participant feedback both during and after each training course. OPRE staff report that the Evaluation and Monitoring 101 training has led to meaningful, sustained collaboration with program offices. The training may also serve as a model for other agencies looking to improve data collection and analysis practices to ultimately improve service delivery.

To better understand the history, context, and learning design of the Evaluation and Monitoring 101 training, the Mathematica team started by reviewing and analyzing available materials from past trainings, including past course materials and feedback forms. The Mathematica team also gathered historical information through a group discussion with OPRE staff who have been involved with the training design and implementation over time.  

Then, a Mathematica team member participated in the April 2022 training to gain first-hand insight into the training content, facilitation methods, and learning outcomes. The team reviewed and analyzed themes of feedback forms from the 2022 training¹ to understand the participant experience and assess which course components resonated most with learners. The Mathematica team is grateful to OPRE for sharing past and current course materials, offering insight into the training’s history, and supporting Mathematica’s participation in the 2022 training.

Alberty, Elizabeth, and Heather Zaveri. “Evaluation and Monitoring 101: The Office of Planning, Research, and Evaluation, within the Administration for Children and Families, builds staff skills to assess program performance and impact.” OPRE brief 2022-232. Washington, D.C., Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

DiSalvo, C., and C. Bridges. “Building ACF Evidence Capacity with Evaluation and Monitoring 101.” OPRE Insights Blog, May 17, 2021. Available at https://www.acf.hhs.gov/opre/blog/2021/05/building-acf-evidence-capacity-evaluation-and-monitoring-101. Accessed May 2, 2022.