Health Profession Opportunity Grants (HPOG) Impact Study

This document describes the Analysis Plan for evaluating the impacts of the Pathways for Advancing Careers and Education (PACE) project and the first round of Health Profession Opportunity Grants (HPOG 1.0) on key outcomes nine years after random assignment into the program.

Three programs increased college credential receipt and one program had earnings impacts at the six-year follow-up.

This document reports on the impacts of the Health Profession Opportunity Grants (HPOG 1.0) Program that arose after about six years.

Out of the 10 evaluations (9 PACE evaluations plus the HPOG 1.0 Impact Study), most programs had large impacts on credential receipt and more modest impacts on training duration but only one had earnings impacts.

This report presents the plan for evaluating the six-year impacts of the HPOG 1.0 grants on various outcomes. It describes the operationalization of outcome measures, including their source data.  

In 2010, the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services awarded the first round of five-year Health Profession Opportunity Grants (HPOG 1.0) to 32 organizations in 23 states; five were tribal organizations. The purpose of the HPOG Program is to provide education and training to Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals for occupations in the healthcare field...

In 2010, the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services awarded the first round of five-year HPOG grants (HPOG 1.0) to 32 organizations in 23 states; five were tribal organizations. The purpose of the HPOG Program is to provide education and training to...

The first round of Health Profession Opportunity Grants (known as HPOG 1.0) funded education, training, support services, and employment assistance for recipients of Temporary Assistance for Needy Families (TANF) and other low-income individuals for jobs in the healthcare field. To assess its effectiveness, the first round of HPOG programs was evaluated using an experimental design...

The HPOG Impact Study will answer questions about the HPOG program’s overall effectiveness and explore how variations in program services affect program impacts. This analysis plan provides detailed information on...

Randomized experiments—in which study participants are randomly assigned to treatment and control groups within sites—give researchers a powerful method for understanding a program’s effectiveness. Once they know the direction (favorable or unfavorable) and magnitude (small or large) of a program’s impact, the next question is why the program produced its effect. Multi-site evaluations offer a chance to “get inside the black box” and explore that question.

This paper considers a new method, called Cross-Site Attributional Model Improved by Calibration to Within-Site Individual Randomization Findings (CAMIC), which seeks to reduce bias in analyses that researchers use to understand what about a program’s structure and implementation leads its impact to vary.

First, researchers estimate the overall impact of the program without selection bias or other sources of bias, and then use cross-site analyses to connect program structure (what is offered) and implementation (how it is offered) to the magnitude of the impacts. However, these estimates are non-experimental and may be biased.

The CAMIC method takes advantage of randomization of a program component in only some sites to improve estimating the effects of other program components and implementation features that are not or cannot be randomized. The paper describes the method for potential use in the Health Profession Opportunity Grants (HPOG) program evaluation.

A simulation analysis of CAMIC shows that the method does not consistently reduce bias and, in some cases, increases bias. Nevertheless, we argue that presenting details of the method is useful. We urge other researchers to consider other settings where the method might be successfully applied in order to help evaluators learn more about what works.

This report focuses on TANF recipients’ engagement and experiences in the HPOG Program, with the goal of helping understand how HPOG programs serve TANF recipients and developing hypotheses for further study...

This report describes the research design of the HPOG Impact Study. The study is designed to answer questions about overall HPOG program effectiveness and explore how variations in program services affect program impacts, including identifying which elements of career pathways programs contribute most to advancing the labor market success of participants...