Health Profession Opportunity Grants (HPOG 1.0) Impact Study: Six-Year Impacts Report

The HPOG 1.0 Impact Study is making an important contribution to the field’s collective knowledge about sector-based and career pathways programs. Most other evaluations focus on a single program usually selected for its promise, and the results of those evaluations are generalizable to programs that are similar to the one evaluated. In contrast, the HPOG 1.0 Impact Study considers a large collection of diverse, locally implemented programs, all operating in their own way under broad ACF guidelines. The benefit of this approach is that it helps to assess whether the general HPOG model—across many implementations of it—is effective in achieving its goals.

Six years after study members enrolled:

• HPOG did not detectably increase receipt of a postsecondary credential requiring a year or more of training among those who had access to HPOG (about one-third of the entire study sample had completed such a credential).
• HPOG increased employment in a healthcare occupation by 5 percentage points, from 32 percent in the control group to 37 percent in the group that could access the program.
• Despite the shift toward healthcare work, HPOG did not detectably increase quarterly earnings (the study sample had quarterly earnings of about $6,000 as of the six-year follow-up).

Beyond those priority impacts:

• HPOG increased receipt of a postsecondary credential of any length by 8 percentage points, from 70 percent in the control group to 78 percent in the group that could access the program.
• Duration of training averaged about 7½ months after six years, with little difference between the control and HPOG group.
• There is no evidence that HPOG’s impact differs—being more or less favorable—for any particular subgroups (such as those defined by race, parental status, baseline education, or prior labor market attachment).

COVID-19, which struck during this report’s follow-up window, affected the levels of labor-market outcomes, but it did not affect HPOG’s impacts, either favorably or not.

Evidence from this evaluation suggests that HPOG 1.0’s impacts are qualitatively similar to other career pathways programs, based on a meta-analysis of other impacts: that is, gains in educational progress and industry-specific employment have not led to earnings gains.

The impact evaluation of HPOG 1.0 uses an experimental evaluation design. By randomizing eligible applicants to treatment and control groups, the evaluation provides rigorous evidence to inform the adult training field about sector-based and career pathways programs. The impact analysis used administrative data from the National Directory of New Hires (NDNH) from the first 13 quarters after study participants were randomized and data from a follow-up survey initiated at about three years after they were randomized. As elaborated in prior analysis plans, the evaluation estimates the impact of HPOG 1.0 using a regression model that adjusts the difference between average outcomes for treatment versus control group members by controlling for characteristics measured at baseline.

Peck, Laura R., Daniel Litwok, and Douglas Walton. (2022). Health Profession Opportunity Grants (HPOG 1.0) Impact Study: Six-Year Impacts Report. OPRE Report 2022-45, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.