Health Profession Opportunity Grants (HPOG 1.0) Impact Study: Three-Year Impacts Report

December 9, 2019
Topics:
Self-Sufficiency, Welfare & Employment
Projects:
Career Pathways Intermediate Outcomes (CPIO) Study, 2014-2019 | Learn more about this project, Health Profession Opportunity Grants (HPOG) Evaluation Portfolio | Learn more about this project, Health Profession Opportunity Grants (HPOG) Impact Study, 2011-2018 | Learn more about this project
Types:
Reports
Cover of Health Profession Opportunity Grants (HPOG 1.0) Impact Study: Three-Year Impacts Report
Download report (pdf)
  • File Size 5mb
  • Pages 94
  • Published 2019

Introduction

In 2010, the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services awarded the first round of five-year Health Profession Opportunity Grants (HPOG 1.0) to 32 organizations in 23 states; five were tribal organizations. The purpose of the HPOG Program is to provide education and training to Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals for occupations in the healthcare field that pay well and are expected to either experience labor shortages or be in high demand. HPOG 1.0 grantees designed and implemented programs to provide eligible participants with education, occupational training, and support and employment services to help them train for and find jobs in a variety of healthcare professions.

The ACF Office of Planning, Research, and Evaluation supports a multipronged research and evaluation strategy to assess the success of the HPOG Program. To assess its effectiveness, the first round of HPOG programs was evaluated using an experimental design in which eligible program applicants were assigned at random to a “treatment” group that could access the program or a “control” group that could not, and then compared their outcomes. This document reports on the impacts that arose about three years after random assignment. It reports an overall average impact across the diverse HPOG 1.0 programs, as well as impacts for selected subgroups of study participants.

Research Questions

  1. 1 What impacts do the locally implemented HPOG programs as a group have on the outcomes of participants and their families?
  2. 2 To what extent do those impacts vary across selected subpopulations?
  3. 3 To what extent do the education and employment experiences of HPOG participants over time suggest that they are following a career pathway?

Purpose

The HPOG 1.0 Impact Study is making an important contribution to the field’s collective knowledge about sector-based and career pathways programs. Most other evaluations focus on a single program usually selected for its promise, and the results of those evaluations are generalizable to programs that are similar to the one evaluated. In contrast, the HPOG 1.0 Impact Study considers a large collection of diverse, locally implemented programs, all operating in their own way under broad ACF guidelines. The benefit of this approach is that it helps to assess whether the general HPOG model—across many implementations of it—is effective in achieving its goals.

Key Findings and Highlights

According to a follow-up survey (initiated about three years after randomization) and national administrative data (available through the 13th follow-up quarter), relative to the control group:

  • 13 percentage points more of the HPOG treatment group had completed training (one of the study’s two confirmatory outcomes); and
  • quarterly earnings (the other confirmatory outcome) among the HPOG treatment group were no different in the 12th-13th quarters after random assignment (both groups earned approximately $5,000 on average per quarter).

In addition to these two prioritized outcomes, the three-year analysis also found that, relative to the control group:

  • 1 percent more of the HPOG treatment group self-assessed as “confident” in their career knowledge;
  • 1 percentage point more of the HPOG treatment group was employed;
  • 12 percentage points more of the HPOG treatment group was employed in healthcare;
  • 2 percentage points more of the HPOG treatment group was employed at a job offering health insurance;
  • 7 percentage points more of the HPOG treatment group experienced career progress, a measure that combines educational progress with earnings growth; and
  • 3 percentage points fewer of the HPOG treatment group reported financial hardship, as indicated by generally not having enough money to make ends meet at the end of the month over the prior year.

These findings are all statistically significantly different from zero.

Methods

The HPOG 1.0 Impact Study used an experimental evaluation design to assess the impacts of 42 local HPOG programs in 23 of 32 first-round grantees nationwide. By randomizing eligible applicants to treatment and control groups, the evaluation provides rigorous evidence to inform the adult training field about sector-based and career pathways programs. The impact analysis used administrative data from the National Directory of New Hires (NDNH) from the first 13 quarters after study participants were randomized and data from a follow-up survey initiated at about three years after they were randomized.

Citation

Peck, Laura R., Daniel Litwok, Douglas Walton, Eleanor Harvill, and Alan Werner. (2019). Health Profession Opportunity Grants (HPOG 1.0) Impact Study: Three-Year Impacts Report. OPRE Report 2019-114. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Last Reviewed: January 8, 2020