Health Profession Opportunity Grants (HPOG) Impact Study

2011-2018

The Health Profession Opportunity Grants (HPOG) Impact Study used an experimental design to examine the short-term impacts of HPOG 1.0 on participants’ educational and economic outcomes at 15 months after random assignment. The study includes 42 local HPOG programs implemented by 23 of the 32 HPOG 1.0 grantees (those awarded in 2010). In 19 of the 42 programs, the evaluation also analyzed how certain program enhancements—emergency assistance, non-cash incentives, and facilitated peer support—might improve on impacts of the “standard” HPOG program. Additionally, the evaluation used non-experimental methods to examine the relative contributions of various program characteristics.

The study’s key evaluation questions include:

  1. What impacts do HPOG programs as a group have on the outcomes of participants and their families?
  2. To what extent do these impacts vary across selected subpopulations?
  3. Which locally adopted program components influence average impacts?
  4. To what extent does participation in a particular HPOG component(s) change the impact?
  5. To what extent do specific program enhancements have impacts, relative to the “standard” HPOG program?

The Career Pathways Intermediate Outcomes Study and the Career Pathways Long-term Outcomes Study  are continuing to track HPOG Impact Study participants, following up approximately three and six years after random assignment, respectively, to assess intermediate and longer term impacts.

The HPOG Impact Study was led by Abt Associates in partnership with The Urban Institute.

Point(s) of contact: Nicole Constance and Hilary Bruck.

This study is registered on the Open Science Framework under the title Health Profession Opportunity Grants Impact Study Visit disclaimer page .

Information collections related to this project have been reviewed and approved by the Office of Management and Budget (OMB) Office of Information and Regulatory Affairs under OMB #0970-0394. Related materials are available at the HPOG 1.0 information collection page on RegInfo.gov Visit disclaimer page .

The most currently approved documents are accessible by clicking on the ICR Ref. No. with the most recent conclusion date. To access the information collections (E.g. interviews, surveys, protocols), click on View Information Collection (IC) List. Click on View Supporting Statement and Other Documents to access other supplementary documents.

Data from the HPOG Impact Study is archived through the Child and Family Data Archive under the title Health Profession Opportunity Grants Evaluation Visit disclaimer page .

 

Related Resources

This report presents the plan for evaluating the six-year impacts of the HPOG 1.0 grants on various outcomes. It describes the operationalization of outcome measures, including their source data.  

In 2010, the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services awarded the first round of five-year Health Profession Opportunity Grants (HPOG 1.0) to 32 organizations in 23 states; five were tribal organizations. The purpose of the HPOG Program is to provide education and training to Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals for occupations in the healthcare field...

In 2010, the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services awarded the first round of five-year HPOG grants (HPOG 1.0) to 32 organizations in 23 states; five were tribal organizations. The purpose of the HPOG Program is to provide education and training to...

The first round of Health Profession Opportunity Grants (known as HPOG 1.0) funded education, training, support services, and employment assistance for recipients of Temporary Assistance for Needy Families (TANF) and other low-income individuals for jobs in the healthcare field. To assess its effectiveness, the first round of HPOG programs was evaluated using an experimental design...

The HPOG Impact Study will answer questions about the HPOG program’s overall effectiveness and explore how variations in program services affect program impacts. This analysis plan provides detailed information on...

Randomized experiments—in which study participants are randomly assigned to treatment and control groups within sites—give researchers a powerful method for understanding a program’s effectiveness. Once they know the direction (favorable or unfavorable) and magnitude (small or large) of a program’s impact, the next question is why the program produced its effect. Multi-site evaluations offer a chance to “get inside the black box” and explore that question.

This paper considers a new method, called Cross-Site Attributional Model Improved by Calibration to Within-Site Individual Randomization Findings (CAMIC), which seeks to reduce bias in analyses that researchers use to understand what about a program’s structure and implementation leads its impact to vary.

First, researchers estimate the overall impact of the program without selection bias or other sources of bias, and then use cross-site analyses to connect program structure (what is offered) and implementation (how it is offered) to the magnitude of the impacts. However, these estimates are non-experimental and may be biased.

The CAMIC method takes advantage of randomization of a program component in only some sites to improve estimating the effects of other program components and implementation features that are not or cannot be randomized. The paper describes the method for potential use in the Health Profession Opportunity Grants (HPOG) program evaluation.

A simulation analysis of CAMIC shows that the method does not consistently reduce bias and, in some cases, increases bias. Nevertheless, we argue that presenting details of the method is useful. We urge other researchers to consider other settings where the method might be successfully applied in order to help evaluators learn more about what works.

This report focuses on TANF recipients’ engagement and experiences in the HPOG Program, with the goal of helping understand how HPOG programs serve TANF recipients and developing hypotheses for further study...

This report describes the research design of the HPOG Impact Study. The study is designed to answer questions about overall HPOG program effectiveness and explore how variations in program services affect program impacts, including identifying which elements of career pathways programs contribute most to advancing the labor market success of participants...