HPOG Logo

Health Profession Opportunity Grants (HPOG) Impact Study, 2011-2018

Project Overview

The Health Profession Opportunity Grants (HPOG) Impact Study is using an experimental design to examine the short-term impacts of HPOG 1.0 on participants’ educational and economic outcomes at 15 months after random assignment. The study includes 42 local HPOG programs implemented by 23 of the 32 HPOG 1.0 grantees (those awarded in 2010). In 19 of the 42 programs, the evaluation is also analyzing how certain program enhancements—emergency assistance, non-cash incentives, and facilitated peer support—might improve on impacts of the “standard” HPOG program. Additionally, the evaluation is using non-experimental methods to examine the relative contributions of various program characteristics.

The study’s key evaluation questions include:

  1. What impacts do HPOG programs as a group have on the outcomes of participants and their families?
  2. To what extent do these impacts vary across selected subpopulations?
  3. Which locally adopted program components influence average impacts?
  4. To what extent does participation in a particular HPOG component(s) change the impact?
  5. To what extent do specific program enhancements have impacts, relative to the “standard” HPOG program?

The Career Pathways Intermediate Outcomes Study and the Career Pathways Long-term Outcomes Study are continuing to track HPOG Impact Study participants, following up approximately three and six years after random assignment, respectively, to assess intermediate and longer term impacts.

The HPOG Impact Study is being conducted through a contract to Abt Associates and its subcontractor The Urban Institute.

The points of contact are Hilary Forster and Nicole Constance.

 

  • Health Profession Opportunity Grants (HPOG 1.0) Impact Study Interim Report: Program Implementation and Short-Term Impacts

    Published: June 18, 2018

    In 2010, the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services awarded the first round of five-year HPOG grants (HPOG 1.0) to 32 organizations in 23 states; five were tribal organizations. The purpose of the HPOG Program is to provide education and training to...

  • Improving Economic Opportunity through Healthcare Training: Short-term Impact Results from the First Round of the Health Profession Opportunity Grants (HPOG 1.0)

    Published: June 18, 2018

    The first round of Health Profession Opportunity Grants (known as HPOG 1.0) funded education, training, support services, and employment assistance for recipients of Temporary Assistance for Needy Families (TANF) and other low-income individuals for jobs in the healthcare field. To assess its effectiveness, the first round of HPOG programs was evaluated using an experimental design...

  • Health Profession Opportunity Grants (HPOG) Impact Study: Technical Supplement to the Evaluation Design Report: Impact Analysis Plan

    Published: March 3, 2017

    The HPOG Impact Study will answer questions about the HPOG program’s overall effectiveness and explore how variations in program services affect program impacts. This analysis plan provides detailed information on...

  • Using Within-Site Experimental Evidence to Reduce Cross-Site Attributional Bias in Connecting Program Components to Program Impacts

    Published: March 3, 2017

    Randomized experiments—in which study participants are randomly assigned to treatment and control groups within sites—give researchers a powerful method for understanding a program’s effectiveness. Once they know the direction (favorable or unfavorable) and magnitude (small or large) of a program’s impact, the next question is why the program produced its effect. Multi-site evaluations offer a chance to “get inside the black box” and explore that question.

    This paper considers a new method, called Cross-Site Attributional Model Improved by Calibration to Within-Site Individual Randomization Findings (CAMIC), which seeks to reduce bias in analyses that researchers use to understand what about a program’s structure and implementation leads its impact to vary.

    First, researchers estimate the overall impact of the program without selection bias or other sources of bias, and then use cross-site analyses to connect program structure (what is offered) and implementation (how it is offered) to the magnitude of the impacts. However, these estimates are non-experimental and may be biased.

    The CAMIC method takes advantage of randomization of a program component in only some sites to improve estimating the effects of other program components and implementation features that are not or cannot be randomized. The paper describes the method for potential use in the Health Profession Opportunity Grants (HPOG) program evaluation.

    A simulation analysis of CAMIC shows that the method does not consistently reduce bias and, in some cases, increases bias. Nevertheless, we argue that presenting details of the method is useful. We urge other researchers to consider other settings where the method might be successfully applied in order to help evaluators learn more about what works.

  • Training TANF Recipients for Careers in Healthcare: The Experience of the Health Profession Opportunity Grants (HPOG) Program

    Published: October 22, 2015

    This report focuses on TANF recipients’ engagement and experiences in the HPOG Program, with the goal of helping understand how HPOG programs serve TANF recipients and developing hypotheses for further study...

  • Health Profession Opportunity Grants (HPOG) Impact Study Design Report

    Published: December 4, 2014

    This report describes the research design of the HPOG Impact Study. The study is designed to answer questions about overall HPOG program effectiveness and explore how variations in program services affect program impacts, including identifying which elements of career pathways programs contribute most to advancing the labor market success of participants...

More Reports on this Project