National and Tribal Evaluation of the 2nd Generation of Health Profession Opportunity Grants (HPOG 2.0): Impact Evaluation Design Plan

September 27, 2019
Topics:
Self-Sufficiency, Welfare & Employment
Projects:
National Evaluation of the 2nd Generation of Health Profession Opportunity Grants (HPOG 2.0 National Evaluation), 2015-2025 | Learn more about this project
Types:
Reports
HPOG 2 IEDP Cover
Download report (pdf)
  • File Size 2mb
  • Pages 84
  • Published 2019

Introduction

The Health Profession Opportunity Grants (HPOG) Program is designed to deliver high-quality training in the health professions to eligible individuals. A National Evaluation of 27 grants awarded in 2015 as part of the second round of HPOG grants (HPOG 2.0) is currently underway. The National Evaluation will include an Impact Evaluation of the Program’s impacts on participants and its costs and benefits, as well as a Descriptive Evaluation of the implementation, outcomes, and local service delivery systems of the grants.

This report presents a research design plan for the Impact Evaluation. The key research questions for the Impact Evaluation concern the impact of HPOG 2.0 on participants relative to what their outcomes would have been without HPOG 2.0. The Impact Evaluation will also consider how those impacts vary with individual and local program characteristics. Together these results will be informative both about the impact of the HPOG 2.0 Program as implemented, and about broader issues in the design and operation of job training programs.

Primary Research Questions

  1. 1 What is the impact of an offer of an HPOG 2.0 slot on participants’ receipt of training and support services, earnings, and broader measures of well-being?
  2. 2 How does the impact of an offer of an HPOG 2.0 slot on participants vary with baseline characteristics or HPOG 2.0 local program characteristics?

Purpose

The HPOG 2.0 Program is a major effort to provide training in the health professions and to study the impact of HPOG’s approach to occupational training. This design report presents detailed plans for the Impact Evaluation of HPOG 2.0, to understand what difference the Program made. The purpose of the design document is twofold—to encourage careful thinking and broad agreement about how the research team will analyze the data to answer the evaluation’s research questions, and to clearly identify – in advance – the evaluation’s key design decisions and plans for analysis.

Methods

The Impact Evaluation will use an experimental design. Eligible applicants will be assigned randomly either to a treatment group that is offered HPOG training and services or to a control group that does not have access to HPOG during the study period. Data sources will include baseline data collected about participants at study enrollment; short- and intermediate-term follow-up participant surveys initiated about 15 and 36 months after random assignment; and national administrative data on participants’ employment and earnings and their educational enrollment and degree receipt.

The Impact Evaluation will generate a Short-Term Impact Report (including the 15-month survey and administrative data results), an Intermediate-Term Impact Report (including the 36-month survey and administrative data results), and a Longer-Term Impact Report (administrative data only). All three documents will include program-specific impact estimates for local programs.

Citation

Klerman, Jacob Alex, David Judkins, and Gretchen Locke. (2019). Impact Evaluation Design Plan for the HPOG 2.0 National Evaluation, OPRE Report # 2019-82. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Glossary

Career Pathways
a framework for occupational training that combines education, training, and support services that align with the skill demands of local economies and help individuals to enter or advance within a specific occupation or occupational cluster
HPOG or HPOG Program
the national Health Profession Opportunity Grants initiative, including all grantees and their local programs
HPOG grantee
the entity receiving an HPOG grant and responsible for funding and overseeing one or more local programs
HPOG (local) program
a unique set of services, training courses, and personnel; a single grantee may fund one or more programs
HPOG partners
other organizations directly involved in the operations of an HPOG program
HPOG program participants
enrollees in education and training programs and related services supported by HPOG grants
Last Reviewed: October 11, 2019