Download ReportDownload Report PDF (1,456.17 KB)
- File Size: 1,456.17 KB
- Pages: N/A
- Published: 2020
- What is the impact of an offer of an HPOG 2.0 slot on participants’ receipt of training and support services, earnings, and broader measures of well-being?
- How does the impact of an offer of an HPOG 2.0 slot on participants vary with baseline characteristics or HPOG 2.0 local program characteristics?
The Health Profession Opportunity Grants (HPOG) Program is designed to deliver high-quality training in the health professions to eligible individuals. A National Evaluation of 27 grants awarded in 2015 as part of the second round of HPOG grants (HPOG 2.0) is currently underway. The National Evaluation will include an Impact Evaluation of the Program’s impacts on participants and its costs and benefits, as well as a Descriptive Evaluation of the implementation, outcomes, and local service delivery systems of the grants.
This report presents a research design plan for the Impact Evaluation. The key research questions for the Impact Evaluation concern the impact of HPOG 2.0 on participants relative to what their outcomes would have been without HPOG 2.0. The Impact Evaluation will also consider how those impacts vary with individual and local program characteristics. Together these results will be informative both about the impact of the HPOG 2.0 Program as implemented, and about broader issues in the design and operation of job training programs.
The HPOG 2.0 Program is a major effort to provide training in the health professions and to study the impact of HPOG’s approach to occupational training. This design report presents detailed plans for the Impact Evaluation of HPOG 2.0, to understand what difference the Program made. The purpose of the design document is twofold—to encourage careful thinking and broad agreement about how the research team will analyze the data to answer the evaluation’s research questions, and to clearly identify – in advance – the evaluation’s key design decisions and plans for analysis.
Key Findings and Highlights
The Impact Evaluation’s estimation of impact will build on random assignment conducted by all 27 non-tribal HPOG 2.0 grantees. This document:
- Provides an overview of the HPOG 2.0 local programs;
- Describes the theory of action underlying HPOG 2.0 and the research questions the Impact Evaluation will address to assess whether the Program is achieving its goals;
- Discusses the data sources that will be used in the analysis, including baseline and follow-up surveys and existing administrative data; and
- Describes the methods to be used to estimate the impact of HPOG 2.0 on participants in the short-, intermediate-, and longer-term follow-up points (approximately 15, 36, and 60 months after randomization, respectively).
The Impact Evaluation will use an experimental design. Eligible applicants will be assigned randomly either to a treatment group that is offered HPOG training and services or to a control group that does not have access to HPOG during the study period. Data sources will include baseline data collected about participants at study enrollment; short- and intermediate-term follow-up participant surveys initiated about 15 and 36 months after random assignment; and national administrative data on participants’ employment and earnings and their educational enrollment and degree receipt.
The Impact Evaluation will generate a Short-Term Impact Report (including the 15-month survey and administrative data results), an Intermediate-Term Impact Report (including the 36-month survey and administrative data results), and a Longer-Term Impact Report (administrative data only). All three documents will include program-specific impact estimates for local programs.
Judkins, David Ross, Jacob Alex Klerman, and Gretchen Locke. (2020). Analysis Plan for the HPOG 2.0 National Evaluation Short-Term Impact Report, OPRE Report # 2020-07. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
- Career pathways:
- A framework for occupational training that combines education, training, and support services that align with the skill demands of local economies and help individuals to enter or advance within a specific occupation or occupational cluster
- HPOG or HPOG Program:
- The national Health Profession Opportunity Grants initiative, including all grantees and their local programs
- HPOG grantee:
- The entity receiving an HPOG grant and responsible for funding and overseeing one or more local programs
- HPOG (local) program:
- A unique set of services, training courses, and personnel; a single grantee may fund one or more programs
- HPOG partners:
- Other organizations directly involved in the operations of an HPOG program
- HPOG program participants:
- Enrollees in education and training programs and related services supported by HPOG grants