Introduction
Research Questions
- How were the outcome measures and subgroups defined?
- How did the analysis adjust for item and survey non-response?
- How sensitive are results to the analytic choices such as regression adjustment, weighting, nonresponse adjustment, and data source?
This technical appendix provides details on the common methodology used to report on the six-year impacts of nine programs being evaluated as part of the Pathways for Advancing Careers and Education (PACE) project. PACE evaluates the effectiveness of nine distinct programs, with study enrollment beginning in 2011. These programs were selected as examples of programs that include key features of a career pathways framework. Each of these programs provided education or training to low-income adults for occupations expected to pay well and be in high demand locally. The evaluation of these nine programs was funded by the Office of Planning, Research, and Evaluation (OPRE) within the U.S. Department of Health and Human Services, Administration for Children and Families (ACF).
The PACE project has previously produced individual reports on the impact of these nine programs at 18 months and three years after random assignment. At six years after random assignment, individual reports were produced for four of the programs. For all three follow-up periods, there is a cross-site report covering all nine programs (the three-year and six-year reports also include findings from the Health Profession Opportunity Grants 1.0 Impact Study for the same time period). All of the reports are published on the OPRE website (https://www.acf.hhs.gov/opre/project/career-pathways). This appendix supports the four impact reports and the cross-site report for all nine PACE programs covering six years after random assignment.
Purpose
This appendix volume reports the technical details for the four individual PACE six-year impact reports and the cross-site report. It provides details on the data sources used, construction of outcome measures and subgroups, the selection of covariates for the impact regressions, the procedures for handling item non-response, survey non-response and weighting adjustment, the treatment of outliers, and sensitivity tests for analytic decision and alternative data sources for outcome measures.
Methods
This volume provides detailed descriptions of the outcome measures used in the impact analysis and provides sensitivity tests for the analytic choices made in the impact reports. For example, it presents with and without weighting for survey nonresponse and earnings impact using the National Directory of New Hires compared to survey data.
Citation
Judkins, David, Emily Roessel, and Gabriel Durham. 2022. Career Pathways Long-Term Outcomes Study: Appendices for PACE Six-Year Impact Reports, OPRE Report 2022-69. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- ACF:
- Administration for Children and Families
- HPOG:
- Health Profession Opportunity Grants
- PACE:
- Pathways for Advancing Careers and Education
- TANF:
- Temporary Assistance for Needy Families