Washington State's Integrated Basic Education and Skills Training (I-BEST) Program: Three-Year Impact Report

Publication Date: July 13, 2021
I-BEST Three Year Impact Report cover image

Download Report

Download Report PDF (2,834.84 KB)
  • File Size: 2,834.84 KB
  • Pages: 83
  • Published: 2021

Introduction

Research Questions

  1. Three years after random assignment, what were the effects of I-BEST on: educational outcomes; entry into career-track employment and higher earnings; and individual and family well-being, including income and other life outcomes?
  2. Did the benefits of the I-BEST program outweigh the costs from the perspectives of government, treatment group members, and society as a whole?

This report documents the impacts three years after random assignment for Washington State’s Integrated Basic Education and Skills Training (I-BEST) program. I-BEST was designed to increase access to and completion of college-level occupational training in a range of in-demand occupational areas for adults with low incomes. I-BEST was developed by Washington’s State Board for Community and Technical Colleges (SBCTC) and has operated statewide since the 2006-07 academic year. The program’s signature feature is a team-teaching approach where students receive instruction from two instructors in the same course: one provides job training and the other teaches basic skills in reading, math, or English. To further support students, the I-BEST programs in this evaluation included dedicated advisors to provide students with guidance on academic issues, navigating the college’s procedures, and career planning. It also provided “fill-the-gap” financial support beyond typical sources, for training and associated materials.

I-BEST is part of the Pathways for Advancing Careers and Education (PACE) project. Funded by the Administration for Children and Families within the U.S. Department of Health and Human Services, PACE is a multi-site experimental evaluation of nine programs for adults with low incomes that incorporate some features of a career pathways framework. The I-BEST evaluation was conducted at three of Washington’s 34 community and technical colleges: Bellingham Technical College, Everett Community College, and Whatcom Community College.

This evaluation is part of the Career Pathways Intermediate Outcomes Study, which extends the follow-up period to three years for programs in the PACE project. It extends analyses conducted for an initial I-BEST report (Glosser et al. 2018) that covered implementation and short-term (18 to 24 months) impacts on education and early career progress. Future reports produced by the Career Pathways Long-term Outcomes Study will extend the follow-up period further.

Purpose

I‑BEST grew out of a concern that adult students who do not have the skills to directly enter college programs were not advancing beyond basic skills courses to college-level occupational programs, and therefore were not earning credentials. I-BEST aims to teach students basic and occupational skills concurrently so they can move more quickly into higher-paying jobs or college-level courses. Colleges in the evaluation operated I-BEST in one or more occupational areas including Automotive, Electrical, Office Skills, Nursing, Precision Machining, and Welding. This research was undertaken to evaluate whether I-BEST was successful in increasing access to and completion of college-level occupational training for adults with low skills and whether the program’s efforts led to impacts on credentials, earnings, and other life outcomes.

Key Findings and Highlights

The three-year findings summarized below focus on the pre-specified confirmatory and secondary outcomes. There is one confirmatory outcome in both the education and the employment domains, which are the outcomes that we determined to be the most critical to judging the program’s success. Secondary outcomes are other important outcomes related to expected changes if the program is successful.

  • I-BEST had no impact on receipt of credentials requiring a year or more of college study—the confirmatory outcome in the education domain.

This outcome was selected as the confirmatory outcome because it reflects the I-BEST program’s goal of promoting additional progress in career pathways beyond the initial I-BEST programs. This result indicates that after completing their one- to three-quarter I‑BEST program, treatment group members were not more likely than the control group members to complete additional courses that lead to higher-level credentials. The short-term impact report found that two thirds of those who attended I‑BEST took courses after the I-BEST program; however, this additional coursework did not result in higher-level credentials during the three-year follow-up period.

  • I-BEST had a large impact on receipt of any college credential.

Almost half (48 percent) of the treatment group received some type of credential from a college, compared to 17 percent of the control group, a 31 percentage point impact. This impact on any college credential was primarily driven by an impact on shorter-term workforce awards earned through completing occupational courses. Credits earned from workforce awards are not transferable to four-year colleges.

  • I-BEST had no detectable impact on average quarterly earnings in follow-up quarters 12-13, using administrative data—the confirmatory outcome in the employment  domain.

Though no earnings impacts were detected in quarters 12-13 using administrative data, imprecision due to small sample sizes does not allow us to rule out that I-BEST might have had substantively important effects on earnings (the uncertainty surrounding the estimate of a +$404 impact indicates a range from as large as +$973 to as small as −$165). However, the study detects statistically significant positive effects of more than $600 per quarter in immediately prior quarters (quarters 10 and 11) and in quarters 10-13 in total. Furthermore, in the survey data, the study detects sustained earnings impacts starting at +$1,029 and increasing to +$1,206 in quarters 10-12. We also detected impacts on working in a job that paid at least $14 per hour. These results lead us to conclude that it is likely that I-BEST produced a positive impact on earnings three years after random assignment.

  • Because of imprecision in the earnings and educational costs estimates, it is not possible to definitively assess whether the benefits of the I-BEST program are outweighed by the costs.

Due to uncertainty in underlying estimates, cost-benefit analysis findings indicate that both positive and negative net benefits are plausible.

Methods

The I­‑BEST evaluation used an experimental design in which program applicants were assigned at random to either a treatment group that could access the program or a control group that could not, then compared their average outcomes. Between November 2011 and September 2014, program staff randomly assigned 315 program applicants to the treatment group and 317 to the control group. The impact study used data from a three-year follow-up survey, administrative records from SBCTC, and earnings records from the National Directory of New Hires. The study measured impacts on education and training, employment and earnings, and other life outcomes approximately three years after random assignment for all measures and up to four years for select earnings measures with available administrative data. The study included a cost-benefit analysis.  

Citation

Martinson, Karin, Sung-Woo Cho, Asaph Glosser, Karen Loya, and Samuel Dastrup. 2021. Washington State’s Integrated Basic Education and Skills Training (I‑BEST) Program: Three-Year Impact Report. OPRE Report 2021-102. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Glossary

ACF:
Administration for Children and Families
PACE:
Pathways for Advancing Careers and Education
TANF:
Temporary Assistance for Needy Families