Table of Contents | Previous | Next |
Chapter 4
Impacts on Employment, Earnings, Public Assistance, and Income
In its effort to promote job retention and advancement among working individuals who had recently left the Temporary Assistance for Needy Families (TANF) program, the Riverside County, California, Department of Public Social Services (DPSS) joined the Employment Retention and Advancement (ERA) project by creating and implementing the Post-Assistance Self-Sufficiency (PASS) program. Chapter 3 examines data from the program’s automated tracking system and from the ERA 12-Month Survey of clients to analyze sample members’ service receipt, participation in work-related activities, and supportive service payments.
This chapter analyzes administrative records — of quarterly earnings in jobs covered by unemployment insurance (UI), of TANF receipt, and of food stamp receipt — to examine whether the PASS program produced impacts on sample members’ employment, earnings, public assistance, and income. The sample includes all 2,770 adults who were in single-parent families and were randomly assigned from July 2002 through June 2003 as part of the Riverside PASS study. Two years of UI earnings data after random assignment and one year of public assistance data are available for all sample members, allowing for an assessment of the short-term impacts of the PASS program.
Study participants were employed at the time of random assignment and had left their welfare cases shortly before random assignment. The average outcomes for control group members represent the benchmarks against which the PASS program is measured. The differences between the averages for the PASS group and for the control group are known as the “impacts,” or “effects,” of the PASS program. Impacts that are statistically significant at the 10 percent level or less are unlikely to have arisen by chance. Because random assignment was used to place sample members in either the PASS group or the control group, statistically significant differences (impacts) are most likely caused by the PASS program.
Estimated Impacts of PASS
This section describes the impacts of Riverside PASS on employment, earnings, public assistance, and measured income. Any comparisons are relative to the control group average. Unless otherwise noted, all increases and decreases that are discussed in the text are statistically significant.
Impacts on Employment
-
Over the two-year follow-up period, PASS group members worked more consistently than control group members. The employment effects grew larger over time.
Table 4.1 summarizes the program’s impacts on UI-covered employment and earnings. As the table shows, 86 percent of PASS group members were employed in a UI-covered job at some point during Years 1 and 2, compared with 82 percent of the control group — an increase of 4 percentage points.1 Encouragingly, the impact on employment grew stronger over time; PASS increased the percentage ever employed by 6 percentage points above the control group level in Year 2, compared with 3 percentage points in Year 1.
Table 4.1 also includes several other measures of employment, such as average quarterly employment and the percentage employed four consecutive quarters (a key measure of employment retention). For both measures, the results indicate that PASS increased employment for the follow-up period as a whole and that the employment increase in Year 2 was larger than the employment increase in Year 1. Further analysis found that PASS increased the percentage of sample members who worked during all remaining quarters of follow-up once they had found a job — by 6.5 percentage points above the control group average of 34 percent.2
Figure 4.1 illustrates the impacts on UI-covered employment and earnings over time. The upper panel shows the employment rates of the PASS and control groups, and the lower panel shows their quarterly earnings. The percentage employed declined throughout the follow-up period for both research groups. Some decline in employment rates is inevitable: Because all the sample members were working when they entered the study, employment rates could only go down after random assignment. Further, some individuals moved out of state after random assignment, and MDRC collected UI records only from California. The difference between the PASS and control groups (that is, the impact on employment) grew larger over time. The impact was strongest in Quarter 9 (the last quarter of follow-up), when the employment rate for the PASS group was 6 percentage points higher than the rate for the control group (58 percent, compared with 52 percent).
| Outcome | PASS Group | Control Group | Difference (Impact) | P-Value | |
|---|---|---|---|---|---|
| Years 1-2 | Ever employed (%) | 86.0 | 82.1 | 3.9 *** | 0.00 |
| Average quarterly employment (%) | 62.1 | 58.1 | 4.0 *** | 0.00 | |
| Number of quarters employed | 5.0 | 4.6 | 0.3 *** | 0.00 | |
| Employed 4 consecutive quarters (%) | 59.6 | 56.9 | 2.7 | 0.13 | |
| Total earnings ($) | 18,368 | 16,578 | 1,791 *** | 0.00 | |
| Earned over $20,000 (%) | 39.9 | 35.1 | 4.8 *** | 0.01 | |
| Year 1 | Ever employed (%) | 80.1 | 77.1 | 3.0 ** | 0.04 |
| Average quarterly employment (%) | 64.6 | 61.6 | 3.0 ** | 0.03 | |
| Number of quarters employed | 2.6 | 2.5 | 0.1 ** | 0.03 | |
| Employed 4 consecutive quarters (%) | 47.9 | 44.8 | 3.2 * | 0.08 | |
| Total earnings ($) | 9,195 | 8,278 | 917 *** | 0.00 | |
| Earned over $10,000 (%) | 41.0 | 37.8 | 3.3 * | 0.06 | |
| Year 2 | Ever employed (%) | 73.6 | 68.0 | 5.6 *** | 0.00 |
| Average quarterly employment (%) | 59.6 | 54.7 | 4.9 *** | 0.00 | |
Number of quarters employed |
2.4 | 2.2 | 0.2 *** | 0.00 | |
| Employed 4 consecutive quarters (%) | 45.1 | 40.7 | 4.4 ** | 0.02 | |
| Total earnings ($) | 9,173 | 8,299 | 873 ** | 0.02 | |
| Earned over $10,000 (%) | 39.8 | 36.0 | 3.8 ** | 0.03 | |
| Sample size (total = 2,770) | 1,627 | 1,143 | |||
| SOURCE: MDRC calculations from California Employment Development Department unemployment insurance records. NOTES: See Appendix D. This table includes only employment and earnings in jobs covered by the California unemployment insurance (UI) program. It does not include employment outside California or in jobs not covered by UI (for example, "off-the-books" jobs, some agricultural jobs, and federal government jobs). |
 [D] |
Impacts on Earnings
- The PASS program produced substantial increases in total earnings.
Table 4.1 shows that, over the two-year follow-up period, PASS group members earned $18,368, compared with $16,578 for control group members. The program thus increased the earnings of the PASS group by an estimated $1,791 (an 11 percent increase). This is a surprisingly large impact for what is primarily a case management intervention. It is important to note that the total earnings figures are overall averages, including zero dollars for sample members who did not work.3 PASS group members earned about $900 more than the control group in both the first and the second year of follow-up. It is encouraging that the earnings gains were still statistically significant in Year 2.
Forty percent of PASS group members earned $20,000 or more during Years 1 and 2 — a nearly 5 percentage point increase over the control group average. This impact on relatively high earnings (for former TANF recipients) was consistent over time. PASS increased the percentage of sample members who earned $10,000 or more by between 3 and 4 percentage points during both Year 1 and Year 2.
The impact on earnings was driven by a combination of employment increases and increases in earnings among those who were employed. Programs like PASS may increase total UI-covered earnings for several reasons. Most commonly, programs increase earnings because a larger proportion of program group members work for pay at some point during the follow-up period or because program group members who work are more likely to remain employed. Less commonly, programs increase earnings because those who work tend to earn more (due to higher wages, more weeks worked, or longer hours). Because UI data in California are available only as total earnings in a quarter and because total hours or weeks worked are not provided, it is not possible to disentangle the relative contributions to earnings of increased wages, hours, or weeks worked. Further analysis, shown in Appendix Figure F.2, found that approximately two-thirds of the impact on earnings in Riverside PASS is attributable to employment increases and that the remaining one-third is attributable to earnings among those employed, which could be an indicator of advancement.4
- PASS generated increases in employment and earnings primarily by increasing the proportion of the sample who found a subsequent job.
Most PASS group and control group members left their initial job (the job held as of random assignment) within the first year. Approximately three-quarters of the sample worked in the quarter of random assignment (Quarter 1). By the last quarter of Year 1, however, less than 30 percent were still working with their initial employer.5 PASS did not increase retention or advancement in these initial jobs.
Table 4.2 shows the key employment outcomes separately for the job held at random assignment and for subsequent jobs. The table indicates that the employment increases are mostly driven by employment in post-random assignment jobs. PASS increased the percentage of sample members who found a subsequent UI-covered job, by 4 percentage points (the control group average was 62 percent). Table 4.2 also reveals an interesting trend: Both PASS and control group members spent more time employed at jobs that they found after random assignment than at the job they had held at the time they entered the study.
The PASS program also generated a large impact on earnings from post-random assignment jobs — an increase of $1,371 over the control group average of $7,712 (result not shown in table). PASS group members who found new jobs earned more, on average, than their counterparts in the control group — by a margin of more than $200 per quarter. (This is a nonexperimental comparison shown in Appendix Table F.1).6 One possible explanation for this is that the PASS program was successful in placing sample members in better jobs. However, this difference could be the result of many factors, including higher hourly wages, more hours of work per day, or more days of work per quarter for those members of the PASS group who found another job. The methodology and data used for this analysis do not allow a precise explanation.7
| Outcome | PASS Group | Control Group | Difference (Impact) | P-Value | |
|---|---|---|---|---|---|
| Working with random assignment employera | Ever employed (%) | 56.5 | 55.5 | 1.0 | 0.58 |
| Average quarterly employment (%) | 31.0 | 29.6 | 1.4 | 0.31 | |
| Number of quarters employed | 2.5 | 2.4 | 0.1 | 0.31 | |
| Working with post-random assignment employers | Ever employed (%) | 66.1 | 62.0 | 4.1 ** | 0.03 |
| Average quarterly employment (%) | 36.6 | 33.8 | 2.8 ** | 0.04 | |
| Number of quarters employed | 2.9 | 2.7 | 0.2 ** | 0.04 | |
| Sample size (total = 2,770) | 1,627 | 1,143 | |||
| SOURCE: MDRC calculations from California Employment Development Department unemployment insurance records. NOTES: See Appendix D. This table includes only employment and earnings in jobs covered by the California unemployment insurance (UI) program. It does not include employment outside California or in jobs not covered by UI (for example, "off-the-books" jobs, some agricultural jobs, and federal government jobs). aThe "random assignment employer" is defined as the employer during the quarter of random assignment from which the sample member received the most money. |
Impacts on Public Assistance and Income During Year 1
- PASS had no statistically significant effects on public assistance receipt. However, its substantial impacts on earnings translated into increases in total income.
Table 4.3 presents the impacts of the Riverside PASS program on public assistance and total measured income from earnings, TANF, and food stamps over the one-year follow-up period. The table shows that a typical sample member received about $1,600 in TANF grants (this finding includes zero dollars for nonrecipients) and that approximately 4 out of 10 sample members’ cases received TANF at some point during Year 1.8 Finally, more than 30 percent of sample members were still receiving welfare in the last quarter of Year 1 (not shown). PASS had no impact on these receipt rates. This is somewhat surprising, given the program’s large impacts on earnings. Unfortunately, public assistance data are not yet available for Year 2, when the impacts on employment and earnings were more consistent, so it is unclear whether earnings increases translated into welfare reductions in Year 2.9 There is evidence of welfare reductions in some subgroups and cohorts that experienced especially large increases in earnings.
Table 4.3 also shows that a slightly higher proportion of sample members — about 47 percent — received food stamps at some point during Year 1. Sample members received nearly $1,000 in food stamps (which again includes zero dollars for nonrecipients). Food stamp receipt rates were rather low in Riverside PASS during the study period, despite the fact that most sample members should still have been eligible.10 The impact of PASS on food stamp receipt is not statistically significant.
Finally, Table 4.3 shows that, during Year 1, Riverside PASS increased total measured income among PASS group members by $906 above the control group average of $10,823. The increase in total measured income reflects the increase in total earnings, without an offsetting decrease in public assistance payments.
| Outcome | PASS Group | Control Group | Difference (Impact) | P-Value |
|---|---|---|---|---|
| Ever received TANF (%) | 40.7 | 43.5 | -2.8 | 0.12 |
| Amount of TANF received ($) | 1,563 | 1,581 | -19 | 0.83 |
| Number of months receiving TANF | 3.3 | 3.4 | -0.1 | 0.58 |
| Ever received food stamps (%) | 46.2 | 47.5 | -1.3 | 0.48 |
| Amount of food stamps received ($) | 971 | 964 | 7 | 0.89 |
| Number of months receiving food stamps | 3.7 | 3.9 | -0.2 | 0.29 |
| Total measured income ($) a | 11,729 | 10,823 | 906 *** | 0.00 |
| Sample size (total = 2,770) | 1,627 | 1,143 | ||
| SOURCES: MDRC calculations from California Employment Development Department unemployment insurance records and TANF and food stamp administrative records from the State of California. NOTES: See Appendix D. This table includes only employment and earnings in jobs covered by the California unemployment insurance (UI) program. It does not include employment outside California or in jobs not covered by UI (for example, "off-the-books" jobs, some agricultural jobs, and federal government jobs). aThis measure represents the sum of UI earnings, TANF, and food stamps. |
Impacts on Key Subgroups
- The PASS program increased employment and earnings in the areas served by the Center for Employment Training (CET), the Volunteer Center, and Valley Restart. PASS had no statistically significant impacts in the areas served by Riverside Community College (RCC) and DPSS Rancho Mirage.
The bars in Figure 4.2 present the program’s impacts for the full sample and for sample members assigned to each of the five service providers that operated PASS.11 During the two-year follow-up period, the programs operated by the Volunteer Center and by CET increased the average quarterly employment among PASS group members by 5 and 8 percentage points, respectively, above the control group levels (which were about 56 percent in both sites). The differences are smaller and are not statistically significant for Valley Restart, though PASS did increase the percentage ever employed at some point during the follow-up period in that site (not shown). PASS generated a strong impact on total earnings (averaging $3,000 per sample member) in the CET and Volunteer Center service areas and produced a somewhat smaller gain for sample members who were assigned to Valley Restart.
 [D] |
The impacts of PASS in the RCC and DPSS Rancho Mirage service areas are not statistically significant. It should be noted that average employment rates and earnings levels were both higher for control group members who were assigned to RCC (compared with the services areas that had impacts).12 These higher control group levels may have made it more difficult for the program to produce impacts. A very small sample size in Rancho Mirage makes the impact analysis less reliable.
One interesting pattern shown in Figure 4.2 is that the impacts on earnings were all concentrated in service areas where the PASS providers were community-based organizations (CBOs). It may be that institutional arrangements can have an impact on the efficacy of PASS services. This pattern may be a coincidence, however, or it may simply reflect the demographic or labor market characteristics of the three CBO areas, rather than the efficacy of service delivery at the CBOs. Thus, further experimentation is warranted. The differences in earnings impacts across subgroups defined by service provider are statistically significant.13
The PASS program’s impacts on key employment and earnings outcomes varied by other subgroups as well. Among Hispanic sample members (who make up nearly half the sample), PASS increased earnings by more than $3,200 above the control group average; the impact among sample members in other racial/ethnic groups was only $558 and is not statistically significant. Increases were particularly large among Hispanic sample members living within the three CBO service areas. Thus, it is difficult to determine whether the PASS program works better for Hispanics or, alternatively, whether the program works better for Hispanics when it is delivered by CBO providers. Impacts were stronger among those who were recently employed in UI-covered jobs. However, impacts on employment and earnings did not differ for subgroups defined on the basis of educational attainment.
Discussion
The evidence in this chapter provides encouraging support for the approach taken by the PASS program to promote retention and advancement among working TANF leavers. While the implementation and participation results presented in Chapters 2 and 3 of this report certainly support the possibility of economic impacts, the size and consistency of the impacts are somewhat surprising. If the program’s goal was simply for sample members to retain the job held as of random assignment, PASS would be judged unsuccessful. However, the program appears to have done a good job of reemploying sample members who left their initial job. As discussed, most of the impacts resulted from PASS group members’ being more likely to find new jobs after they lost or moved on from the job that they had held at random assignment. It appears that PASS offered some combination of services, supports, and institutional arrangements that enabled more frequent reemployment than was observed for the control group.
It is also worth noting that Riverside PASS apparently worked best when delivered by CBOs. DPSS chose these agencies for the study because CBOs had more experience working with employed welfare leavers than DPSS staff did; CBOs were more familiar with jobs and services available in their neighborhoods; and DPSS thought that welfare leavers would be more likely to voluntarily receive services from CBOs than from the welfare department. It may be that institutional arrangements played a role in the efficacy of the program, but the research design for this study does not permit a reliable analysis of this factor.
1As discussed in Chapter 1, all the employment and earnings outcomes in this report are obtained from UI wage records. Research has found that UI records cover nearly 90 percent of all jobs (Kornfeld and Bloom, 1999). However, Hotz and Scholz (2002) note that this percentage is likely lower among low-income populations. Even though PASS was a postemployment program, only about 85 percent of sample members had employment recorded in UI records during the quarter before or the quarter of random assignment. (back)
2Appendix Table F.8 shows the impacts on this measure and various other measures of employment stability. (back)
3Quarterly earnings amounts were top-coded at $15,000. This means that earnings amounts above $15,000 were set equal to $15,000, which was done in order to protect against the possibility of high earnings values having undue influence. Thus, the impacts on earnings are unlikely to be influenced by statistical outliers. (back)
4This result can also be obtained by dividing the percentage impact on average quarterly employment by the percentage impact on earnings. In Table 4.1, ((4.0/58.1) / (1,791/16,578)) * 100 = 64 percent. (back)
5These results are shown in Appendix Table F.1. The “initial job” is defined as the job from which the participant received the highest UI-reported earnings during the quarter of random assignment. Of those working in Quarter 1, 80 percent worked for only one employer; the remaining 20 percent worked for two or more employers. In the latter case, a decision was made to consider whichever employer provided the highest earnings to be the employer at random assignment. None of the differences in employment or earnings that are attributable to post-random assignment employers occurred in Quarter 1. Thus, the differences in the proportions of PASS and control group members who were reemployed are unlikely to be affected by the decision to follow the employer that provided the highest earnings. (back)
6As explained in Chapter 3 (Box 3.1), nonexperimental comparisons include only a subset of the full report sample. Because participants in the PASS group may have different characteristics than participants in the control group, differences in these outcomes may not be attributable to the program. Statistical significance tests are not conducted for these measures. (back)
7It is possible that the PASS group members who were reemployed had different background characteristics than control group members who found new jobs. For example, PASS group members who were reemployed may have had higher educational attainment or other factors associated with labor market success. Reemployed PASS group members, however, look similar to reemployed control group members on several important measurable background characteristics. For example, reemployed PASS group members earned $1,935 in the quarter prior to random assignment, which was quite close to the $1,940 earned by reemployed control group members. It is possible, however, that the groups differ on unmeasurable characteristics, such as motivation. (back)
8Given that Riverside PASS served welfare leavers, these TANF receipt rates may seem high. However, for this study, a “welfare leaver” is defined as an individual who leaves a case. Welfare receipt rates are tracked based on cases, not individuals. Thus, if the case is still active (perhaps for dependents of a sample member who has been sanctioned), the TANF receipt rates reflect welfare received on behalf of the case. About 25 percent of the sample received a welfare grant during the month prior to random assignment. A quality check showed that most sample members who received welfare grants at the time of random assignment were themselves sanctioned off the welfare case, even though they received grants for their dependents. A sanction, as defined by the State of California, means that an adult (usually the case head) has his or her portion of the case’s monthly TANF grant subtracted from the grant amount for noncompliance with the GAIN program’s mandate. Additional analysis found that the impacts were the same for those whose welfare cases were completely closed as for those who had left cases that were still open. (back)
9Three years of public assistance data will be available for the final report on the PASS program. (back)
10Part of the reason for this is that sample members in Riverside PASS had relatively high earnings and that some therefore may qualify for grants that are too low to be worth navigating the application process. Sample members with above average earnings, however, qualify for less. (back)
11 Chapter 2 fully describes the five PASS service providers. (back)
12 These outcomes are shown in Appendix Table F.11. (back)
13This analysis compared the impacts of the three CBO providers with the impacts of the two non-CBO providers. It is important to note that impacts across the provider subgroups might differ due to variation in services, labor market conditions, or background characteristics of sample members in each provider’s service area. In order to examine such issues, a conditional subgroup analysis was conducted. This analysis found that the impacts were significantly larger in the CBO sites even after controls were added for demographic factors. While this suggests that PASS was more effective in CBO areas (net of demographic factors), labor market conditions cannot be ruled out as a factor. The conditional subgroup analysis also found that the impacts were stronger for Hispanic sample members, even after controlling for other factors, and that the impacts were notably strong for Hispanic individuals living in CBO areas. (back)
| Table of Contents | Previous | Next |