Table of Contents | Previous | Next |
Appendix A
EFFECTS OF WELFARE REFORM FOR SUBGROUPS
A key question for many policymakers is whether different subgroups respond to welfare reform differently. Since groups facing different barriers or advantages may respond differently to different reforms, it is possible that some groups could be left behind, even as others fare well under the new regime. Understanding such responses would enable legislators and administrators to refine their programs, targeting groups most in need of assistance and improving their programs’ overall performance. In this appendix, we discuss what is known about the effects of the various welfare reforms and outcomes considered in Chapters 4 to 10 on different segments of the welfare population.
In the case of the random assignment studies, it is possible to consider impacts for subgroups of the populations served by a particular demonstration.108 Table A.1 shows the subgroups analyzed across the random assignment studies, with codes to indicate which outcomes are analyzed for each subgroup. For example, Vermont’s WRP examines impacts on welfare use (W), employment and earnings (E), use of other government programs (G), and income and poverty (I) for subgroups defined by a composite measure of disadvantage and a measure of prior welfare receipt. Depending on the study, the composite measure of disadvantage may be designed to reflect risk of long-term dependency, barriers to employment, or other combined measures of disadvantage. Other subgroups are defined by single dimensions that capture prior welfare use or labor market history, human capital accumulation, demographic characteristics, health and family structure, or child characteristics. All characteristics are measured as of the time of randomization.
Table A.1–Subgroups Analyzed by Random Assignment Studies Included in Synthesis (Click on this link to view table)
It is evident from Table A.1 that some subgroups receive more analysis than others, and that most subgroup analyses are confined to the basic outcomes of welfare use, employment and earnings, and income and poverty. Subgroup differences for family structure and other measures of well-being are examined by only one study each. A number of studies do not analyze subgroups for any of the outcomes shown in Table A.1.
Given the differences in which subgroups are examined for a given outcome, it will often not be possible to cross-validate findings across multiple studies as to whether the impacts of a given policy or bundle of policies vary by subgroup. Even when two studies within our classification scheme consider differences for the same outcome and subgroup category, differences across studies in the way the subgroups are defined often make results less comparable. Some subgroups that might be of interest, for example immigrants are not covered at all by the studies in Table A.1.
Two other issues complicate our ability to examine differences by subgroups. The first involves sample sizes. In many random assignment studies, the overall sample size is chosen to ensure that the overall program impacts will be statistically significant if their magnitudes are economically meaningful. However, detecting differences in effects between subgroups requires samples sizes several times larger than those required to detect an overall effect. Thus, there may be important variation in effects across subgroups, but they will not be detected unless the sample is much larger than that needed to generate significant results in the full sample. Many studies are based on samples that are too small to reliably detect even important differences between groups.
The second issue involves hypothesis testing. All the studies report whether the subgroup-specific estimates are significantly different from zero. Of equal importance is whether they are significantly different from each other. However, tests for the heterogeneity of impacts across groups are reported in only a few studies, leaving the reader to draw conclusions about group-specific differences from less objective criteria.
The remainder of the appendix discusses the results for program impacts by subgroups for the outcomes examined in Chapters 4 to 10, following the same order as the chapters. In the case of the Chapter 7 analysis of fertility and marriage, only one study includes any subgroup analyses, so those results are reported as part of the main discussion in Chapter 7. For the outcomes covered in this appendix, we note that the subgroup analyses may be based on a shorter follow-up period than the main results presented in the body of the report. For example, throughout the appendix, our analysis of subgroup impacts for Jobs First is based on outcomes measured in the 18-month follow-up survey (Bloom et al., 2000) or administrative data through years two or three (Hendra, Michalopolous, and Bloom, 2001). Subgroup results based on the three-year follow-up survey and administrative data through year four provide a more limited set of analyses (specifically by level of disadvantage, welfare recipiency status, and race/ethnicity) than those reported here (Bloom et al., 2002). Likewise, the subgroup analyses for the NEWWS programs are based on a pooled analysis using data through the third follow-up year (see the discussion of Michalopoulos and Schwartz, 2000, below). Subgroup results are not available in the five-year follow-up study of the NEWWS programs (Hamilton et al., 2001).
A.1. WELFARE USE
Subgroup differences for welfare use are reported for a subset of the random assignment studies reviewed in Chapter 4. The available results for ten of the programs are reported in Table A.2. The table records impacts for up to four different subgroups, with subgroups arrayed, to the extent possible, from most to least disadvantaged. Some studies report only whether subgroup impacts are statistically significant (denoted using asterisks next to the impact estimate); others also report whether differences in impacts across subgroups are statistically significant (denoted in the first column by x’s). We indicate when the statistical significance of subgroup differences is not available. Thus, when that cell in the table is empty, it means that the differences by groups are not statistically significant.
A.1.1. Programs That Focus on Financial Work Incentives
As seen in Panel A of Table A.2, there are two programs in this category, WRP-IO and MFIP-IO, that provide subgroup results for welfare use. Both provide subgroup-specific impact estimates, but the MFIP impacts for subgroups are available only for the recent applicants.109 None of the subgroup-specific impacts from WRP-IO are significant. In MFIP-IO, program impacts are significantly different from zero for both short-term recipients and new applicants. There is no way to determine whether the subgroup-specific estimates are significantly different from each other, however.
A.1.2 Programs That Focus on Financial Work Incentives Tied to Hours of Work
New Hope provides tests for subgroup differences; none of these tests rejects homogeneity across the three groups shown (Panel B). The evaluations of SSP Applicants and SSP Plus provide no subgroup analyses. The main SSP evaluation estimates subgroup impacts on employment (see below), but not on welfare use.
A.1.3. Programs That Focus on Mandatory Work-Related Activities
A different type of evidence on subgroup effects is available for policy reforms involving mandatory work-related activities (Panel C). For this type of reform policy, there are estimates from a large number of studies, all of which define subgroups in a consistent manner. Michalopoulos and Schwartz (2000) reanalyze the microlevel data from 20 studies that involve mandated work-related activities, including the eleven NEWWS programs discussed above as well as MFIP and FTP.110 By systematically reanalyzing the microdata from these studies, they are able to construct more rigorous and powerful tests for subgroup differences than can be obtained by simply comparing group-specific program impacts from separate studies. Unfortunately, Michalopoulos and Schwartz (2000) do not analyze welfare use, but rather focus on welfare payments. As a result, we defer our discussion of their results to Section A.4 below, where we discuss components of income.
Subgroup results for L.A. Jobs-First GAIN, which was not included in the Michalopoulos and Schwartz analysis, are presented in Panel B of Table A.2. Few substantial between-group differences are readily apparent, with the possible exception of the results for Asians in the race-ethnicity breakdown. None of the differences are statistically significant.
A.1.4. Programs That Combine Financial Work Incentives and Mandatory Work-Related Activities
Although a substantial number of studies focus on this particular combination of reforms, as a whole the studies provide little useful evidence about the combination’s effects on different groups (Panel D). The limitations pertaining to WRP-IO and MFIP-IO discussed above pertain to the full WRP and MFIP programs as well. None of the subgroup estimates from WRP are significant. Both subgroup estimates from MFIP are significantly different from zero, but they are similar, and no test for their difference is available.
In FIP, none of the estimates for families without prior-year earnings or with children under three are significant, compared to three of the four estimates for families with prior-year earnings or with no children under three. There are no tests to determine whether the subgroup differences are statistically significant. The evaluation of TSMF provides no subgroup analyses for welfare use.
A.1.5. Programs That Focus on TANF-Like Bundles of Reforms
Among studies that combine time limits with other policy reforms, ABC, FTP, and Jobs First provide subgroup impacts on welfare use (Panel F). However, the subgroups are defined differently across the studies, as seen in Table A.2, although all the subgroup classifications are intended to reflect measures of disadvantage. As noted above, all are arrayed from most to least disadvantaged across the table. In the case of FTP, the length of the time limit provides a measure of disadvantage because participants with low levels of education, short employment histories, and lengthy welfare histories were given a 36-month time limit, whereas other participants were given a 24-month time limit. Using participants’ applicant/recipient status at random assignment as a measure of disadvantage, as with Jobs First, is based on the observation that most new entrants will stay on welfare for a relatively short period, whereas most ongoing recipients are in the midst of what will become a lengthy spell on aid (Bane and Ellwood, 1994).
Across these groups, there is no clear tendency for program impacts to vary by the level of disadvantage. The effects of ABC were insignificant at both levels of schooling. They were largest for those with intermediate levels of prior aid use and were smallest for those with the most recent employment history. FTP’s effects were similar for all groups. Jobs First increased aid use by a greater amount among disadvantaged groups during the pre—time limit period. By some measures, the program decreased aid use among more disadvantaged groups during the post—time limit period; by other measures, the pattern is less clear. The change in impacts between the pre— and post—time limit period is fairly similar across the groups. Based on this relatively small number of studies and differing definitions of disadvantage, we cannot conclude whether the effects of policy reforms involving time limits have greater or lesser effects on relatively disadvantaged groups.
A.2. EMPLOYMENT AND EARNINGS
Although only a subset of the random assignment studies provides subgroup analyses for employment and earnings, the subset is larger in this case than it was in the case of welfare use. Thus, we can consider subgroup differences for a broader range of policy reforms than we could above. Table A.3 presents subgroup-specific program impacts for employment and Table A.4 presents corresponding estimates for earnings.111 As a whole, the available evidence on subgroups does not suggest that any of the reforms consistently work to the greater employment or earnings detriment–or benefit–of relatively disadvantaged groups.
| Group 1 | Group 2 | Group 3 | Group 4 | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Name | Cases served | Signif. of group diff. | Measure | Control mean | Impact | % | Control mean | Impact | % | Control mean | Impact | % | Control mean | Impact | % |
| A. Programs that focus on financial work incentives | |||||||||||||||
| WRP-IO | Single-parent recipients and applicants | n.a. | By status at RA: | Recipient | Applicant | ||||||||||
| Employed in last 3 mos. of FU | 46.0 | 2.5 | 5.4% | 51.4 | 3.3 | 6.4% | |||||||||
| n.a. | By level of disadvantage: | Most disadvantaged (1) | Moderately disadvantaged (1) | Least disadvantaged (1) | |||||||||||
| Employed in last 3 mos. of FU | 33.3 | 9.9 | 29.7% | 44.9 | 0.3 | 0.7% | 59.5 | 5.5 | 9.2% | ||||||
| MFIP-IO | Urban single parents recipients | By years of schooling: | Less than 12 | 12 or more | |||||||||||
| n.a. | Quarterly employment rate, year 3 | 35.7 | -0.9 | -2.5% | 49.6 | 4.8* | 9.7% | ||||||||
| By employment in year before RA: | No work in year before RA | Worked in year before RA | |||||||||||||
| n.a. | Quarterly employment rate, year 3 | 35.4 | 5.1* | 14.4% | 57.8 | 1.0 | 1.7% | ||||||||
| By years on aid prior to RA: | 5 or more years | Less that 5 years | |||||||||||||
| n.a. | Quarterly employment rate, year 3 | 42.9 | 5.1* | 11.9% | 47.7 | 1.6 | 3.4% | ||||||||
| B. Programs that focus on financial work incentives tied to hours of work | |||||||||||||||
| New Hope | Poor families employed FT at RA | By barriers to employment: | Two (3) | One (3) | None (3) | ||||||||||
| Quarters employed, year 1 | 2.1 | 0.6*** | 28.6% | 2.3 | 0.5*** | 21.7% | 2.5 | 0.4** | 16.0% | ||||||
| xx | Quarters employed, year 2 | 2.4 | 0.3 | 12.5% | 2.3 | 0.5*** | 21.7% | 2.8 | -0.2 | -7.1% | |||||
| All poor families | By race/ethnicity: | African-American | Hispanic | White | |||||||||||
| Quarters employed, year 1 | 2.6 | 0.5*** | 19.2% | 2.6 | 0.4*** | 15.4% | 2.6 | 0.3 | 11.5% | ||||||
| Quarters employed, year 2 | 2.6 | 0.3*** | 11.5% | 2.7 | 0.2 | 7.4% | 2.8 | 0.0 | 0.0% | ||||||
| SSP | Single-parent recipients | By age of youngest child: | 0 to 2 | 3 to 11 | 12 to 15 | 16 and over | |||||||||
| Months of FT employment | 4.4 | 3.5*** | 79.5% | 5.4 | 3.2*** | 59.3% | 5.3 | 3.4*** | 64.2% | 6.1 | 2.0* | 32.8% | |||
| By education: | No high school credential | High school credential | |||||||||||||
| Months of FT employment | 3.3 | 3.3*** | 100.0% | 7.2 | 3.1*** | 43.1% | |||||||||
| By employment status at RA: | Out of labor force | Unemployed | Employed PT | Employed FT | |||||||||||
| Months of FT employment | 2.6 | 3.0*** | 115.4% | 5.1 | 4.1*** | 80.4% | 7.7 | 4.6*** | 59.7% | 20.0 | 2.7** | 13.5% | |||
| By months on aid in prior 3 years: | 36 months | 24 to 35 | 10 to 23 | ||||||||||||
| Months of FT employment | 3.9 | 3.3*** | 84.6% | 5.5 | 3.6*** | 65.5% | 6.4 | 3.0*** | 46.9% | ||||||
| C. Programs that focus on mandatory work-related activities | |||||||||||||||
| LA Jobs-First GAIN | Single-parent recipients and applicants | By race/ethnicity: | African-American | Hispanic | Asian | White | |||||||||
| Ever employed in years 1-2 | 62.0 | 6.6*** | 10.6% | 57.5 | 12.0*** | 20.9% | 41.9 | 13.0*** | 31.0% | 54.8 | 8.3*** | 15.1% | |||
| By English proficiency: | Not proficient | Proficient | |||||||||||||
| xx | Ever employed in years 1-2 | 46.7 | 12.4*** | 26.6% | 60.3 | 9.0*** | 14.9% | ||||||||
| By education: | No diploma or GED | Diploma or GED | |||||||||||||
| Ever employed in years 1-2 | 52.3 | 10.1*** | 19.3% | 64.0 | 9.0*** | 14.1% | |||||||||
| By status at RA: | Long-term recipient | Short-term recipient | New applicant | ||||||||||||
| Ever employed in years 1-2 | 55.5 | 10.2*** | 18.4% | 62.5 | 8.5*** | 13.6% | 67.5 | 4.7 | 7.0% | ||||||
| By employment in year prior to RA | Not employed | Employed | |||||||||||||
| Ever employed in years 1-2 | 43.4 | 12.8*** | 29.5% | 82.4 | 4.1*** | 5.0% | |||||||||
| By disadvantage: | Most disadvantaged (2) | ||||||||||||||
| xxx | Ever employed in years 1-2 | 39.3 | 13.2*** | 33.6% | |||||||||||
| D. Programs that focus on financial work incentives and mandatory work-related activities | |||||||||||||||
| WRP | Single-parent recipients and applicants | By status at RA: | Recipient | Applicant | |||||||||||
| n.a. | Employed in last 3 mos. of FU | 46.0 | 10.3*** | 22.4% | 51.4 | 6.6*** | 12.8% | ||||||||
| By level of disadvantage: | Most disadvantaged (1) | Moderately disadvantaged (1) | Least disadvantaged (1) | ||||||||||||
| n.a. | Employed in last 3 mos. of FU | 33.3 | 10.9* | 32.7% | 44.9 | 9.8*** | 21.8% | 59.5 | 7.0** | 11.8% | |||||
| MFIP | Urban single parents recipients | By years of schooling: | Less than 12 | 12 or more | |||||||||||
| n.a. | Quarterly employment rate, year 3 | 35.7 | 9.5*** | 26.6% | 49.6 | 12.2*** | 24.6% | ||||||||
| By employment in year before RA: | No work in year before RA | Worked in year before RA | |||||||||||||
| n.a. | Quarterly employment rate, year 3 | 35.4 | 16.5*** | 46.6% | 57.8 | 4.3 | 7.4% | ||||||||
| By years on aid prior to RA: | 5 or more years | Less that 5 years | |||||||||||||
| n.a. | Quarterly employment rate, year 3 | 42.9 | 11.5*** | 26.8% | 47.7 | 10.8*** | 22.6% | ||||||||
| FIP | Recipients | By earnings in year before RA: | No earnings | Some earnings | |||||||||||
| n.a. | Any employment, year 2 | 59.2 | 2.2 | 3.7% | 81.0 | 1.5 | 1.9% | ||||||||
| By children less than 3: | Any children less than 3 | No children less than 3 | |||||||||||||
| n.a. | Any employment, year 2 | 68.7 | 4.4** | 6.4% | 71.9 | -0.3 | -0.4% | ||||||||
| Applicants | By earnings in year before RA: | No earnings | Some earnings | ||||||||||||
| n.a. | Any employment, year 1 | 48.9 | 6.3* | 12.9% | 85.8 | 2.7 | 3.1% | ||||||||
| By children less than 3: | Any children less than 3 | No children less than 3 | |||||||||||||
| n.a. | Any employment, year 1 | 70.0 | 6.9*** | 9.9% | 77.0 | 0.9 | 1.2% | ||||||||
| E. Programs that focus on other individual reforms | |||||||||||||||
| F. Programs that focus on TANF-like bundle of reforms (time limits with financial incentives, work-related activities, or both) | |||||||||||||||
| ABC | Recipients and applicants | By age of mother: | Under 25 | 25 to 34 | 35 and over | ||||||||||
| n.a. | Percent working at survey interview | 44.6 | 1.3 | 2.8 | 42.2 | 9.9*** | 23.5 | 36.1 | 11.5** | 31.8 | |||||
| By age of youngest child: | Under 3 | 3 to 5 | 6 and older | ||||||||||||
| n.a. | Percent working at survey interview | 43.7 | 2.6 | 6.1 | 48.8 | 4.5 | 9.3 | 34.1 | 16.8*** | 49.3 | |||||
| By when last employed: | 24 or more months before RA | 6 to 23 months before RA | Within 6 months of RA | ||||||||||||
| n.a. | Percent working at survey interview | 34.8 | 6.3 | 18.1 | 39.1 | 3.7 | 9.3 | 49.3 | 9.1** | 18.5 | |||||
| By years of schooling: | Less than 12 | 12 or more | |||||||||||||
| n.a. | Percent working at survey interview | 34.3 | 7.4** | 21.6 | 47.9 | 6.5** | 13.5 | ||||||||
| By years on aid in past 5 years: | 3 to 5 | 1 to 2 | Less than 1 | ||||||||||||
| n.a. | Percent working at survey interview | 37.8 | 6.4 | 17.0 | 42.1 | 11.8*** | 28.0 | 46.5 | 3.0 | 6.5 | |||||
| FTP | Recipients and applicants | By length of time limit: | 36 months (4) | 24 months (4) | |||||||||||
| n.a. | Ever employed in years 1-2 | 70.4 | 5.8** | 8.2 | 72.0 | 4.8** | 6.6 | ||||||||
| By age of youngest child: | Under 3 | 3 or over | |||||||||||||
| n.a. | Ever employed in years 1-2 | 71.2 | 6.6*** | 9.2 | 71.5 | 4.3** | 6.0 | ||||||||
| By level of risk: | Most risk (5) | Medium risk (5) | Least risk (5) | ||||||||||||
| n.a. | Quarterly employment, year 2 | 37.8 | 6.4** | 16.9% | 38.1 | 7.5*** | 19.7% | 58.4 | 5.4* | 9.2% | |||||
| JOBS First | Recipients and applicants | By status at RA: | Recipient | Applicant | |||||||||||
| n.a. | Avg. employment rate, years 1-2 | 41.8 | 10.9*** | 26.0 | 49.9 | 2.8* | 5.6 | ||||||||
| n.a. | Avg. employment rate, year 3 | 50.7 | 8.0*** | 15.8 | 55.3 | 3.7** | 6.7 | ||||||||
| By disadvantage: | Most disadvantaged (6) | Least disadvantaged (6) | |||||||||||||
| n.a. | Avg. employment rate, years 1-2 | 19.3 | 14.1*** | 73.0 | 65.9 | 3.7 | 5.6% | ||||||||
| n.a. | Avg. employment rate, year 3 | 30.2 | 11.6*** | 38.6 | 71.3 | 3.2 | 4.5% | ||||||||
| By age of youngest child: | Less than 6 | 6 to 11 | 12 to 18 | ||||||||||||
| n.a. | Avg. quarterly employment, Q1-6 | 44.1 | 7.2*** | 16.3% | 46.6 | 10.0*** | 21.5% | 38.0 | 7.6** | 20.0% | |||||
| n.a. | Any employment, Q8 | 49.2 | 9.1*** | 18.5% | 51.6 | 7.1*** | 13.8% | 43.0 | 5.5 | 12.8% | |||||
| By aid receipt in year before RA: | Long-term recipient (7) | Short-term recipient (7) | New applicant (7) | ||||||||||||
| n.a. | Avg. quarterly employment, Q1-6 | 37.7 | 12.1*** | 32.1% | 48.7 | 6.1*** | 12.5% | 46.8 | 3.1* | 6.6% | |||||
| n.a. | Any employment, Q8 | 44.4 | 11.3*** | 25.5% | 53.7 | 6.3** | 11.7% | 52.5 | 3.8 | 7.2% | |||||
| By employment in year before RA: | Not employed | Employed | |||||||||||||
| n.a. | Avg. quarterly employment, Q1-6 | 23.3 | 10.5*** | 45.1% | 61.9 | 5.2*** | 8.4% | ||||||||
| n.a. | Any employment, Q8 | 31.1 | 11.4*** | 36.7% | 66.0 | 4.4** | 6.7% | ||||||||
| By earnings in year before RA: | No earnings | $1 to 5000 | Over $5000 | ||||||||||||
| n.a. | Avg. quarterly employment, Q1-6 | 23.3 | 10.5*** | 45.1% | 55.0 | 6.9*** | 12.5% | 72.8 | 3.2 | 4.4% | |||||
| n.a. | Any employment, Q8 | 31.1 | 11.4*** | 36.7% | 61.0 | 5.0** | 8.2% | 73.8 | 3.6 | 4.9% | |||||
|
NOTES: For full program names
and citations, see Table 3.4. Significance tests for treatment-control
differences is indicated by: Abbreviations: Q=quarter; RA = random assignment; n.a. = not available. 1. Persons classified as "most disadvantaged" (1) had been on aid for at least 22 of the 24 months prior to random assignment; (2) had not worked in the prior year; and (3) did not have a high school credential. Persons classified as "least disadvantaged had none of these traits; persons classified as "moderately disadvantaged" had one or two. 2. "Most disadvantaged" consists of long-term recipients who did not have a diploma or GED and did not work for pay in the year prior to random RA. 3. Barriers include not having worked in the past six years; being arrested since age 16; having 2 or more children under six or four or more under 12; having been fired from one's longest-lasting jobs; and not having a high school credential. 4. Participants with low levels of education, short employment histories, and long welfare histories were assigned a 36-month time limit; other participants were assigned a 24-month time limit. 5. "Most at risk" sample members are those whose risk score is in the top quartile of the distribution of a "dependency index." "Least at risk" sample members are those whose risk score is in the bottom quartile of the distribution of the dependency index. "Medium risk" sample members are those whose risk score is in the interquartile range of the distribution of the dependency index. 6. "Most disadvantaged" is defined as having no high school credential, not having worked in the year prior to RA, and having been on aid at least 21 of the 24 months prior to RA. The "least disadvantaged" meet none of these criteria. 7. "Long term" recipients are those who received aid for at least 22 of the 24 months prior to RA. "Short term" recipients are those who received aid for 1 to 22 months during the 24 months prior to RA. "New applicants" are those who received no aid during the 24 months prior to RA. |
A.2.1. Programs That Focus on Financial Work Incentives
The two programs that focus on financial work incentives provide no clear evidence that the effects of this policy vary according to recipients’ level of disadvantage (Panel A). Although the group-specific employment impacts vary for WRP-IO, they are insignificant and display no consistent pattern. MFIP-IO has greater effects for participants with higher levels of schooling, but also for persons with shorter employment histories and longer welfare histories. The estimates in Table A.4 show that these programs had insignificant effects on earnings for all the subgroups, which is consistent with the finding that the programs had no significant effect on third-year earnings overall, as shown in Table 5.1.
A.2.2 Programs That Focus on Financial Work Incentives Tied to Hours of Work
As seen in Panel B, the first-year employment effects of New Hope are essentially uniform across groups, defined by differing numbers of barriers to employment, although there are significant differences in the second-year effects. In both years, employment impacts for whites are insignificant and smaller than the impacts for blacks and Hispanics. The effects of SSP are nearly uniform across the distribution of the age of the youngest child, the mother’s education, employment history, and welfare history. Indeed, formal tests indicate that these effects are homogenous across levels of disadvantage for all the disad