Table of Contents | Previous | Next |
11. ANALYSIS OF BENEFIT-COST FINDINGS
To determine whether welfare-to-work programs are socially efficient, evaluations cannot be limited only to effects on persons in the program. They must take account of effects on society, which includes persons not in the program. If a program has a positive earnings effect on those in the program, then the effect on society may be positive or negative, depending on the costs incurred by the government in operating the program. The usual way of taking account of societal effects is through benefit-cost analysis. An illustration of the framework that is typically used in conducting benefit-cost studies of welfare-to-work programs appears in Table 18.21
Only benefits and costs that are typically estimated are listed in Table 18. Dollar amounts are indicated in the table as resulting from program impacts on earnings, tax payments, output produced while participating in work experience, AFDC payments and payments from other government transfer programs, and net operating costs. The plus and minus signs indicate whether each amount is expected to be a benefit (+) or cost (-) from the perspectives of four groups: persons assigned to the program, non-assignees (i.e. all persons outside the program group, including taxpayers who pay the cost of operating the program), the government, and the whole of society (those assigned plus those not assigned). As indicated, it is usually assumed that benefits and costs to non-assignees and the government are identical except that the former benefits from output individuals produce while assigned to work experience programs and the government does not. As can be seen, benefits and costs to society are simply the algebraic sum of benefits and costs to those assigned and those not assigned to a program. Hence, the framework implies that if a welfare-to-work program causes a decline in transfer payments received by program group members (for example, in AFDC receipts and food stamps), then this decline should be regarded as a cost to program group members (albeit one that may be offset by earnings increases); as a savings or benefit to taxpayers; and as neither a benefit nor a cost to society, but simply as a transfer of income from one segment of society to another.
One goal of benefit-cost analyses of welfare-to-work programs, as the last row of Table 18 suggests, is to determine whether the program being evaluated has positive or negative net benefits (i.e., benefits less costs) from each of the four perspectives represented by the four columns. The societal perspective is usually viewed by economists as the appropriate one for assessing the efficiency of social programs. Policymakers, however, often focus on the government perspective, that is, on whether the program increases or decreases government budgetary requirements.
11.1 DESCRIPTIVE ANALYSIS
Table 19 provides summary information on net benefits from each of the four perspectives. Estimates of the government’s net operating costs are also presented. The estimates are measured in terms of costs and benefits per individual assigned to the evaluated interventions. The estimates in Table 19 are for the 49 benefit-cost studies included in our database. One set of estimates is used from each study. Because most evaluations of welfare-to-work programs conduct only one benefit-cost study, summing annual or quarterly estimates and projections of benefits and costs over several years (most often five), only one set of estimates is usually available. If more than one estimate was available, however, we use the most recent.
Both unweighted and weighted means are provided in Table 19. However, because net benefit estimates are a composite from several different impact estimates, and also incorporate information from other sources, standard errors do not exist for them. Yet, the estimates are subject to sampling error. Thus, weighting is appropriate. As mentioned earlier in the report, the weights we use are the square roots of the number of observation in the total evaluation sample.
Except for the program group, the median net benefit estimates in Table 19 are somewhat larger than their corresponding means, indicating that some under-performing interventions are pulling down the average. In addition, the weighted values are larger than their unweighted counterparts, implying that that evaluations with larger samples tend to produce larger net benefit estimates than those with smaller samples. It is not apparent why this should be the case. Net benefits received by non-assignees are somewhat larger than those received by the government, although not by very much. This is unsurprising because, as shown in Table 18, the computations of net benefits from these two perspectives are identical, except that non-assignees are credited with the value of output produced in work experience programs and the government is not. In most welfare-to-work programs, the value of such output is small, and in many it does not exist.
Keeping in mind that, unlike the estimates of impacts on earnings and AFDC payments reported in Table 2, the mean and median net benefit estimates in Table 19 are intended to capture the total effects of interventions over several years, not just effects for a single quarter, they are surprisingly small from all four perspectives. The largest are a little over $500 and the smallest a couple hundred dollars below zero. The weighted medians, which tend to be the largest values, indicate that society receives net benefits of around $500 from a typical welfare-to-work program, savings to the government are around $400, and the welfare of program group members are barely affected.
It is likely that the net benefits of a typical welfare-to-work program are actually even smaller than the estimates in Table 19 imply because benefit-cost analyses are less likely to be conducted for those programs with especially small impacts. For example, the weighted means of the estimated earnings impacts in the 3rd and 7th quarters for our sample of studies were over twice as large for those interventions for which benefit-cost analyses were conducted as for those for which they were not. The gap is less striking for the unweighted averages but still substantial.
Thus, it seems apparent that the net benefits from a typical welfare-to-work intervention are modest indeed! However, as the standard deviations and the minimum and maximum values reported in Table 19 make clear, the variation across the 49 studies for which benefit-cost analyses were conducted is enormous. Thus, according to the benefit-cost findings, there were some impressively successful welfare-to-work programs and a few spectacular failures.
As is the case with the impact estimates presented in Table 2, this variation is due both to true differences among programs and to sampling error. Unlike the impact analysis, however, it also results because benefit-cost analyses of welfare-to-work programs are based on a large number of assumptions (see Boardman et al., 2001, Chapter 11 for a detailed discussion) and different evaluators make somewhat different assumptions. Although a discussion of these assumptions is beyond the scope of this report, this last factor suggests that some of the individual cost-benefit estimates may be subject to considerable error and these errors may differ in unsystematic ways across the studies.
11.2 REGRESSION ANALYSIS
To determine the factors that might influence the size of the net benefits from welfare-to-work, we conducted a regression analysis with the net benefit estimates from all four perspectives as dependent variables and the variables used in the previously discussed regressions on impacts as explanatory variables. The regressions were weighted by the square root of the sample size. However, we also estimated (unreported) unweighted regressions, which turned out to be very similar. The regression estimates are presented in Table 20. Because of inaccuracies and inconsistencies in the estimates of net benefits, because benefit-cost analyses are conducted for a somewhat unrepresentative subset of all evaluations, and because it was not possible to use the weighting scheme recommended in the meta-analysis literature, findings from these regressions should be considered as exploratory and viewed with considerable caution.
Perhaps, at least in part, because of these shortcomings, relatively few of the coefficients in Table 20 are statistically significant. However, there may be other reasons as well. For example, as we saw earlier, greater use of job search increases the earnings of participants, but it also reduces their transfer benefits. Thus, as implied by the first two columns of Table 20, the net effect of job search on benefits received by those assigned to welfare-to-work programs (the program group) is small and statistically insignificant. Similar reasoning suggests that the negative, but insignificant, coefficients on sanctioning in the first two columns may result because increases in sanction rates cause program group members to lose more in transfer benefits than they gain in earnings. It is more difficult to interpret the negative, marginally statistically significant, and fairly large effect of increases in participation in vocational education on the net benefits received by program group members. However, some of our earlier reported regressions implied that there was a negative relationship between vocational education and earnings.
Some of the other findings in Table 20 are straight-forward to interpret. For example, two-parent families programs appear to benefit substantially less than one-parent families from assignment to welfare-to-work programs and this translate into lower benefits to society from requiring two-parent families to participate in these programs.22 The regressions on net benefits received by non-assignees and the government imply that these benefits are larger when AFDC payments are more generous. This is consistent with some of the earlier reported regressions that suggest that the impacts of welfare-to-work programs on welfare payments increase with the generosity of the AFDC system. Recall, however, that sensitivity tests indicated that this result is not very robust.
Unsurprisingly, net benefits are higher for those assigned to programs that offer financial incentives than for programs that do not provide these incentives, and smaller for non-assignees and the government. These effects are large and highly statistically significant. The two columns on the right of Table 20 indicate that the increases in net benefits to the program group are offset or nearly so by the reductions in non-assignee and government benefits. Thus, the social cost of financial incentives appears to be small or negligible. Because our earlier findings suggest that they do little to increase employment or earnings, financial incentives that are provided through welfare-to-work programs are perhaps best viewed as simply transferring income from the government to low wage welfare recipients who find jobs. Their small social costs suggests that they are nonetheless efficient in this respect, as some research has suggested that it typically costs taxpayers about $1.50 to $2.00 to transfer one dollar to low income persons (for example, see Gramlich, 1990, pp. 123-127 and Browning and Johnson, 1984). However, the results shown in Table 20 do not take account of potential social costs resulting from distortions in the labor supply and investment behavior of taxpayers, which are caused by transfer programs.
In interpreting some of the remaining regression coefficients in Table 20, it is helpful to be aware of how increases in participation in various services affect the net operating costs of welfare-to-work programs. Operating costs are the cost to the government of providing program services, but do not include transfers of income, such as AFDC payments. Table 19 indicates that these costs were around $1800 (in year 2000 dollars), on average, although the median value is considerably smaller. A weighted regression on net operating costs appears below (with the standard errors in parentheses):
| Constant | 132.764 (455.47) |
| Intervention impact on % sanctioned | 110.711 *** (26.52) |
| Intervention impact on % participated in job search | 4.623 (13.54) |
| Intervention impact on % participated in basic education | 104.051 *** (18.83) |
| Intervention impact on % participated in vocational education | 48.174 (50.72) |
| Intervention impact on % participated in work experience | -9.611 (35.54) |
| Intervention included financial incentive=1 | 520.460 (493.07) |
| Adjusted R-squared | .549 |
These results imply that program net costs increase by over $100 per program group member for every percentage point increase in participation in basic education and by nearly $50 for every percentage point increase in vocational education, although the latter estimate is statistically imprecise. Increases in participation in work experience do not seem to increase costs. This may be because work experience participants are often assigned to agencies other than those operating welfare-to-work experience and whatever operating costs are involved may not get incorporated into the net cost estimates. Consistent with usual views on the topic, increases in participation on job search appear to result in very small increases in cost, although the estimate is very imprecise. Although the coefficient is also imprecisely estimated, financial incentives appear fairly costly to administer. Finally, the estimates also indicate that one percentage point increase in the sanction rate cost over $100, presumably because of agency expenditures required for administering and enforcing sanctions.
Turning back to Table 20, it is not surprising that because basic education is costly to provide, it significantly reduces the net benefits to non-assignees and the government. Our earlier results suggested that basic education has a more or less negligible net effect on the earnings and transfer receipts of program group members. Thus, there are few benefits to offset these costs. There is also some indication in Table 20 that net benefits to non-assignees and the government are reduced by increasing participation in vocational education, although this result is statistically insignificant and disappears once the socio-economic contextual variables are added to the regression model. Although sanctions are somewhat costly, our previous results indicate that they also reduce AFDC payments. The findings in the middle columns of Table 20 suggest that this reduction in AFDC benefits may more than offset increases in costs that result from sanctions. Table 20 also suggests that increases in participation in job search do not increase net benefits received by non-assignees and the government and may even decrease them a bit. This finding cannot be easily reconciled with our previously discussed results, which indicate that job search reduces transfer payments and is inexpensive to provide; but may result from some of the limitations of the regressions on net benefits mentioned earlier.
| Table of Contents | Previous | Next |