Report on a Meta-Analysis of Welfare-to-Work Programs

This report uses meta-analysis, a set of statistically based techniques for combining quantitative findings from different studies, to synthesize estimates of program effects from random assignment evaluations of welfare-to-work programs and to explore the factors that best explain differences in the programs' performance. The analysis is based on data extracted from the published evaluation reports and from official sources. All the programs included in the analysis targeted recipients of Aid to Families with Dependent Children (AFDC; now called Temporary Assistance for Needy Families, TANF). The objective of the analysis is to establish the principal characteristics of welfare-to-work programs that were associated with differences in success, distinguishing between variations in the services received, differences in the characteristics of those who participated in each program, and variations in the socio-economic environment in which the programs operated.

Meta-analysis is a powerful instrument for analyzing the combined impacts of comparable policy interventions, while controlling for a range of factors pertaining to these interventions or the environment, in which they took place. However, like other statistical techniques, meta-analysis can be subject to data limitations that adversely affect its capacity to produce robust and reliable results. Multi-collinearity of variables (resulting from small sample size), inconsistencies in the information provided in different evaluation reports, and omitted or misspecified variables are some of the data analysis risk that we sought to minimize, for instance, by verifying data entries and carefully considering the specification of the regression equations that are estimated. Yet, it would have been impossible, as well as impractical, to eradicate all risk of error in the analyses, much of which would have beyond the control of this study and could be traced back to the original evaluations. In the light of such limitations, many of the conclusions that are reached are subjected to sensitivity tests. These tests were conducted to establish the robustness of the meta-analyses’ key findings.

Separate meta-analyses of both voluntary and mandatory programs were conducted. Voluntary programs provide services (e.g., help in job search, training, and remedial education) for those who apply for them and they sometimes provide financial incentives to encourage work. Mandatory programs are targeted at recipients of government transfer payments. They also provide employment-orientated services and sometimes provide financial work incentives, but differ from voluntary programs by requiring participation in the services by potentially subjecting individuals assigned to the program to fiscal sanctions (i.e., reductions in transfer payments) if they do not cooperate.