Table of Contents | Previous | Next |
1. INTRODUCTION
The research presented in this report uses meta-analysis to conduct a statistical synthesis of findings from random assignment evaluations of welfare-to-work programs and to explore the factors that best explain differences in performance. The analysis is based on data extracted from the published evaluation reports and from official sources. All the programs included in the analysis were targeted on recipients of Aid to Families with Dependent Children (AFDC).2 The objective of the analysis is to establish the principal characteristics of welfare-to-work programs that were associated with differences in success, distinguishing among variations in the services received, differences in the characteristics of those who participated in each program, and variations in the socio-economic environment in which the programs operated.
In a very real sense, the origins of this report can be traced back to the greater use of the “1115 waiver authority” in the 1980s and, especially, the 1990s. Although available since the 1960s, waivers were increasingly applied for by U.S. states that wanted to experiment with their welfare provisions, including not only welfare-to-work programs, but also involving child support measures and Food Stamps and Medicaid provisions. The federal government became more receptive to the idea of welfare-to-work experimentation and increasingly granted state waivers, leading to a rapid rise in new welfare-to-work programs being tried and tested. In exchange, states were usually required to evaluate the policy changes they implemented, with the federal government increasingly requiring more rigorous evaluations including the use of random assignment.
Thus, the waivers eventually spawned a plethora of evaluations of welfare-to-work programs designed to promote work and reduce welfare caseloads, the results of which have been widely disseminated (Greenberg and Shroder, 2004; Friedlander, Greenberg, and Robins, 1997; Gueron and Pauly, 1991; Greenberg and Wiseman, 1992; and Walker, 1991). The evaluations measured the effects (usually called “impacts”) of welfare-to-work programs on outcome indicators, such as the receipt of welfare, the employment status of welfare recipients, their earnings, and the amount of welfare benefit they received. Some, but not all, of these evaluations also estimated the overall costs and benefits of the evaluated programs. Some of the more recent evaluations have also measured program impacts on measures of the welfare of the children of program participants. These impact measures are all examined in the report.
While the evaluations were able to gauge the effectiveness of each welfare-to-work program, they were rarely able to determine reliably the program features that contributed to success or failure. For instance, social and environmental conditions affecting program sites were seldom taken into account, nor were the characteristics of programs. In fact, they did not need to be because the evaluation designs used by many studies, based on random assignment of welfare recipients into experimental and control groups, guaranteed that individuals in the two groups shared environmental conditions and characteristics. In addition, evaluations often recorded impacts for only the first one, two, or three years after program implementation and were thus unable to assess the long-term performance and viability of interventions. Because their evaluation period was short-term, there was again less need to control for conditions that might have affected impacts over time.
Meta-analysis provides a statistically based means for assembling and distilling findings from collections of policy evaluations. The approach is based on a well-established statistical methodology. Based on a comprehensive, systematic review of available evidence, meta-analysis is a check against unwarranted generalizations and unfounded myths and, therefore, can help lead to a more sophisticated understanding of the subtleties of policy impacts.
The remainder of this report first provides additional background information, including a discussion of previous statistical syntheses of welfare-to-work programs and a description of the specially constructed database of welfare-to-work evaluations that is used in the study. It then outlines the methodological principles of meta-analysis. This is followed by a discussion of findings from a formal meta-evaluation of the welfare-to-work programs in our sample. Finally, the findings are summarized and conclusions are drawn about their policy implications.
| Table of Contents | Previous | Next |