Table of Contents | Previous | Next |
APPENDIX A
The Welfare-to-work Program (Meta-Analysis) Database
The welfare-to-work program database is the largest and most comprehensive information source on evaluations of U.S. welfare-to-work programs that is currently available. A public use copy of it can be obtained from the Office of Planning, Research and Evaluation (OPRE) of the U.S. Department of Health and Human Services, Administration for Children and Families. The database can be used in several different ways. For example, it allows researchers to conduct complex statistical studies of welfare-to-work programs. It also provides a readily accessible means for analysts to make simple comparisons among welfare-to-work programs. A Users’ Guide that provides a detailed description of the database is also available from OPRE. In this report, we only give a brief description of the database.
The database is maintained in Microsoft Access. It contains detailed information on each of the random assignment evaluations listed in Table 1 in the main body of the report. In the database, evaluation studies are first classified by evaluation, then within each evaluation by site, and then within each site by program. For example, in the case of the very large, and complex, National Evaluation of Welfare-to-Work Strategies (NEWWS), there is one evaluation and there are 7 sites. For three of these sites, there is one program in each; and for the remaining four sites, there are two programs in each. Thus, there are separate sets of program impact estimates for 11 interventions. In our analysis, we treat each as a separate observation. In addition, for each of the 11, there are separate sets of effect estimates for subgroups (for example, AFDC applicants and AFDC recipients, and persons with and without a high school degree). In some (but not most) of the research described in this report, each of the subgroup impact estimates is treated as a separate observation.
Although the NEWWS evaluation was limited to program impacts on one-parent families on AFDC, other evaluations estimated separate impacts for one- and two-parent AFDC families (one prominent example is the GAIN evaluation that was conducted at six sites in California during the late 1980s and early 1990s). When available, these separate impact estimates are both included in the database and treated in the analysis as separate observations (that is, as pertaining to separate interventions).
The database is divided into five “levels”:
-
Average earnings during the quarter or year;
-
Percentage ever employed during the quarter or year;
-
Average AFDC payment amount during the quarter or year
-
Percentage ever in receipt of AFDC payments during the quarter or year;
-
Net program benefits (that is, program benefits less program costs) from the perspectives of the experimental (or program) group, the control group, the government, and society as a whole
-
Assorted measures of program impacts on children (e.g., parent reports of school achievement, general health, behavior problems, and ever suspended or expelled from school) are recorded for three separate age groups (young, school age, and adolescents).
Level 1 lists the title of the evaluations, their evaluators, and the reports used in constructing the database.
Level 2 contains the relevant sampling information for each intervention, including the sample sizes and the characteristics of the sample population (gender, ethnicity, age/age group, education, number of children, welfare and employment experience prior to random assignment, and so forth).
Level 3 contains both annual and quarterly estimates of program impact measures, as well as their levels of statistical significance. Impacts are recorded for all the years (up to five) and all the quarters (up to 20) for which they are available. Whenever it was possible, the impacts were recorded separately by program site. Program impacts were collected for one-parent families and two-parent families separately, whenever they were available. In addition, whenever they were available, impacts were recorded for both the overall program target group and for program sub-groups. The impact measures in Level 3 include:
Level 4 records program participation statistics (for example, overall program participation rates; program sanction rates; and rates of participation in job search, basic education, vocational training, and work experience). Whenever it was available, this information was obtained separately for the program treatment group and the control group and separately for different program sites. Level 4 also records the gross and net cost per program group member to the government of operating each program and indicates whether each program tested financial incentives and time limits. Information is also provided on the dollar value of the financial incentives tested by the evaluated programs and on the characteristics of sanctioning under the program.
Level 5 contains socio-economic background data for each of the program sites during each of the evaluation years, including the unemployment and poverty rates, the percentage of the workforce in manufacturing employment, the annual percentage change in manufacturing employment, median household income, and the maximum AFDC payment for which a family of three is eligible. Unlike the data for levels 1 through 4, which were extracted directly from the evaluation reports, the level 5 data were obtained from government sources, mainly the U.S. Census Bureau and the U.S. Bureau of Labor Statistics websites.
All levels of the database are linked via unique identifiers for each of the evaluations and evaluation sites and, therefore, are available for analysis. For reasons of comparability, all the financial data (including earnings, AFDC payments received, median household income, and maximum AFDC payments) have been inflated to year 2000 U.S. dollars.
| Table of Contents | Previous | Next |