Table of Contents | Previous | Next |
SUMMARY
The Research Synthesis project was undertaken to inform public debate on issues relating to the reauthorization of the Temporary Assistance for Needy Families (TANF) program in 2002 and to help states in refining the designs of their TANF programs. To this end, this report–the final report of the project–synthesizes the current state of knowledge about the effects of the TANF legislation and the TANF programs of individual states. It considers a range of outcomes, including the welfare caseload, employment and earnings, use of other government programs, fertility and marriage, household income and poverty, food security and housing, and child well-being. The primary focus is on the net effects of TANF, taking into account the impact of other factors such as the economy and other policy changes that may have affected the outcomes of interest. Like the literature on which it is based, the synthesis considers both the effects of specific policies, such as benefit structures, time limits, work requirements, and sanction policies, as well as the effect of the TANF reforms as a whole.
We begin this summary by highlighting the key findings from the synthesis. We then provide some relevant background for the study, describe our approach to conducting the synthesis, and discuss the results in more detail, including the limitations of the current knowledge base. Finally, we consider directions for future research.
FINDINGS IN BRIEF
The following key findings emerge from our synthesis of the research literature:
- Many of the effects of welfare reform on welfare use have been well studied. Over a dozen econometric studies have attempted to estimate the effects of welfare reform taken as a whole, and all but a few report that reform had substantial effects on reducing the caseload.
- Most of the reforms that were introduced in the 1990’s had positive effects on employment and earnings. Thus, it seems likely that welfare reform is responsible for a portion of the increase in work and earnings among single mothers during the last decade. Nearly all of the evidence, from both experimental and econometric studies, points in this direction.
- There is little information available about the effects of welfare reform on the use of other government programs. The research is generally consistent with the hypothesis that welfare reform has caused part of the recent decline in food stamp use, but it does not explain the mechanisms that underlie this linkage. Only a few studies analyze the effects of welfare waivers on the use of school nutrition programs; fewer still have considered housing subsidies. The existing evidence also provides too narrow a basis to draw general conclusions about whether welfare reform caused part of the initial decline in Medicaid enrollment that followed the implementation of TANF.
- The evidence from both experimental and econometric studies is insufficient to draw any firm conclusions about the effects of welfare reform on marriage or fertility. Evidence from the Minnesota Family Investment Program (MFIP) suggests that providing generous financial work incentives, either alone or with work requirements, may increase marriage or keep existing marriages intact. The mixed results from the Canadian Self-Sufficiency Project (SSP), however, suggest caution in interpreting the MFIP results.
- Some welfare reform components can raise incomes and reduce poverty, although this result is not associated with all policy components and there is reason to believe that some of the initially favorable effects will not persist over time. Generous financial work incentives–high earned income disregards inside the welfare system or earnings supplements outside the welfare system–generate the strongest income gains and anti-poverty effects.
- There is evidence of both positive and negative effects on child well-being of various components of welfare reform. Positive and negative effects were observed for indicators that capture socio-emotional behavior, academic performance, and health. The most favorable effects are associated with financial work incentives, most likely because of the increase in family income that results from combining work and welfare. But even for these programs, there is some evidence of unfavorable impacts for some subgroups of participants, particularly for adolescent children and for younger children of parents who do not experience large income gains. Work requirements do not appear to have strong impacts on children, either favorable or unfavorable, although again there is evidence of unfavorable impacts for adolescents, especially in school performance.
BACKGROUND
The last decade has witnessed significant changes in welfare policy, beginning with state waivers under the Aid to Families with Dependent Children (AFDC) program and culminating in the Temporary Assistance for Needy Families (TANF) block grant implemented by the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA). These reforms produced changes in the structure of benefits, introduced time limits, strengthened requirements for mandatory participation in work-related activities, and changed various administrative procedures.
During the same period, welfare-related outcomes have also changed, with the welfare caseload falling to approximately 2.1 million families as of September 2001, less than half its all-time peak level of 5.0 million families in 1994; with the fraction of welfare recipients participating in welfare-to-work activities, or actually working, increasing rapidly; with employment rates and earnings of single mothers rising substantially; and with family income increasing and the poverty rate falling. These improvements in labor market and economic outcomes have been accompanied by a leveling off of the prior upward trend in nonmarital fertility.
Because these changes occurred as welfare reform took place, some observers have concluded that welfare reform caused them. However, this inference ignores the fact that other policy changes–such as increases in the Earned Income Tax Credit (EITC), expansion of subsidized health insurance de-linked from welfare receipt, and increases in the minimum wage–took place during the same time. Perhaps most important, there was a long and robust economic expansion. Thus, at least some of the improvements in welfare-related outcomes likely resulted from changes in other policies and the improving economy, rather than from changes in welfare programs.
Understanding the role welfare reform played in general–and the role that specific reform policies played in particular–is important, because policymakers at all levels are once again debating the direction of welfare policy. The fiscal provisions of PRWORA must be reauthorized by September 30, 2002, and it seems likely that that reauthorization will serve as an opportunity for Congress to review the decisions made in the 1996 legislation. Any significant changes at the federal level will, in turn, require subsequent changes at the state and local levels. In addition, it seems likely that many state legislatures will use the opportunity of federal reauthorization to revisit their choices about how to implement TANF in their states.
With funding from the Administration for Children and Families (ACF) of the U.S. Department of Health and Human Services (USDHHS), we have synthesized the current state of knowledge from the growing base of research literature on the effects of welfare policies, with the aim of informing the ensuing debates about TANF reauthorization and state implementation.
HOW WE CONDUCTED THE SYNTHESIS
Conceptually, we organized the synthesis using the two-dimensional matrix presented in Table S.1. Along the rows are reform policies, preceded by TANF as a whole (labeled "TANF as a bundle") and followed by its main components; along the columns are welfare-related outcomes. Table S.1 is only exemplary. The following is a complete list of the policies studied: (1) financial work incentives, including earnings disregards and benefit reduction rates; (2) requirements to work or participate in work-related activities (and sanctions for failing to satisfy those requirements); (3) time limits on the receipt of benefits; (4) family caps and minor residence requirements; and (5) parental responsibility requirements. The full set of outcomes includes the following: (1) welfare use and the caseload; (2) employment and earnings; (3) utilization of other government programs (e.g., food stamps and Medicaid); (4) family structure, specifically marriage and fertility; (5) income and poverty; (6) other measures of well-being (e.g., food security, housing security, and health insurance coverage); and (7) child well-being (e.g., child development and school progress).
Ideally, the goal of the synthesis would be to fill in each of the cells in the matrix, expressing in a common format how each policy affected each outcome. However, this goal is not realistic, because the literature we synthesize has not yet covered each policy-outcome pair. Moreover, some of the cells in the matrix are "populated" by studies that, for one reason or another, are too tentative or otherwise inconclusive to be relied on for policy purposes. Nevertheless, the empty cells provide useful diagnostic information: They tell us where additional research results would help in making policy decisions and where we do not have enough knowledge.
While the above paragraph makes the synthesis effort seem straightforward, filling in the cells in the matrix is not straightforward. The synthesis aims to answer the question: What is the effect of a given policy (e.g., a lower benefit reduction rate or a time limit) on a given welfare-related outcome (e.g., the caseload or child development), holding all else equal? If all else is not held equal, then confounding influences–or more simply, confounders–can yield misleading results. There are two general research strategies for dealing with confounding influences: random assignment and econometric methods using observational data. After an extensive review of available studies, we identified 34 random assignment studies and 33 econometric studies that form the basis for our synthesis across the policies and outcomes.1
|
Policy Reforms |
Outcomes (Impact on What) |
||||
|---|---|---|---|---|---|
|
Caseload |
Employment |
Earnings |
Income |
. . . |
|
|
TANF as a "bundle" |
|||||
|
Specific TANF policies |
|||||
|
Work-related activity requirements |
|||||
|
Time limits |
|||||
|
Financial work incentives |
|||||
|
. |
|||||
Each of these approaches has strengths and weaknesses
and may be more or less successful in controlling for confounding factors.
One strength of the random assignment studies is that, when implemented
well, they control for all non-random confounding factors. One drawback
of these studies is that they were largely conducted prior to the enactment
of PRWORA and are not necessarily representative of the policies adopted
by the states under TANF. In contrast, the econometric studies exploit
the observed variation in policies as they were adopted over time and
across states during the waiver period before PRWORA’s passage and
after the passage of PRWORA. In the absence of experimental conditions,
these observational studies control for as many confounding factors as
possible but they are subject to the critique that there may be other
unobserved factors that are responsible for some of the changes in behavior
rather than changes in welfare policies.
Another difference between the two approaches concerns entry effects. Random assignment studies estimate effects from the moment of randomization, usually for those already on (or at least applying for) welfare. They thus miss any effects of policies on the likelihood of applying for welfare or on behaviors such as education, marriage, and fertility that might make one less likely to apply for welfare. In contrast, observational studies are able to analyze the entire population and thus can capture entry effects.
In filling in the matrix, it is not enough simply to tally all the findings across the available studies. Rather, we implement an approach for weighing the findings for each analysis and assessing the strength of the cumulative evidence for each policy-outcome pair. Our approach accounts for the number of studies in any given cell, the quality of the methodology employed, and the statistical significance of the estimated impacts. Using this approach, our synthesis assigns a qualitative summary of the direction of the effect of each policy on each outcome and an indicator of the depth of the knowledge base associated with that effect; that is, how much is known about the effect of that policy reform on that outcome.
WHAT THE SYNTHESIS SHOWS
Table S.2 shows the filled-in version of the matrix in Table S.1. The columns of the matrix correspond to the outcomes we listed above. In the case of child well-being, given the expectation that impacts may differ by the age of the child, we assess the impacts for three age groups defined at the time of follow-up: preschoolers, grade schoolers, and adolescents. The rows of the matrix correspond to the reform policies or groups of policies for which there is some evidence base in the literature. We differentiate between the impact of financial work incentives (e.g., higher earned income disregards, lower benefit reduction rates) alone (row (1)) or when tied to hours of work (row (2)) or when combined with mandatory work-related activities (rows (5) and (6)). In the latter case we differentiate between strong and weaker financial work incentives, where programs in the latter category involve implicit tax rates that may be higher than those under AFDC.2 We also consider the impact of mandatory work-related activities alone (row (3)) and the sanctions that accompany them (row (4)). For time limits, we distinguish pre—time limit effects from post—time limit effects (row (7) and (8)). We also assess the impacts of family caps (row (9)) and parental responsibility requirements (row (10)), two of the other reform policies that have received some study. The effects of reform as a bundle, in row (11), pertain only to the pre—time limit period.
The cell entries qualitatively summarize the effect of each policy on each outcome. The words indicate the direction of the effect, while the shading indicates how much is known about the effect of that policy reform on that outcome (the knowledge base). Starting first with the effect, "increase" indicates that a majority of the studies that populate the policy-outcome pair in question show that the policy increases the outcome, "decrease" indicates the opposite, while "mixed" indicates that there are roughly as many results showing a decrease as an increase. "No change" indicates the estimated impacts are mixed in sign and nearly all are small and insignificant.
Turning to the knowledge base, cells populated by several high-quality studies, most of which yield similar and significant estimates, are indicated by the dark gray shading. At the other end of the spectrum, cells with a shallow knowledge base are indicated by no shading. These cells are populated by a single high-quality study that yielded a significant estimate, two moderate-quality studies that yield similar and significant estimates, or similar constellations of evidence. Cells between these two categories are indicated by intermediate gray shading. Cells populated by a single moderate-quality study, or one or more high-quality studies whose results were insignificant, are indicated by an asterisk, denoting a nearly empty knowledge base. Cells for which there are no studies are left blank.
The entries in column (B) indicate that most reforms or combinations of reforms considered in Table S.2 increase employment, although we assign varying degrees of confidence to this qualitative assessment. Column (C) shows that many of these policies also increase earnings. Beyond employment and earnings, however, the impacts of specific policies vary to a greater extent. Thus, we organize the remainder of our discussion of Table S.2 by the table rows.3
What Does the Synthesis Tell Us About the Effect of Financial Work Incentives?
Although all the recent reform policies are capable of increasing employment (column (B)), they involve different trade-offs between reducing welfare use (column (A)) and increasing income (column (H)). Programs with generous financial work incentives generally increase welfare use, as seen in the intersection of column (A) and row (1)–which we refer to hereafter as cell A1. This is also true for financial work incentives tied to hours of work through an earnings supplement outside the welfare system (cell A2), where transfer payments (welfare payments or the earnings supplement) increase, although welfare use per se may decrease. Welfare use also increases when generous financial incentives are combined with mandatory work-related activities (cell A5), but the opposite is true for programs with weaker incentives (cell A6).
We are unable to assign a direction for financial work incentives alone on the use of food stamps or Medicaid (cells D1 and E1). A shallow evidence base suggests financial work incentives tied to hours of work may increase food stamp use (cell D2). When combined with work requirements, it appears that both strong and weak financial work incentives may decrease food stamp use (cells D5 and D6), but the effect on Medicaid use depends on the strength of the financial work incentives. A very shallow knowledge base suggests that financial work incentives alone may increase marriage (cell F1), but we do not have enough evidence to say how marriage is affected when financial work incentives are tied to hours worked or combined with work requirements for single parents. There is some suggestive evidence from MFIP that programs that provide generous financial work incentives combined with work requirements may increase marriage or keep existing marriages intact. However, the mixed results for the Canadian SSP suggest caution in interpreting the MFIP results. There is no evidence base from which to assess the relationship between financial work incentives and fertility.
Since financial work incentives allow families to keep more of their benefits as their earnings rise, they also increase income and decrease poverty, as shown in columns (H) and (I). However, even the programs that do increase income do so by only modest amounts, even when the financial work incentives are quite generous–more generous than most state TANF plans. With one exception (cell K6), financial work incentives (alone, or tied to hours of work, or combined with work requirements) are also associated with improvements in other measures of well-being such as food security, children’s health insurance coverage, and savings (the intersection of rows (1), (2), (5) and (6) with columns (J) to (K)). However, several of the relevant cells are empty, and those that indicate a favorable impact are derived from a shallow knowledge base.
The impact on child well-being is more uncertain, and, when we are able to assign a direction, it is almost always based on a shallow evidence base. For children who are school-aged at the time of follow-up, strong financial work incentives (alone, or tied to hours of work, or combined with work mandates) appear to decrease behavior problems and possibly also school achievement problems as well (the intersection of rows (1), (2), (5), and (6) with columns (Q) to (S)). The available measures of child health suggest potential unfavorable effects for financial work incentives alone (cell S1) or when combined with work requirements (cell S5), but the reverse is true for financial work incentives tied to hours of work (cell S2). Thus, it appears that for this age group, the increased income associated with reforms that incorporate strong financial work incentives may lead to some improvements in children’s outcomes in certain domains.
In contrast, for adolescents at follow-up, the various policies that include financial work incentives consistently appear to increase behavior problems and school achievement problems (the intersection of rows (1), (2), (5), and (6) with columns (T) and (U)). The evidence base available to assess the impact of financial work incentives on outcomes for pre-school-aged children at the time of follow-up is almost nonexistent although some of the impacts recorded in Table S.2 for grade-school-aged children pertain to children who were preschoolers at the time of random assignment.
What Does the Synthesis Tell Us About the Effect of Mandated Work-Related Activities?
Mandated work-related activities have been studied more than any other reform. Consequently, most of the cells with the darkest shading are in row (3). A substantial body of evidence shows that they generally reduce welfare use (cell A3). When implemented in conjunction with the AFDC benefit structure, in which benefits fell nearly dollar-for-dollar as earnings rose, however, they have little effect on income (cell H3). Even so, the limited evidence available suggests that these programs decrease poverty somewhat (cell I3). This may indicate that such programs are able to raise incomes for families just below the poverty line. This is consistent with the evidence that such programs have greater effects on income among relatively advantaged recipients than among disadvantaged recipients.
However, viewed from a different perspective, row (5) of the table shows that it is possible to require work and raise income (and more substantially reduce poverty) at the same time. The key is to combine the work requirement with a strong financial incentive, so that earnings rise more rapidly than benefits fall. The price for raising incomes is higher welfare use, which illustrates a central trade-off facing efforts to reform welfare.
Turning to the other outcomes in Table S.2, the evidence base is deep in indicating that mandated work-related activities reduce food stamp use (cell D3), while more limited evidence suggests food security declines as well (cell J3). We also place less confidence in the negative impact on Medicaid use (cell E3) but somewhat more confidence on the corresponding decrease in children’s health insurance coverage (cell K3). This policy has no effect on marriage or fertility (cells F3 and G3), a conclusion that is based on five years of follow-up data for seven programs and two years of follow-up data for five other programs (hence the dark shading). A somewhat less substantial evidence base provides a very mixed picture of the impact of these programs on child well-being for all three of the age groups shown in Table S.2 (the intersection of row (3) with columns (N) to (V)). The only favorable assessment is for health problems of grade schoolers (cell S3), while the one clear unfavorable impact is for school achievement problems of adolescents (U3).
What Does the Synthesis Tell Us About Sanctions?
Sanctions are another policy reform about which little is known. Many states have enacted sanctions substantially more stringent than those under AFDC. Consequently, many families have lost their aid, or at least part of their aid, because of sanctions. No experiments have been conducted to isolate the effects of sanctions, however; None of the experiments we consider involve any experimental variation in sanction policy, except in conjunction with other policy reforms. Some econometric studies of the caseload indicate that stricter sanctions have greater effects on welfare use, but evidence showing that substantial declines in welfare use preceded the imposition of such sanctions by several years clouds the interpretation of those findings (cell A4). Except for one econometric study of child maltreatment (cell M13), there are no other studies on the effects of sanctions on employment, earnings, income, or other measures of family well-being.
What Does the Synthesis Tell Us About Time Limits?
Although time limits have been analyzed in several econometric studies, the few experiments involving time limits also incorporated other major reforms, making it difficult to isolate the effects of time limits alone. Moreover, the econometric studies only speak to the impact of time limits before recipients begin to exhaust their benefits (row (7)). Nonexperimental estimates from two random assignment studies, along with one econometric study of employment, provide some insights into how behavior changes once recipients begin reaching the time limit. Such evidence supports only a few rather limited statements about time limits.
First, most of the econometric studies suggest that time limits reduce welfare use during the pre—time limit period (cell A7), indicating that recipients change their behavior even before their benefits are exhausted. Evidence from one set of econometric studies is consistent with the notion that recipients leave welfare before reaching the time limit in order to "bank" their months of eligibility for future use.
Second, only two studies suggest that time limits also increase employment during the pre—time limit period (cell B7), so we place less confidence on this cell entry. There is insufficient evidence or no evidence available for assigning the direction of impact of time limits before recipients reach the limit for any of the other outcomes shown in Table S.2, including child well-being.
Third, the knowledge base regarding the post—time limit effects of time limits is even shallower. Two studies show that welfare use falls sharply once recipients begin to exhaust their benefits. Effects on employment are mixed, but none of the evidence suggests that it changes much, either up or down, once recipients start reaching the limit. Clearly, the post—time limit consequences of time limits could increase substantially once a higher proportion of the caseload reaches the limit.
What Does the Synthesis Tell Us About Family Caps and Parental Responsibility?
The table documents that we know relatively little about how family caps and parental responsibility requirements affect key outcomes. Limited evidence points to a mixed impact of family caps on fertility (cell G9). An equally shallow evidence base also produces mixed evidence with respect to the impact of family caps on welfare use (cell A9). Parental responsibility requirements, specifically those related to well-baby and well-child services (e.g., vaccinations), have been assessed in terms of their direct impact on the behaviors they seek to change, with some evidence of favorable effects in terms of child health for young children (cell M18). However, how this policy affects other outcomes is unknown.
What Does the Synthesis Tell Us About Welfare Reform as a Bundle?
Looking beyond specific policy reforms, there is a modest base of evidence to assess the impact of welfare reform as a bundle–either as implemented under state waivers or TANF–on some key outcomes. The available evidence comes from econometric studies that evaluate the impact of reform as a whole under state waivers or TANF and also from a handful of random assignment studies that implemented reform bundles that resemble state TANF plans. These experimental evaluations do not, however, necessarily represent the range of policy bundles implemented across the states, especially at the less generous end of the spectrum (i.e., lower benefit levels, weaker financial work incentives, stricter work requirements and sanctions, and shorter time limits).
With this caveat in mind, welfare reform as a bundle appears to produce impacts similar to those seen for mandatory work-related activities with weak financial work incentives (compare rows (6) and (11)): a decline in welfare use and use of food stamps and an increase in employment, earnings, and income. This is plausible given that most states implemented weaker financial work incentives combined with mandatory work-related activities, and given that what is known about the behavioral impacts of time limits suggests that they operate in the same direction as financial work incentives and work mandates (compare cells A6 and A7, and B6 and B7).
However, this comparison is not available for all outcomes, and the evidence base for the impact of welfare reform as a bundle on the other outcomes in Table S.2 is shallower. The impact on Medicaid use, marriage, and fertility is mixed (cells E11, F11, and G11), while poverty appears to decrease (cell I11). There is too little evidence to assess the impact of reform as a bundle on other measures of well-being in columns (J) to (L). In the case of the child well-being outcomes in columns (N) to (V), the limited available evidence appears to show a mixed impact on behavior problems of young children and adolescents, and an increase in school achievement problems for adolescents. There is some indication of reduced health problems for grade schoolers. However, the cells that are signed in these columns are based exclusively on results from two random assignment studies. The bundle of reforms implemented in these two states is not very representative of the reforms implemented in other states in terms of the length of the time limit (two years or less in both cases) or the generosity of the financial work incentives (very generous in one case). Thus, the impacts in row (11) for these columns should be interpreted cautiously.
It is also important to note that, regardless of the depth of the knowledge base, the entries in row (11) represent the effects of reform as a bundle during the pre—time limit period. Post—time limit evidence is very limited, and most studies summarized in this row cover time periods prior to when recipients could have exhausted their benefits. Once recipients reach the limit in substantial numbers, these effects could change.
What Does the Synthesis Tell Us About Welfare Reform Effects on Subgroups?
The effects of reform do not appear to differ widely across subgroups. Many observers would view this as good news, since there was widespread concern when PRWORA was enacted that only relatively advantaged recipients would respond, leaving the most disadvantaged behind. The subgroup-specific analyses provide no compelling evidence of this. In some cases, subgroup-specific impacts are similar for persons of different levels of disadvantage. In other cases, different measures of disadvantage generate different patterns, some appearing to favor the relatively advantaged and some appearing to favor the relatively disadvantaged. In many cases, subgroup-specific estimates are insignificant, in part because subgroup-specific sample sizes are too small to generate precise results even when the program may have had a substantial effect.
THE STATE OF OUR KNOWLEDGE BASE
Another way to use Table S.2 is to look across the whole table and assess the state of the knowledge base (the types of shading or lack of shading in the various cells). The table reveals that the knowledge base is strongest for understanding the impact of various welfare reform policies on welfare use, employment, earnings, and income. The base is weakest for assessing the impact of policies on broader measures of well-being, especially child outcomes, most notably those for pre-school-age children. Among the policies, a solid base of research exists to evaluate the impacts of mandatory work-related activities on most outcomes, and it is nearly as strong for financial work incentives, either alone or when tied to hours worked or in combination with mandatory work-related activities. As we have already discussed, several reform policies have received less attention, most notably sanctions, family caps, and parental responsibility requirements. Overall, just under half the cells in our matrix (120 out of 242 cells) are empty, indicating no research base exists to assess the policy-outcome pair. Another 36 cells (those with an asterisk) are nearly empty.
Some of the gaps in the knowledge base are particularly relevant for policy. For example, there have been relatively few causal studies of how welfare reform has affected poor families’ participation in the Medicaid program, as shown in column (E), or the health care coverage of children more generally, as shown in column (K). This omission is particularly important in light of the initial decreases in Medicaid enrollment that occurred following the implementation of TANF–despite 15 years of policy initiatives designed to increase the coverage of poor children. As seen in columns (F) and (G), less is known about the impact of individual welfare reform policies and reform as a whole on marriage and fertility despite continued interest among many policymakers in policies to promote the formation and maintenance of two-parent families and to reduce out-of-wedlock childbearing. With the increased emphasis on work for mothers of children as young as age one or even younger, it is unfortunate that so little is known about welfare reform and child development prior to school entry, as shown in columns (N) to (P). This is an issue that is particularly relevant for policies aimed at improving early care and education.
For the policy-outcome combinations where we have a more substantial knowledge base, a nearly universal limitation of our conclusions is that they apply mostly to the short run. Most of the studies present evidence from follow-up periods of roughly two years, although the 11 programs in the National Evaluation of Welfare-to-Work Strategies (NEWWS) and several others provide results based on four or five years of follow-up data. The limited available evidence suggests that some of the effects vary with time since the policy was implemented.
The short-run nature of the evidence limits our understanding of whether reform has accomplished its goals of reducing unwed childbearing, encouraging marriage, and maintaining two-parent families. Marriage and fertility involve substantially more inertia than other aspects of behavior. As a result, we would expect the effects of welfare reform on such outcomes to become apparent only over a longer horizon. With mostly a short-run follow-up period to draw on, it should come as little surprise that most of the evidence from high-quality studies is mixed and statistically insignificant.
The short-run nature of the data also poses a problem for assessing how welfare reform affects the well-being of children. Although some aspects of a child’s well-being, such as behavior problems, may respond quickly in reaction to changes in his or her parent’s behavior, other aspects, such as cognitive skills, are likely to take much longer to change. Furthermore, even effects apparent in the short-term may change as children are exposed to cumulatively lower levels of welfare use and higher levels of employment on the part of their parents. In the short run, depending on the reform policy or policies, there is evidence of both favorable and unfavorable impacts on various aspects of children’s development. In the case of adolescents, there is more consistent evidence of unfavorable behavioral and school achievement impacts associated with the policies that have been evaluated, in some cases up to five years after reform. Whether these same patterns will continue in the longer run–or whether they will be attenuated or exacerbated–remains to be determined.
A more general omission is any understanding of how reform has affected families’ decisions to enter welfare. Random assignment experiments are a powerful research design for revealing how policy reforms affect those entering or on welfare at a point in time. However, they provide no information at all on how families decide to join the rolls. Econometric studies of welfare use reflect the effects of entry decisions, but they do not distinguish them specifically. To date, there have been only a few econometric studies that focus specifically on welfare entry. This omission is significant because entry appears to be important. Theoretical considerations lead us to expect that most policy reforms are likely to affect both entry and exit. Recent empirical work indicates that as much as one-half of the recent decline in the caseload is attributable to declining rates of entry.
DIRECTIONS FOR FUTURE RESEARCH
Despite the gaps in our knowledge base, we draw a broader lesson for future research: Our knowledge base in 2002 is stronger because of research programs put in place in the late 1980s and early 1990s under the strong guidance of USDHHS. That increase in knowledge occurred only as a result of major expenditures on program development and evaluation. Likewise, the inclusion of research funding in the PRWORA legislation supported a continuation of the research and evaluation studies that were initiated prior to federal reform. Consequently, the available knowledge base associated with the welfare reforms implemented in the last decade is superior in many respects to that available for many other areas of social policy.
To add to that knowledge base, it is desirable to learn about current policies that are poorly understood and about reforms that may be proposed in the future. Since the research cycle is at least as long as the policy cycle, we need to continue to put research efforts in place now for what we will need to know when the nation next considers major welfare reform.
Several specific areas deserve priority. To begin, more long-run information on the effects of current policies is crucial. Current long-run studies should be continued and, where possible, extended. Long-term evaluations should include such outcomes as family structure and child well-being, where the impacts may become apparent only over time or where they may vary with the stage of child development. Further research is also needed to understand the effectiveness of alternative strategies for promoting the transition from welfare to work for subgroups of the welfare population, such as for recipients with substance abuse problems and those who experience domestic violence.
Other policies that are less well understood need further evaluation. Time limits represent an important example. The number of families exhausting their benefits may grow sharply in the near future as recipients reach the federal five-year time limit. Studies to assess how families respond are critical. Sanctions are among the most poorly understood of all of the policy reforms. This is an area where both econometric and experimental work would be useful. Econometric analyses that incorporate information on the likelihood of sanctioning and the monetary value of sanctions would provide a more complete understanding of this policy than the studies that are currently available. Experimentation could also help reveal how different levels of sanctions affect a broad range of outcomes. In either case, future research should continue on the path of expanding the range of outcomes examined, in addition to welfare use, employment, and earnings, which have been the focus of most studies to date.
Entry effects also need to be better understood, both to fully grasp how reform has affected welfare use and labor market behavior and to understand how it affects fertility and the utilization of important in-kind services. This is an area where experimentation has less to offer. What is needed are high-quality econometric studies that focus directly on entry decisions.
Initiatives sponsored by USDHHS and other agencies in several of these areas will add to the current knowledge base. For example, follow-up studies continue for a number of the experimental evaluations we examined in our synthesis, and evaluations are under way in a number of other states that implemented other bundles of reforms. Reports are expected soon with longer-term results for several of the experimental evaluations which will add to the small base of results currently available with follow-up periods as long as five years after randomization. Other studies are under way to understand issues regarding accessing Medicaid and the Food Stamp Program, to evaluate the effectiveness of programs serving particularly disadvantaged segments of the welfare population, and to evaluate alternative approaches to promoting job retention and advancement among TANF recipients.
As in the past, advancing such an ambitious research agenda will require substantial federal participation. Many of the experiments reviewed here were conducted to satisfy the requirement that waiver-era reforms be evaluated and because the federal government paid for a portion of the costs. Under PRWORA, states are no longer obligated to rigorously evaluate their reforms. If we are to increase our knowledge base between now and the next time the nation considers major welfare reform, federal funds need to be invested to continue the evaluation of state investments under TANF. Even given TANF’s devolution of welfare policy to the states, a strong federal role in research and evaluation remains appropriate. As this study demonstrates, knowledge gained in one state may be broadly applicable in others. Because of these knowledge spillovers, the states cannot be expected to finance and carry out the needed amount of evaluation research without federal assistance.
1We also draw on the so-called "leaver" studies that examine post-exit outcomes under welfare waivers and PRWORA for former welfare recipients (e.g., USDHHS, 2001a). While these studies provide relevant context and are essential for monitoring the status of families that discontinue receiving aid, they do not purport to identify the causal impact of welfare reform on outcomes.(back)
2The Vermont Welfare Restructuring Project (WRP) is an example of a program with a weaker financial work incentive which can be compared with a program like the Minnesota Family Investment Program (MFIP) which we classify as having a strong financial work incentive. (See Table 3.5 for details regarding the features of the financial work incentives for these programs.)(back)
3In Chapters 4 to 10 of the report, we organize our synthesis around outcomes rather than policies, with chapters devoted to the various outcomes listed in the columns of Table S.2. We return to a discussion of the impact of specific reform policies and reform as a bundle in Chapter 11.(back)
| Table of Contents | Previous | Next |