Table of Contents | Previous | Next |
Chapter Ten
CHILD OUTCOMES
10.1. BACKGROUND
As part of the debate that preceded the passage of PRWORA, there was considerable discussion about the potential for both negative and positive impacts of the new TANF program on child well-being. Some were concerned that increased work effort by welfare-reliant mothers would be harmful to their children. A related concern was that the loss of welfare income might further increase child poverty, again with negative consequences for children. Others suggested that the transition from dependency to self-sufficiency would increase income and provide a positive role model for disadvantaged children and youth. Promoting marriage and family stability was also viewed as beneficial for children.
There are a number of reasons to expect welfare reform to affect child well-being. First, some welfare reform policies are directly aimed at changing parental behavior or investments in their children through features such as parental responsibility requirements regarding school attendance or immunizations and well-child care, and through requirements for parenting classes. Second, welfare reform policies may change other behaviors that have indirect effects on child well-being. For example, as we have seen in earlier chapters, work effort by mothers with children may change as a result of welfare reform policies. Research suggests that the relationship between maternal employment and child health and development depends on the nature of the mother’s job, the change in family resources, the quality of child care and activities for older children, and the mother’s psychological well-being (Morris et al., 2001). Family income may also be affected by welfare reform through changes in welfare payments received, earnings, and other transfers. Again, there is a body of research that indicates that family income can affect child development, showing a link between childhood poverty and detrimental outcomes for children. (See, for example, the studies in Duncan and Brooks-Gunn, 1997, and the reviews provided by Haveman and Wolfe, 1995; Mayer, 1997.) Other outcomes relevant for children’s development that might be affected by welfare reform include maternal schooling, child care utilization and quality, access to health insurance, and living arrangements. Indeed, we have seen evidence in prior chapters that welfare reform as a whole, and specific policies and programs embedded in the reforms in particular, has almost certainly affected work effort, welfare receipt, and family income.
As investigations of the linkages between child outcomes and welfare reform have multiplied, researchers have developed a model of the pathways through which changes in welfare policy might affect child well-being (see Duncan and Chase-Lansdale, 2001). That model generally posits that welfare reform will have immediate or direct effects on parental work effort, welfare receipt, family income, child care, family structure, and educational attainment. These outcomes, in turn, affect the amount and composition of the resources–financial and otherwise–available for raising children. In addition to resources, other intervening or intermediate behaviors and outcomes can change as a result of the direct impacts. Examples include parent psychological well-being (e.g., self-esteem, sense of self-efficacy, stress, depression, substance abuse), parent-child interaction (i.e., the quantity and quality of time available for positive interaction and supervision), child socialization (e.g., for older children, messages about work, responsibility, and self-sufficiency), and access to services (e.g., health care).
These direct and intermediate impacts are then expected to affect child health and development in a number of domains, including cognitive development, behavioral and emotional adjustment, school achievement and attainment, antisocial and delinquent behavior, child safety, and physical and mental health. Within this framework, welfare reform might be expected to have both negative and positive effects on children’s outcomes, and some outcomes might remain unchanged because of opposing forces. This framework also suggests that child impacts might vary with the age or gender of the child, as well as with other family background characteristics. Finally, some aspects of child development may be more responsive within a short period to the effects of welfare reform, while other indicators of child development may take time for the impacts to cumulate. For example, child behavior problems at both younger and older ages may manifest themselves within a short time frame, while it may take longer for effects on child health to become apparent.
Many of the child outcomes of interest are not measured routinely as part of large nationally representative surveys (e.g., the CPS). In addition, more specialized smaller-scale surveys may not have sufficient sample sizes to implement the DoD methodology reviewed in Chapter 3. Likewise, there are few administrative databases that track relevant outcomes in a consistent manner over both geographic space and time. Consequently, in contrast to outcomes like welfare caseloads, employment and earnings, and income, there are considerably fewer econometric studies that employ the DoD methodology to examine child well-being; in fact, we are aware of just one study that analyzes direct measures of child well-being.88
Instead, much of the research on child outcomes and welfare reform is conducted in the context of experimental evaluations. A child outcome component has been included in a number of the welfare experiments, with most studies relying primarily on data collected from parents (and sometimes teachers or the children themselves). A few experimental studies also use administrative data for outcomes such as child maltreatment and foster care. However, across the experimental studies, data on child outcomes are not universally available. Of the studies listed in Table 3.5, the programs in Arizona (EMPOWER), Arkansas (AWWDP), Indiana (IMPACT), Iowa (FIP), New Jersey (FDP), and Virginia (VIP/VIEW) do not report results for child outcomes in the domains we list above.89 The other evaluations report results for at least one child outcome for at least one treatment-control contrast.90
Since our synthesis of the impact of welfare reform on child well-being will, of necessity, draw almost exclusively on random assignment studies–and then only on a subset of the studies covered in this report–the caveats discussed in Chapter 9 for other measures of well-being are equally relevant here. As with other measures of well-being, the sample sizes available for analysis are often smaller than those available for the full evaluation. Studies focus on various child outcome domains, and the measures used in a given domain are not always comparable across the studies that cover that domain.91 Again, this affects our ability to draw more general inferences from the collection of studies. The limitations of experimental studies–the inability to capture program entry effects, questions about generalizing from a local or state demonstrations to national reform, and problems with maintaining ideal experimental conditions–will also affect the broader conclusions we draw in this chapter.
With these concerns in mind, the remainder of this chapter will first discuss the findings from the experimental studies that focus on child well-being. Our understanding of the differences by subgroups gleaned from these studies are discussed in Appendix A and are also summarized in this second section. We then turn to a discussion of the one relevant econometric study. The results from the experimental and econometric studies are synthesized in the fourth section. A concluding section ends the chapter.
10.2. RANDOM ASSIGNMENT STUDIES OF THE EFFECTS OF WELFARE REFORM ON CHILD WELL-BEING
For the studies that do include child outcome measures, there is a wealth of information, which presents a number of challenges. Unlike some of the other chapters in this report, where there is considerable uniformity in the outcome measure across studies, child well-being can be conceptualized in many ways, with a myriad of indicators for any given domain, whether it be child behavioral problems, academic success, physical and mental health, or some other area of functioning. Very few studies measure the exact same sets of indicators, or even the same indicators within the same domains.
In some cases, the measures are straightforward indicators of child outcomes. Good examples include whether a grade has been repeated since random assignment or whether a child has made an emergency room visit since random assignment. In other cases, the indicators are standard scales or test batteries with well-understood psychometric properties (e.g., reliability and validity).92 For example, the Behavioral Problems Index (BPI) is a frequently used measure of child problem behavior in both small- and large-scale studies, and it appears in the child outcome impact analyses for several of the demonstration studies. However, the BPI is not the only measure of problem behavior used in these studies, so differences across studies in child behavior measures may arise from the scales themselves and from the dimensions of behavior they measure rather than from true differences in behavior. As another example, the Peabody Picture Vocabulary Test-Revised (PPVT-R), a standard measure of receptive vocabulary, is used in just one of the demonstration studies we review. Other studies that measure language or reading ability do so with different measures. The reliance on the same well-validated child outcome measures in many demonstrations is an advantage for making cross-study comparisons. However, cross-study comparisons are made more difficult when studies do not focus on the same domains, or when they do not use a similar set of indicators within a domain or the same metric or scale for a given indicator.
Another feature of this literature is that some outcome measures represent favorable outcomes, so a positive numerical impact is desirable; in other cases, the metric represents an unfavorable outcome, so a negative impact is the goal. Our analysis is further complicated by the expectation, as noted in the introductory section, that the impact of welfare reform may vary with the age of the child. Hence, we are interested in differential impacts for children by age, with age groups typically defined as preschool, primary school age, and adolescents.
In light of these challenges, we have organized the tabular presentation of the results from the experimental literature in a format different from the one we used in earlier chapters. First, for any given study, we have grouped the measures of child outcomes into four broad categories: behavior, school performance, health, and other.93 The first captures both positive and negative aspects of behavior, ranging from such measures as the BPI or an index of positive social behaviors for younger children, to being suspended or expelled from school, to being involved in criminal or delinquent activity (for older children). The second outcome category includes various measures of school performance and achievement, including subject-specific test scores (e.g., reading and math), parental reports of school performance, the extent of grade repetition, and use of special education. General measures of health status are included in the third category, along with other indicators of health and safety, such as reports of child abuse and neglect. A fourth residual category captures other outcomes such as foster care placements and participation in clubs or organizations.
Many of these measures relate to the child’s current status at the time of data collection (e.g., health status), while others capture outcomes over a child’s lifetime (e.g., ever repeated a grade in school). Others ask about behavior or outcomes since random assignment. We have tried to clearly indicate whether a measure is cumulative ("ever") or measured since random assignment ("since RA"). Those not explicitly designated are assumed to relate to current status.
Given the multiple measures that are often not comparable across studies, we do not record the numeric level of the outcome for the control group or the impact estimate (treatment-control difference), as we have done in previous tables. Instead, we record impact estimates in a favorable direction (whether numerically positive or negative) as "F," while those in an unfavorable direction are recorded as "U." Impact estimates that are zero are recorded as "0." The statistical significance of each impact is indicated along with the effect size for the impact estimate when it is available.94 The directional indicator and the effect size are in bold type for those impact estimates that are statistically significant.
Finally, where possible, we record outcomes first for all children and then stratified by age group. Since studies often use different age cutoffs in their age strata, we separately record results for the youngest age group (typically preschool), middle age group (typically primary grades), and oldest age group (typically preteens and teens in secondary grades and above) and indicate for each study what age cutoffs are used. The ages recorded are the ages at follow-up, so they can be compared across studies with different follow-up intervals. Given the outcome domains considered and the metrics available, infants and toddlers are often excluded from the analyses.
We now turn to a summary of the results for experimental evaluations of programs grouped by their policies. The results are recorded in Table 10.1 using the approach we have just outlined.
10.2.1. Programs That Focus on Financial Work Incentives
Of the programs that primarily evaluated financial work incentives (Panel A of Table10.1), both MFIP-IO and WRP-IO evaluated child outcomes. The MFIP study focuses on a cohort in a more narrow age range (5 to 12), while WRP reports results for all children, and for children age 10 and above. The follow-up periods were between three and three and one-half years.
Vermont’s WRP evaluation included a follow-up telephone and in-person survey with both recipients and applicants in WRP and WRP-IO (Bloom, Hendra, and Michalopoulos, 2000). Compared with MFIP, WRP examined a somewhat more limited set of child indicators. In general, the results show no consistent effect of the WRP-IO program for the sample of nearly 1,200 children studied. Children in the treatment group had a significantly higher likelihood of missing a day or more of school in the last month, but a significantly higher rate of participation in clubs and organizations. For those age 10 and above, the treatment group reported a statistically significant higher rate of ever being in trouble with the police (26.8 percent versus 17.2 percent for controls), but there were no significant differences in school dropout behavior or behavior problems (although the impact estimates were in the favorable direction).
The MFIP child analysis (both the Incentives Only and full program) focuses on a random subset of families in the evaluation sample who entered the study in the first six months (April to October 1994) and who had at least one child age 5 to 12 at the time of the survey, three years after random assignment (Gennetian and Miller, 2000). Much of the data collected refers to a "focal" child rather than to all children in the family.95 With this narrower age group, MFIP-IO does not provide results for infants and adolescents, unlike some of the other evaluations. As with other MFIP analyses, results are available for long-term recipients and applicants, with about 600 and 400 children in the combined treatment and control groups, respectively.
In general, MFIP-IO produced largely favorable effects for children of long-term recipients. Statistically significant beneficial impacts were concentrated in the behavioral and schooling domains. The impact for one health indicator, emergency room visits for any child in the family, was unfavorable and statistically significant for recipients. For applicants, most effects were not significant, but a few in the school performance domain were unfavorable. Other results (not shown) indicate that MFIP-IO improved the physical home environment and reduced harsh parenting for recipient children while the reverse was true for applicants (Gennetian and Miller, 2000). Recall that in Chapter 8, MFIP-IO had stronger effects on income for long-term recipients than for recent applicants. Also, applicant children in the control group performed better than recipient children in the control group, indicating there was less room for the program to improve child outcomes among the more advantaged applicant children.
10.2.2. Programs That Focus on Financial Work Incentives Tied to Hours of Work
Child impacts are available for two of the programs–New Hope and SSP– that are classified in Panel B of Table 10.1 as evaluating financial work incentives tied to hours of work. An analysis of child outcomes is expected in 2002 for the SSP Applicant study.
The New Hope Child and Family Study administered questionnaires to parents and teachers for the sample of families at random assignment with at least one child age 1—10 (Bos et al., 1999). At the two-year assessment, a focal child, then age 3—12, was the subject of parental reports on child behavior and school progress. Teachers also provided ratings on indicators in these domains for children in kindergarten and above. Information was also collected directly from children starting at age 6. In addition to presenting results for all children, results are often stratified into three age groups: 3—5, 6—8, and 9—12. The sample sizes for many of the impact results shown in Table 10.1 are among the smallest of the studies we consider, ranging from under 250 combined treatment and control children aged 3—5 and 6—8, to over 600 children of all ages with parental reports of school behavior. In addition to these results, Bos and Varga (2001), in a separate analysis, report results for adolescents age 12 to 18 based on data collected for all children in the New Hope sample.
Even though the New Hope parental outcomes, such as employment, earnings, and income, shown in previous chapters differed by employment status at the time of random assignment, there were few differences in child outcomes across the two groups. Pooled results are summarized in Table 10.1. With the exception of the adolescent results, only two of the outcomes recorded in the table show a significant favorable effect, while none shows a significant unfavorable effect. The teacher’s report of school performance and positive social behavior both favor the treatment group, with an effect size that equals about 0.25 of a standard deviation in each case. Parents’ reports of total positive behavior are also more favorable for the treatment group. None of the outcomes measures shown separately for the two younger age groups are statistically significant, but small sample sizes mean that small differences are unlikely to be detected. The impacts for New Hope adolescents are mixed, with both favorable and unfavorable statistically significant impacts across the behavior and school achievement domains. On the positive side, New Hope adolescents are less likely to be in special education and more likely to be in a gifted or talented program. On the negative side, they are more likely to have repeated a grade, have a higher number of contacts by the school for behavior or academic problems, and are reported by their parents to be performing more poorly in school.
The SSP evaluation assessed child outcomes three years post-randomization through a survey of participants and their children (Morris and Michalopoulos, 2000).96 Outcome measures covered the child’s social and antisocial behavior, school progress and achievement, and health and safety. In addition, it is among the few studies to administer achievement tests to children–the PPVT-R for children age 4 to 7 and a math skills test for children age 7 to 15–to directly assess academic performance. SSP reports results stratified into three age groups (3 to 5, 6 to 11, and 12 to 18), with about 1,000 children each in the youngest and oldest age strata, and about 450 in the middle strata.
The impacts for SSP are striking in how consistent the results are within the age strata. For the youngest children, none of the outcomes were significantly affected by the program. Of the significant impacts for children age 6—11, all were favorable and centered on measures of school achievement (math score and maternal report of achievement in specific subjects) and health (general health and presence of long-term health problems).97 The effect sizes were generally small, however. In contrast, for the oldest age group, all the statistically significant effects were in the unfavorable direction, with detrimental effects concentrated in the behavior domain. For instance, children age 12—18 at follow-up had higher rates of school behavior problems, minor delinquent activity (15—18-year-olds only), and use of tobacco, alcohol, and drugs, with effect sizes that range from about 0.10 to 0.20 standard deviations. At the same time, there were no significant treatment-control differences in many of the other indicators measured for this age group, including math and reading test scores. It is worth noting that adult outcomes for SSP families with children age 12—18 at follow-up were as favorable as they were for families with children in the youngest and middle cohorts.
10.2.3. Programs That Focus on Mandatory Work-Related Activities
Panel C of Table 10.1 records the child outcomes measured as part of L.A. Jobs-First GAIN (Freedman et al., 2000b), as well as the 11 NEWWS programs (Hamilton, Freedman, and McGroder, 2000; Hamilton et al., 2001).98 These programs, which focus on mandatory work-related activities, collected information at the two-year or five-year follow-up on the children of recipients and applicants, either single parents (Los Angeles) or all parents (NEWWS). In 6 NEWWS programs (LFA and HCD programs in Atlanta, Grand Rapids, and Riverside), the Child Outcomes Study (COS) collected additional measures for a focal child age 3 to 5 at randomization (ages 8 to 10 as of the five-year follow-up). We discuss those findings in the context of the measures reported in Table 10.1 for children in this age range, but we do not report the additional COS measures in the table.
L.A. Jobs-First GAIN collected child-level information through a client survey two years past baseline for close to 1,600 children (Freedman et al., 2000b). Information collected covered such areas as academic achievement and school performance, behavioral and emotional adjustment, and safety. Results are recorded for all children, as well as for children classified by age at the time of follow-up: 5 to 7, 8 to 11, and 12 to 20, with over 400 children in each group. For the pooled sample of children, there was only one statistically significant effect (and that only at the 10 percent level): a more favorable outcome on school expulsions/suspensions (9.3 percent treatment versus 12.9 percent for controls).
Children in the youngest age group (5 to 7 at the time of the follow-up) were significantly more likely to repeat a grade (6.2 percent versus 0.4 percent for controls) and to have a special physical, emotional, or mental condition that made their parents’ work difficult. This effect may be the result of the higher work effort among the treatment group parents. For children 8 to 11 at follow-up, the treatment group experienced a significantly higher rate of attending a special class for physical, emotional, or mental condition (15.5 percent versus 9.8 percent for the control group). The other impact estimates were mixed. For the oldest age group, up to 18 at random assignment and 20 at the follow-up, there were no statistically significant treatment-control differences on any of the indicators. In some cases, the impact estimates were favorable and in others not favorable.
Outcomes in the NEWWS evaluation–collected for 4 programs only for the 2-year follow-up and for the other 7 programs as of the five-year follow-up–focused on maternal reports of behavioral adjustments, school progress, and health and safety for all children age 18 or under at random assignment, with samples that range from 500 to 1,200 as of the final follow-up. The COS, also collected through a survey at the two-year and five-year follow-ups, focuses in more detail on academic functioning, social skills and behavior, and health and safety for young children age 3 to 5 at random assignment based on reports from mothers, teachers, and the children. Sample sizes range between 250 and 550 depending on the measure and the site. For the 4 programs with only two-year follow-up impact estimates, since many of the indicators are only relevant for school-age children, the results recorded in Table10.1 for all children are for the analyses conducted on the sample of families where all children were age 6 and above.
Overall, the NEWWS child outcome results show no clear pattern of beneficial or harmful effects for children up to age 14 at follow-up across domains within the same program or across the 11 programs. Both favorable and unfavorable effects are found across all the domains, sometimes for the same program. Typically, one or just two specific outcomes out of the seven reported outcomes in the five-year follow-up or five reported outcomes in the two-year follow-up have statistically significant impacts. Results for infants and toddlers (those age 1 to 2 at random assignment and 6 to 7 at follow-up), available for only two sites, show largely favorable effects for the two Grand Rapids programs and more unfavorable effects for the Portland program. The results of the COS (not shown) which focuses on pre-school-age children at random assignment also shows no clear pattern of favorable or unfavorable effects, and the impacts that were found were not related to the program approach (Hamilton et al., 2001). For the adolescents at random assignment (those age 15 to 23 as of the five-year follow-up), however, there is higher prevalence of statistically significant unfavorable effects, especially for schooling outcomes in the Riverside HCD program, the program with the largest income declines (see Figure 8.2).
10.2.4. Programs That Focus on Financial Work Incentives and Mandatory Work-Related Activities
Child impacts are available for three of the programs–WRP, MFIP, TSMF–that are classified in Panel D of Table 10.1 as combining financial work incentives and mandatory work-related activities.99 The results for the full WRP evaluation show even fewer significant impacts on children in total and for those age 10 and above than the Incentives Only component of the program (Bloom, Hendra, and Michalopoulos, 2000). The only significant impact is a higher rate of participation for all children in clubs and organizations (34.2 versus 26.5 percent). There is no clear pattern with respect to the relative contribution of work requirements on top of the incentive program. The only statistically significant difference is in the measure of school absence, which has a more favorable outcome in the combined program.
The results for the combined MFIP closely mirror those seen earlier in Panel A for MFIP-IO for children age 5—12 at follow-up. In particular, children aged 5—12 of long-term recipients experienced several favorable impacts concentrated in the behavior and school domains. Many of the same indicators significant in Panel A are likewise significant in Panel D. When selected indicators were considered for recipient children younger than age 9 versus age 9 and above, the latter group (those who were school-age at random assignment) had stronger impacts (not shown). Two school performance measures and the BPI had favorable effects for the older subgroup. There were no significant effects for the youngest children, those who were pre-school-age at the start of the experiment. Compared with the older cohort, adult MFIP participants in this group experienced a larger increase in employment and income.
A comparison of the two MFIP interventions (full MFIP versus MFIP-IO) for long-term recipients indicates that the favorable effects on child outcomes in terms of behavior and school performance can be attributed to the financial work incentives component of the program. The addition of the work requirements had an unfavorable effect on a measure of positive behavior–the opposite of the effect of financial work incentives alone–so that the impact of the full MFIP was close to zero (not shown). At the same time, all other child indicators were unaffected by the addition of the work requirements. This suggests that adding mandatory work-related activities to a program with more generous financial work incentives may not be that harmful to children. This is despite the fact that adding the work requirements further increased full-time employment and lowered welfare benefits.
For recent applicants, the full MFIP generally had no effect on the child indicators measured for those aged 5 to 12. The one exception was a measure of whether the focal child had been suspended or expelled since random assignment. Small sample sizes make it more problematic to separate the effects of work requirements versus financial work incentives for the applicants in the study. Compared with the long-term recipients, children in the recent applicant group began with fewer disadvantages and were more heterogeneous, which may explain some of the differential impact.
MFIP also provides a more limited number of measures, mostly in the schooling domain, for children age 13 and above at the time of follow-up (age 10 and above at random assignment). As seen in Panel D of Table 10.1, the impacts for these adolescents are all insignificant for long-term recipients and mixed in sign, but they are all unfavorable and, with one exception, statistically significant for the adolescents of recent applicants. It is unclear whether these unfavorable impacts might be associated with the work requirements or financial work incentives components of the program, although recent applicants were not subject to the MFIP work requirements for much of the follow-up period.
The evaluation of the Michigan TSMF program is one of the few to rely exclusively on administrative data to assess child outcomes. The evaluation shows no significant differences between treatment and controls in any of the measures considered: substantiated reports of abuse and neglect, placement in foster care, and for older children (age 12 and above), employment and earnings. The labor market outcomes for youth were examined because Michigan’s program allowed a 100 percent disregard of earnings from dependent children. Despite this program feature, it does not appear to have significantly affected work effort on the part of teens.
10.2.5. Programs That Focus on Other Reforms
Of the four studies classified in our residual category and listed in Panel E of Table 10.1, only PPI and PIP include analyses of child outcomes. Since these two demonstrations focused on parental responsibility requirements, specifically with regard to preventative health care (PPI) or immunizations (PIP), the main outcomes of interest pertain to the domain of child health for younger children.
The PPI program in Maryland required families with children age 3—24 months to verify that their children received preventative health care, including immunizations. Data collected through medical records abstraction for nearly 1,800 treatment and control children one and two years after randomization provided information on preventative visits per year and on whether children were up to date for three specific vaccinations (Minkovitz et al., 1999). The results show no significant treatment-control differences at either the first or second year follow-up in whether vaccinations were current, and no significant differences in the number of well-child visits to a primary care provider or whether at least one well-child visit was made a year.
Georgia’s PIP, which required families to demonstrate proof of up-to-date immunization status semiannually (up to 1996) or annually (after 1996), served families with children age 6 or younger. Medical records were examined for about 2,800 treatment and control children in the demonstration four years after randomization to determine age-appropriate rates of immunization annually for five specific vaccinations (Kerpelman, Connell, and Gunn, 2000). In each of the four years post-randomization, children in the treatment group were significantly more likely to be current on at least four, and most often all five, of their vaccinations. For example, four years after randomization, 87.5 percent of treatment children were up to date on their polio vaccination compared with 80.1 percent of the control group. The impacts for this vaccination and two other vaccinations with relatively high rates of immunization even in the control group (specifically DTP, Diphtheria-Tetanus Toxoids-Pertussis, and MMR, Measles-Mumps-Rubella) are considered large given the potential for a "ceiling effect" (i.e., efforts to increase immunization rates often reach a "ceiling" beyond which further increases to 100 percent coverage are difficult to achieve).
10.2.6. Programs That Focus on TANF-Like Bundles of Reforms
Of the six programs classified in Panel F with a focus on TANF-like bundles of reforms, to date only ABC, FTP, and Jobs First report analyses of child outcomes.100 Delaware’s ABC evaluation examined child protective services administrative data for nearly 4,000 children in the treatment and control groups to assess the impact of the program on substantiated reports of child abuse and neglect (in aggregate and for subcomponents of maltreatment) and placements in foster care (Fein and Lee, 2000).101 The results show a statistically significant increase in the incidence of child neglect in years one and three but not in year two.102 For example, in year one, 2.6 percent of the control group had a substantiated report of child neglect compared with 4.1 percent for the treatment group. No significant differences were found for other types of maltreatment (e.g., physical and emotional abuse or sexual abuse) and foster care placements. The combination of benefit decreases and increased work effort in years one and three is suggested by Fein and Lee (2000) as an explanation for this result. In contrast, while benefits fell in the second year, earnings did not increase. This interpretation is consistent with Paxson and Waldfogel (1999) who estimate that increased maternal employment in single-parent families increases child maltreatment. Since income–which is sometimes negatively associated with the incidence of abuse and neglect in Paxson and Waldfogel’s (1999) study–did not increase, the negative employment effect would be expected to dominate.
Florida’s evaluation of FTP covered behavior problems, school outcomes, health status, and, for older children, delinquency and child bearing measured through a client survey four years post-randomization (Bloom et al., 2000). There are few statistically significant impact estimates on a range of child outcome measures, evaluated separately for about 1,100 children age 5—12, and nearly 750 children age 13—17. For the younger age group, FTP led to a statistically significant unfavorable outcome on the positive behavior scale, but the mother’s report of the child’s general health was significant and favorable. There were no statistically significant treatment-control differences for the youngest children in terms of current school achievement as reported by the mother, use of special education, or school suspensions, with impact estimates in both the favorable and unfavorable direction.
For the older children, FTP resulted in an increase in the rate of school suspensions (41 percent for the treatment group versus 33 percent for the control group), equal to 0.17 of a standard deviation. The maternal report of educational success was also unfavorable and marginally significant. No significant differences in other outcomes for the older children such as grade repetition, ever arrested, or having a baby were found in the four-year follow-up; all but one of these insignificant impact estimates was in the unfavorable direction, however. The contrast between the younger and older children in FTP is not as sharp as it was for the age differences observed in SSP.
The Jobs First three-year follow-up survey collected information on school achievement for all children under age 18, collected information on contact with the police and fertility for older children age 13 to 17 (a sample of about 1,000 adolescents), and collected more detailed information on behavior and functioning for about 1,500 "focal" children age 5 to 12 at follow-up (Bloom et al., 2002). (We do not include results based on a survey of the teachers of a subset of the focal children.) For the focal children, the impacts are largely favorable, and in the case of behavioral outcomes also statistically significant (although the effect sizes never exceed 0.1). For adolescents, however, the results are more mixed. Impacts in the schooling and health domains are all unfavorable, although only one impact (current school achievement) reaches statistical significance, perhaps because of smaller sample sizes. At the same time, adolescents of Jobs First participants were less likely to be convicted of a crime.
10.2.7. Subgroup Differences
Given the variation in the subgroups analyzed across random assignment studies (summarized in Appendix A), it is difficult to draw firm inferences about subgroup differences associated with different policies or programs. None of the programs that focus on financial work incentives alone included analyses by subgroups. Likewise, there are too few analyses by subgroups for the programs that focus on work requirements alone to draw solid conclusions about likely differential impacts.
Among the programs that combine financial work incentives and mandatory work-related activities, more differences by subgroup are analyzed, but they do not show a clear pattern. Impacts by child gender sometimes favor boys and at other times favor girls. MFIP appears to generate more favorable impacts for those at greater socioeconomic risk. At the same time, New Hope and SSP do not reveal any differences by characteristics that also capture risk of dependency on welfare.
10.3. ECONOMETRIC STUDIES OF THE EFFECTS OF WELFARE REFORM ON CHILD WELL-BEING
Given the data issues discussed in the introductory section, it should not be surprising that there are so few econometric studies of the impact of welfare reform on child outcomes. Most of the outcomes of interest–such as impacts on cognitive, emotional, and social development; behavior problems; school performance; and child health–are simply not collected for large nationally representative samples over time. Without such data, it is difficult to implement the DoD methodology required to control for unobserved confounding factors.
We are aware of just one relevant econometric study, one that uses administrative data on child maltreatment and the DoD methodology to investigate the impact of welfare reform in general and specific reform policies. Paxson and Waldfogel (2001) use state-level administrative data on child maltreatment and foster care to model the relationship between these outcomes and welfare policies, measured by the existence of a pre-TANF waiver, the existence of a family cap under AFDC or TANF, benefit levels under AFDC or TANF, and then specific policies post-TANF (work requirements, sanctions, and time limits). The outcomes they model by state and year from 1990 to 1998 include the log of reports of child abuse and neglect (in total and disaggregated), substantiated cases of abuse and neglect and the substantiation rate, and the number of children in out-of-home (primarily foster) care. Controls are included for the size, age, and race composition of the child population; the fraction of children in urban areas; the proportion of children whose mother has less than a high school degree; the unemployment rate; and state and year fixed effects. Controls are not included, however, for other state-level child welfare policy variables that were changing over this period.
Table 10.2 reports the estimated model parameters. Results for the six outcomes are provided in each row, each representing one regression model with controls for the six welfare policy variables recorded in each column. Across the models they estimate, there is some evidence to suggest that welfare policy may affect child maltreatment outcomes, but many of the estimated effects are not statistically significant. The statistically significant coefficients suggest that a family cap lowers substantiated cases but raises out-of-home care. Paxson and Waldfogel hypothesize that a family cap reduces abuse by limiting family size, but the other studies in Chapter 7 suggest that family caps do not have a major impact on childbearing. Among the other effects, immediate work requirements under TANF are associated with increased foster care, while a first full-family sanction under TANF is estimated to raise reports of physical abuse by 16 percent and substantiated cases by 22 percent. None of the coefficients on the TANF time limit measure are statistically significant. Likewise, the estimated parameters reported in Table 10.2 show no statistically significant effects of a state waiver for a work requirement, welfare time limit, or work incentives prior to implementing TANF.
| Welfare Policy Variable | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| AFDC/TANF Family Cap | TANF Immediate Work Requirement | TANF First Full Family Sanction | TANF Time Limit |
ln (AFDC/TANF Benefit Level) | Any waiver | ||||||
| Study | Data | Begin | End | Outcome | Dep var. | % effect (t-stat) | % effect (t-stat) | % effect (t-stat) | % effect (t-stat) | Elasticity (t-stat) | % effect (t-stat) |
| Paxson and Waldfogel (2001) | State Child Protective Systems aggregated data | 90 | 98 | Reports of child abuse and neglect | Log | 0.0 (0.00) | 4.9 (0.86) | 9.7 (1.39) | 6.2 (0.71) | -0.37 (0.72) | -6.8 (1.33) |
| Reports of physical abuse | Log | -6.8 (1.12) | -3.9 (0.63) | 15.7 (2.06) | 4.3 (0.45) | -0.20 (0.36) | -4.9 (0.89) | ||||
| Reports of neglect | Log | -15.2 (1.19) | -2.9 (0.35) | 13.9 (1.38) | 4.4 (0.35) | -3.15 (4.29) | -4.3 (0.59) | ||||
| Substantiated cases | Log | -14.0 (2.39) | -9.4 (1.55) | 22.1 (2.98) | 4.2 (0.45) | -0.41 (0.75) | -5.2 (0.97) | ||||
| Substantiation rate | Log | -14.0 (2.04) | -14.3 (2.02) | 12.4 (1.42) | -2.1 (0.19) | -0.04 (0.06) | 1.6 (0.26) | ||||
| Out-of-home care | Log | 15.6 (3.64) | 8.7 (2.03) | 2.1 (0.37) | -7.4 (1.16) | -0.80 (2.19) | 0.6 (0.18) | ||||
| NOTE: All models include state and year fixed effects and controls for the size, and age and race composition of the child population; the fraction of children in urban areas; the proportion of children whose mother has less than a high school degree; and the unemployment rate. |
Taken together, the welfare policy variables are only jointly significant in the model of substantiated cases and out-of-home placements. As with the econometric models reviewed in other chapters, there may be too little variation in the time period–with data extending only to 1998–to separately sort out the effects of the different welfare policy variables on child outcomes. In addition, the absence of controls for other potentially relevant policy variables may bias the estimated welfare policy impacts. For these reasons, we place less weight on the results from this one econometric study.
10.4. EVALUATING THE EFFECTS OF WELFARE REFORM ON CHILD WELL-BEING
The studies reviewed in the prior section paint a complex portrait of the multiple domains and varied indicators of child well-being that can be affected by welfare reform. Our objective in the remainder of this section is to bring these results together to provide a more coherent understanding of the impact on child outcomes of welfare reform as a whole, and the specific policies and programs embedded in TANF.103
10.4.1. Effects of Specific Reforms
Much of our information about the effects of specific reforms comes from the experimental studies, although the one econometric study reviewed in this chapter also looked at policy components. There are some clear patterns across some of the specific reforms, while other specific reforms suggest more uncertain or mixed relationships.
Financial Work Incentives
Strong financial work incentives that lead to greater work effort and higher earnings and income, either alone or when tied to hours worked or in combination with work requirements, are often beneficial for children who are pre-school and elementary-school age at the time they entered the program. However, there are a few examples of unfavorable impacts as well. Both MFIP and MFIP-IO show positive impacts in the behavior and schooling domains for children of the more disadvantaged group of recipients who were between the ages of 2 and 9 at random assignment. A few outcomes were unfavorable for the more advantaged MFIP-IO and MFIP applicant children in the same domains. For longer-term recipients in MFIP, the addition of work requirements on top of a program of generous financial work incentives appears to have little in the way of positive or negative consequences for children, at least those who are preteenagers at random assignment. Coupled with the financial work incentives, MFIP recipient children appear to benefit from the higher income that results from the welfare benefit structure, even with the work-related mandates. The absence of unfavorable outcomes for WRP and TSMF–two programs with weaker financial work incentives–further supports this view, although these two studies considered a more limited set of child indicators.
The patterns are similar when financial work incentives are tied to hours work as in SSP and New Hope. SSP, which includes a younger age cohort, shows no significant impacts on the youngest children, and only favorable significant impacts for the children in the middle age range. The New Hope findings of positive impacts for children overall in school performance and a measure of positive behavior, and no statistically significant unfavorable effects for the other measures (or across age groups), reinforce the view that financial work incentives can be beneficial, or at least not harmful, for pre-school-age and primary-school-age children.104
There is some evidence, but more limited, that programs with financial work incentives alone or those tied to hours worked or work requirements may have unfavorable impacts on adolescents even when they increase family income. WRP-IO and WRP produced one negative impact (significant only for the former program) in the behavioral domain (trouble with the police), although this program had small impacts on adult behavior and family income. Adolescents of the more advantaged MFIP applicants experienced unfavorable impacts in the behavioral and schooling domains that were not evident for the more disadvantaged adolescents of longer-term recipients. (Impacts for adolescents in MFIP-IO are not reported.) SSP and New Hope, both programs with financial work incentives tied to hours worked that produced significant increases in income, also find unfavorable impacts for teens in the behavioral and schooling domains, although New Hope also had a few favorable schooling outcomes.
Thus, the experimental evidence suggests that more work and more income resulting from financial work incentives alone, or in combination with work requirements, are generally neutral or favorable for younger children (preschool and elementary age) but may be detrimental for adolescents, at least for some areas of development. The favorable effects are concentrated in the behavior and school performance domains for the younger children, while the negative impacts for the older youth fall primarily in the behavior domain. These results are consistent with the broader literature that evaluates the relationship between family income and child outcomes across the life course. Duncan and Brooks-Gunn (1997), in their synthesis of recent studies on this topic, conclude that "[f]amily economic conditions in early and middle childhood appear to be far more important for shaping ability and achievement than they do during adolescence (p. 597)." The MFIP applicants, a more advantaged group, provide the one exception where even younger children experienced some unfavorable impacts for a program with incentives alone or combined with work requirements. Since the income gains for these families were smaller, it is not clear what other factors can explain these less favorable outcomes.
Mandatory Work-Related Activities
The L.A. Jobs-First GAIN study and NEWWS results demonstrate small but mixed effects on children of programs that require mandatory work-related activities in general and of programs with an employment versus education focus. Recall that L.A. Jobs-First GAIN led to a significant increase in earnings that was offset by falling cash welfare payments because of the high benefit reduction rate. Thus, pretax income changed little for the participants compared with the controls. Accounting for the EITC and other taxes resulted in a slight gain in income for the Los Angeles participants. Like the L.A. Jobs-First GAIN study, the NEWWS programs generally produced small, if any, income gains and little change in poverty. In general, these studies provide very mixed impacts for pre-school-age and primary-school-age children across all three domains, with both favorable and unfavorable impacts but many that were not statistically significant. Again, there is more consistent evidence of unfavorable impacts for adolescents, especially in the school achievement domain. In the NEWWS demonstrations, there was no clear relationship between program impacts on income and impacts on children, although there is some evidence to suggest worse child outcomes with higher employment and lower income. It would appear from these results that reductions in welfare dependency without significant gains in income result in ambiguous effects on child outcomes, with examples of both favorable and unfavorable impacts. The limited econometric evidence also suggests a weak or inconsistent impact of work requirements.
Time Limits
The evidence with respect to time limits is more limited. The one econometric study suggests no impact of time limits on child maltreatment, but we have placed less weight on this study because of methodological concerns. Of the experimental studies that include time limits, none is designed to estimate the specific impact of time limits separate from the other program features in the bundle of reforms. Moreover, unlike some of the other outcomes considered in earlier chapters, FTP and Jobs First do not provide impacts for child outcomes before and after time limits begin to become binding. Thus the DoD strategy employed in earlier chapters to infer the mechanical effects of time limits is not available to assess the impact on child outcomes. The pattern of impacts for FTP and Jobs First for both school-age children and adolescents is not markedly different from that for MFIP, which includes financial work incentives and work requirements but no time limit. Thus, it is not clear whether, on the margin, the addition of time limits in programs like FTP and Jobs First has favorable or unfavorable impacts on child well-being.
Parental Responsibility Requirements
Of the two studies that focus on parental responsibility requirements related to child health (i.e., preventative care or vaccinations), PPI had no effect on the required behaviors, while PIP had a sizeable and significant favorable impact. This difference may be attributed to the fact that PIP had larger sanctions compared with PPI. In the case of PIP, the sanction equaled a portion of the nonimmunized child’s grant. PPI effectively levied a $10 per month sanction against a family that was out of compliance with the verification requirement.105 Another explanation may be that recipients responded more to the PIP initiative because the intervention was focused only on changing immunization outcomes–with expectations that were easier to understand and comply with–compared with the broader set of requirements under PPI regarding health care (e.g., preventative care more generally and prenatal care) and school attendance (Kerpelman, Connell, and Gunn, 2000).
10.4.2. Effects of Reform as a Bundle
As noted in prior chapters, the econometric studies are potentially the best methodology for estimating the impact of waivers or TANF as a bundle. However, the limited number of studies on child outcomes using this approach makes it more difficult to ascertain the effect of welfare reform as a whole on the multidimensional concept of child well-being. The one econometric study reviewed in Section 10.3 suggests that waivers as a whole had very little impact on child maltreatment and placement in foster care. In the absence of similar econometric analyses of other child outcomes in domains such as behavior and cognition, school progress, other aspects of child health, and so on, it is not possible to conclude whether waivers as a whole had any effect on children.
In other chapters, we have also looked at the experimental evidence to gauge the possible effects of welfare reform as a bundle, especially for the demonstration studies that combine the three key features of most state TANF plans: time limits, financial work incentives, and mandated work-related activities. Panel F in Table 10.1 shows that three of the programs with these features have results for child outcomes. The ABC results suggest a possible unfavorable impact on child maltreatment (specifically neglect). For the school-age children at follow-up, FTP and Jobs First show both favorable and unfavorable impacts in the behavior domain, no effects on the school performance measures, and one favorable effect for health. Likewise, for adolescents at follow-up, there is mixed evidence in the behavior domain, more consistent evidence of unfavorable impacts on school performance, and no significant impact in the health domain (here a measure of teen fertility). These results, while far from conclusive, suggest that welfare reform as a package may affect several domains of child well-being, including antisocial and problem behavior, school achievement, and health, but the specific impacts and their differences by child age are less well understood.
The subgroup results for the programs that evaluate TANF-like bundles of reform are also mixed, with less favorable effects in ABC for families at greater socioeconomic risk and better outcomes for families in FTP at the greatest risk of long-term dependency. There is some nonexperimental evidence that the parents in the lower risk group in Florida were less likely to closely supervise their children, so these children may have been more prone to problem behavior. This group had the largest earnings impacts, especially near the end of the follow-up period.
10.5. CONCLUSIONS
The studies reviewed in this chapter reveal that there is scope for both positive and negative effects on child well-being of various components of welfare reform policies and programs. Positive and negative effects were observed for indicators that capture socioemotional behavior, academic performance, and health. The most favorable effects are associated with financial work incentives, most likely because of the increase in family income that is accompanied by combining work and welfare. But even for these programs, there is some evidence of unfavorable impacts for some subgroups of participants, especially for adolescent children of participants and for younger children of participants who do not experience large income gains. Work requirements do not appear to have either strong favorable or unfavorable impacts on children, although again there is evidence of unfavorable impacts for adolescents, especially in the school performance domain. There is too little evidence regarding the specific impacts of time limits to draw firm conclusions.
There is also relatively little evidence on which to draw solid inferences about the impact of welfare reform as a bundle on child well-being, based either on econometric or experimental data. In the case of the econometric literature, there is just one study and it is limited to one outcome domain. It is also difficult to extrapolate from the three currently available random assignment studies that evaluated the impact on child well-being of TANF-like bundles of reform since the policy combinations evaluated are not representative of the full range and mix of policies implemented by the states under PRWORA. To the extent that there are favorable effects from these studies, they are concentrated in outcomes for children who are school aged at the time of follow-up. The unfavorable impacts, in contrast, are concentrated in outcomes for adolescents, particularly in the area of school performance.
Thus, the impacts of welfare reform appear to differ with the stage of the child’s development, regardless of the policy component or bundle of reforms considered, and for a given age, impacts may be favorable or unfavorable depending on the outcome domain considered. Based on the experimental evaluations that assess child well-being, it appears that there are countervailing forces that both promote and diminish healthy child behavioral, social, cognitive, and physical development. The resulting impacts of welfare reform policies on child outcomes are likely to depend on the strength of the opposing forces and the child’s stage of development and other circumstances. Moreover, it is possible that some consequences for children will not materialize until more time has passed under the new policy regime, with the potential for cumulative favorable and unfavorable impacts. Effects that are small now, whether positive or negative, may become more pronounced as more time passes.
88Haider, Jacknowitz, and Schoeni (2002) use aggregate state-level data on rates of breast-feeding from 1990 to 2000 to estimate the impact of various features of state work requirements under waivers and TANF on breast-feeding rates. They find that the most stringent work requirements that apply to women with infants have significantly reduced the prevalence of breast-feeding six months after birth for all mothers and for women on WIC. Since the incidence of breast-feeding is a more indirect measure of child well-being, we do not include their study in our synthesis.(back)
89In some cases, like Indiana and Iowa, future reports are planned with impact estimates for child outcomes.(back)
90In the case of the multitreatment SSP study, child outcomes are reported for only the primary study.(back)
91Issues associated with measuring child outcomes are discussed more fully in the next section.(back)
92A test is reliable if an individual has similar scores on repeated applications of the test. A valid test is one that measures what it purports to measure.(back)
93These categories overlap to some extent given that some outcomes could be easily classified in more than one of the areas we have defined. For example, school suspensions could be a behavior problem or a school outcome. We have classified it as the former, but we recognize that others might place it in the latter category. For our purposes, the goal was to be consistent rather than rigid in defining these broad outcome categories.(back)
94The effect size is a standardized measure of impact and is defined as the program impact (treatment minus control group difference) divided by the standard deviation of the outcome for treatment and control groups combined. In Table 10.1, we report the absolute value of the effect size for those studies that report it. Since standard deviations are typically not provided, it is not possible to calculate effect sizes when they are not reported by the study authors.(back)
95Some measures are available for all children. Impacts for the full MFIP for adolescents age 13 and above, in addition to those for the focal children age 5 to 12, are discussed in Section 10.2.4 below.(back)
96For some outcomes, reports were made by both parents and children, while others were collected from parents only or children only. When results are available from both parent and child reports, we record the parent result in Table 10.1. Unless otherwise noted, the parent and child impact results were very similar.(back)
97There was no statistically significant difference in the child report of average school subject achievement and in general health status. Both measures were reported by those age 10Ð11 only. For both of these indicators, the favorable impact measured in the parental reports is strongest for children age 6Ð8, suggesting that the difference in parent and child reports results from a lower impact among the older children in this age range.(back)
98Child outcomes were not assessed as part of the Indiana IMPACT evaluation.(back)
99Iowa's report on child outcomes in FIP is expected later in 2002.(back)
100A child outcome study is included as part of the Indiana evaluation with results expected to be released in late 2002.(back)
101The Delaware evaluation will include an analysis of child schooling outcomes in the future. ABC required parents to ensure children were attending school as part of a Contract of Mutual Responsibility (Fein, Lee, and Schofield, 1999). If attendance is unsatisfactory, the parent has agreed to cooperate with efforts to address the problem. Violations can lead to a reduction in the size of the cash grant, up to a permanent loss of benefits.(back)
102Recall that in Delaware, the control group was enrolled in ABC 18 months after the initial randomization. Thus, by the second year of follow-up, some controls had received the treatment for up to 6 months. Also, by year three a substantial fraction of the treatment group had begun to hit their 24-month time limits (but none of those in the treatment group whose clocks started 18 months later).(back)
103Other recent syntheses of this literature include Michalopoulos and Berlin (2001), Bloom and Michalopoulos (2001), Duncan and Chase-Lansdale (2001), Hamilton, Freedman, and McGroder (2000), and Morris et al. (2001).(back)
104New Hope also offered a generous child care subsidy for children up to age 13. The increased use of formal child care centers and after-school programs may explain some of the favorable impacts on child outcomes for this program.(back)
105The stated sanction was $25 per month but compensatory policies in food stamps and housing vouchers effectively reduced the sanction to $10.(back)
| Table of Contents | Previous | Next |