13. ANALYSIS OF CHILD OUTCOMES

Longitudinal studies have found that welfare use directly affects the economics of the household and the behavior of the parents receiving it. This has serious implications for the children in those households (Duncan & Brooks-Gunn, 1997). Specifically, factors such as the level of employment, the level of earnings, how much welfare improves overall income, the level of stress, and the schedule and conventionality of working hours can indirectly affect child wellbeing (Huston, 2002). Figure 1 depicts a conceptual model of how welfare policies can affect child outcomes.

Although employment for single parents on AFDC is a key goal of current welfare policy, the effects of maternal employment on child outcomes are not clear. When maternal employment does appreciably increase income, the effects on children are positive, but employment may not increase income if there is a loss of cash transfers and in-kind services (Bloom and Michalopoulos, 2001). Also, increasing a family’s material resources by increasing maternal employment can change the amount of time parents spend with children, as well as relationships among family members. For example, increased employment can lead to less supervision for children, which can be particularly problematic for adolescents in low-income neighborhoods. Research shows that in such cases, adolescents displayed increased behavior problems and lower educational attainment (Huston, 2002).

While there is debate about the effects of income on child wellbeing (Mayer, 1997), there is evidence that improving a family’s overall financial and material resources during a child’s early years has significant, positive benefits on that child’s cognitive abilities, educational attainment, and employment status later in life (Duncan and Brooks-Gunn, 1997). However, it is possible that other factors are affecting these outcomes. For example, working families with higher incomes use formal childcare centers more than low-income families, who are more likely to use informal childcare arrangements with members of extended family, and studies have shown that formal childcare arrangements produce better outcomes for children (NICHD, 1997). Also, low-income is correlated with higher parenting stress and the use of harsher punishments, which is associated with children’s diminished socio-emotional wellbeing (Conger and Elder, 1994; McLoyd, 1998). Indeed, the receipt of temporary cash assistance alone has not been shown to have much of a relationship with child outcomes after controlling for other demographic, human capital, and environmental factors (Yoshikawa, 1999).

The problem with the interpretation of these findings is that the data are largely correlational in nature. In order to control for other endogenous variables, such as parental characteristics, experimental studies are needed. Huston (2002) refers to the relationship between welfare, financial resources, family systems, and child development as a “black box” in which researchers need to find causal links and pathways. The best way to discover these links is through random assignment experiments.

The design of random assignment experiments enables researchers to control for unmeasured differences such as the individual characteristics of the parents and children and their family histories, while determining the effects of the tested interventions on children. When measured, child outcomes are generally broken down into three categories: academic/cognitive, behavioral/emotional, and health/safety. Seven of the random assignment evaluations of mandatory welfare-to-work programs in our database provided sufficient information on child outcomes for analysis as part of this report. The nature and respective findings of each of these programs are discussed briefly below, with particular emphasis on their impacts on child outcomes.

13.1 EVALUATIONS THAT MEASURED CHILD OUTCOMES

Connecticut’s Jobs First Program: This demonstration project was one of the first to focus on strict time limits for its recipients. In addition to time limits, however, it also offered financial work incentives and an intensive focus on quick job placement. While Jobs First attained its goal of replacing welfare with work among the treatment group, material hardship stayed high for both the control and treatment groups. Jobs First had mixed effects on children. Children between the ages of 5 and 8 exhibited more positive behavior and fewer behavioral problems. Likewise, adolescents in the treatment group were less likely to be convicted of a crime; however, their overall school achievement was significantly less than adolescents in the control group (Bloom et al., 2002).

Florida’s Family Transition Program (FTP): This demonstration project focused primarily on time limits for welfare receipt. It also offered financial incentives. The overall effects on adults showed that the time limits increased employment and earnings, but did not significantly increase overall family income. Further, by the time of the four-year follow-up, all of these effects had disappeared. For children aged 5 to 12, there were no significant effects in the academic/cognitive area, but there were weak decreases in behavioral/emotional outcomes and weak increases in health and safety. For adolescents, school suspensions significantly increased (Morris et al., 2001).

Los Angeles Jobs First Greater Avenues for Independence (LA GAIN): The overarching goal of LA GAIN was to convert an education-first welfare-to-work program to a work-first focused program. It was successful for single parent families; by two years after random assignment, the intervention caused employment rates to increase substantially, welfare participation to fall, and earnings to increase. However, little to no systematic effects on children were found, although there was a slight increase in behavioral problems and academic achievement among a small group of preschoolers (Freedman, 2000).

The National Evaluation of Welfare-to-Work Strategies (NEWWS): This demonstration project included mothers who were 19 years of age or older with children who were three years old or above. Child outcome data were collected in four of its seven sites, including three sites in which mothers could be assigned to one of two treatment groups: education-first or work-first. The impacts on children from each group were generally weak, and when they did occur, they were typically positive in some sites and negative in others. The research suggests that even though the mothers in NEWWS had increased employment, their overall income levels did not increase much because of lost welfare benefits. This may help explain the weak and inconsistent impacts on children (Morris et al., 2001).

The Minnesota Family Investment Program (MFIP): MFIP provided families with strong financial work incentives by providing cash supplements and subsidizing child and health care. Additionally, there were no time limits placed on the receipt of welfare benefits. As a result, employment and overall income increased throughout the final three-year follow-up assessment, while poverty decreased among the families assigned to welfare-to-work programs. Impacts on children were mixed, however. There were very weak or no impacts on children under 5; positive impacts on behavioral/emotional and academic/cognitive measures for children age 5 to 12 (Morris et al., 2001), and negative impacts on externalizing behaviors such as smoking, drinking, and substance abuse for adolescents. Further, there was a significant decrease in injuries and accidents for children (Gennetian et al., 2002).

Vermont’s Welfare Restructuring Project (WRP): WRP was an intensive demonstration project that aimed to promote work and decrease dependency on welfare. Time limits were enforced and if members of the program group did not find employment, they were placed in minimum-wages jobs. Failure to comply with WRP work requirements resulted in the state taking over the welfare grant. Financial incentives were offered to encourage work. Although WRP increased employment and reduced cash assistance, income and material hardship did not change because the increased income generated through employment was offset by the reduction in cash assistance. Program effects on children were minimal. Absenteeism for children aged 10-13 decreased, but adolescents were much more likely to get in trouble with the police (Scrivener et al., 2002).

Iowa’s Welfare Reform: The goal of Iowa’s welfare reform plan was to reduce the receipt of cash assistance by increasing employment through employment-oriented services, sanctions for non-compliance, and expanding earned income disregards. The program had mixed results. There was higher participation in job training and placement programs and employment increased. However, outcomes were poor among AFDC applicants, as their earnings and incomes decreased substantially. Furthermore, the children of applicants had poorer school achievement outcomes (Fraker et al., 2002).

13.2 A META-ANALYSIS OF PROGRAM EFFECTS ON CHILDREN

This brief review of the mixed impacts of welfare-to-work interventions on children provides a basis for further investigation. Using information in our database on demonstration projects that examined child outcomes, it is possible to get a clearer picture of the possible systematic effects welfare-to-work programs have on children. We hypothesize that increasing the net income of members of the program group (as measured by program impacts on net benefits) is positively related to program impacts on child outcomes. As previously demonstrated, one way welfare-to-work programs increase the incomes of AFDC recipients who go to work is through financial incentives. Thus, we also hypothesize that when financial incentives are offered by an intervention, this will improve impacts on child outcomes. On the other hand, time limits and program-induced increases in sanction rates are likely to increase stress within a family and reduce family income. Hence, they may have negative effects on program impacts on children. We also investigate whether impacts on the percentage of program group members who participated in job search, basic education, vocational training, and unpaid work experience affect program impacts on child outcomes. Participation in all of these activities may increase family stress. Moreover, mothers are taken out of the household while they are participating. Thus, we hypothesize that the impacts of welfare-to-work programs on children will be negatively affected to the extent programs increase participation in these activities. Finally, we examine whether more costly welfare-to-work programs (as measured by the government’s net operating cost) produce more positive impacts on children. A summary of the explanatory variables just considered is provided in Table 24 for each of the evaluations that measured program impacts on children.

The various child impacts that were measured by the evaluations include positive behavior, school achievement, suspension, expulsion, repeating a grade, behavioral or emotional problems, and/or child health. However, because different evaluations measured these impacts of welfare-to-work programs on children differently (see Appendix B for details of measurement by evaluation), it was necessary to make them comparable. In meta-analysis, this is usually done by converting impact estimates into an “effect size” measure. Using a method developed by Glass (1976), we do this by first subtracting the control group mean for each outcome from the treatment group mean for the outcome and then dividing the resulting impact estimate by the standard deviation of the control group means.

13.3 FINDINGS

Table 25 presents unweighted and weighted descriptive statistics for the child outcomes. A positive effect size for an outcome measure indicates that the treatment group had a positive impact on the child outcome, and visa versa. Cohen (1988) suggests that effect sizes can be interpreted as follows: small effect size = .20, medium effect size = .50, and large effect size = .80. The means and medians indicate that the effect size of impact was quite small for each respective outcome, as all are well below .20. However, the standard deviations and minimum and maximum values indicate that there is substantial variation among the evaluated programs.

The Q-statistic is used to test whether this variation is attributable to program differences or to sampling error. The results of these tests also appear on Table 25. All of the outcomes that were aggregated across age groups failed the test for homogeneity. Even when each age group is examined separately, all but two of the outcomes (young age suspension/expulsion and school age positive behavior) far exceed the critical value of chi-squared for p =.05. These results indicate that the observed variability among the effect sizes is unlikely to have resulted from sampling error alone and, hence, that some of the variation results from systematic differences among the interventions.

Next, we use weighted regressions to explore different possible explanatory variables that may account for some of the variation in effect sizes among child impacts. To maximize sample size in computing the regressions, we pooled the effect size estimates for the three age groups, using dummy variables to control for differences in effect size across these groups. Multicollinearity was a serious problem in conducting the regression analyses. Many of the program characteristics were highly correlated (r >.50) with one another and other contextual variables, such as race and poverty rate. This, combined with a small sample size, made it necessary to restrict the explanatory variables severely. This limitation should be considered carefully when interpreting the regression results.

As previously discussed, the most logical possibilities were whether the program offered a financial incentive (see Model 1, below), enforced time limits, or increased the sanction rate (Model 2). We also examined whether program impacts on children were influenced by the net cost of the programs to the government and net program benefits to members of the program group (Model 1). Finally, we tested whether program impacts on the percentage of parents participating in job search, basic education, vocational training, or unpaid work on child outcomes had an effect on program impacts on children (Model 3).24

13.4 REGRESSION FINDINGS

The regressions include all the intervention characteristic variables listed in Table 24 but net program benefits to members of the program group, as this was never a significant predictor of any of the program impacts on children. We computed exploratory regressions on all the child impact measures except the two for which we had only four observations (positive behavior and health). However, statistically significant coefficients were obtained only for regressions that used the impact on the “Behavioral/Emotional Problems” measure as a dependent variable. Thus, these are the only findings we report. These findings appear in Table 26 and are discussed below.

Model 1. Model 1 in Table 26 indicates that welfare-to-work programs have a less positive impact on the emotional and behavioral problems of school age than younger children, the comparison group. The coefficient for adolescents is also negative, but not statistically significant. The coefficient for the financial incentives dummy variable is negative and statistically significant, indicating that including financial incentives in a welfare-to-work program negatively affects the program’s impact on childhood emotional and behavioral outcomes even though they increase the incomes of working AFDC recipients. This result is the opposite of what we hypothesized. On the other hand, the coefficient on the net cost variable is positive and highly statistically significant, suggesting that more expensive welfare-to-work programs have more positive effects on this impact than less costly interventions.

Model 2. Like Model 1, Model 2 indicates that welfare-to-work programs have less positive impacts on the emotional and behavioral outcomes of school age children than they do on younger children. Again, adolescents have a negative, but statistically insignificant coefficient. In support of our hypothesis, the coefficient on the dummy variable for time limits is negative and statistically significant, indicating that time limits have a negative effect on the impact of welfare-to-work programs on children’s behavioral or emotional outcomes. In contrast to our hypothesis, however, increases in the use of sanctions appear to have a positive effect on this impact.

Model 3. The school age dummy variable is again statistically significant and negative in Model 3, adding further support to the finding that impacts for this age group are smaller than for younger children. The positive, statistically significant coefficient for participation in basic education and work experience indicates that increasing the use of these services has positive effects on program impacts on children’s behavioral and emotional outcomes.

13.5 SUMMARY OF KEY FINDINGS

Conclusions based on the results discussed above are summarized below. These findings, while limited, provide some insight into how the characteristics of welfare-to-work interventions influence program impacts on children. The findings should be considered highly tentative because important contextual variables were not included as controls.25 Further research on the factors that influence the impacts of welfare-to-work programs on children is clearly warranted.

• Overall, program impacts on children were small.

• There is evidence that the considerable variation across programs in their impacts on children is not entirely due to sampling error, but is partially attributable to systematic differences among the interventions. However, with the exception of impacts on the emotional and behavioral problems of children, we were unable to determine what these systematic differences might be.

• There is no support in our data that increasing the net income of welfare families improves child outcomes. However, the welfare-to-work programs that were examined did not produce large changes in the incomes of those assigned to them.

• When various program characteristics are controlled, impacts on emotional and behavioral problems are less positive for school age children than for young children.

• Three program features appear to positively affect the impact of welfare-to-work interventions on children’s behavioral and emotional outcomes: sanctions, participation in basic education, and participation in unpaid work. These findings are inconsistent with what we predicted. However, it may be worth noting that increasing the sanction rate was also found to have a positive effect on program impacts on getting program group members off AFDC and into jobs.

• Two features of welfare-to-work programs exert negative influences on impacts on childhood behavioral or emotional impacts: financial incentives and time limits. The result for financial incentives, while not supportive of our hypothesis, is consistent with the previously discussed finding that financial incentives decrease program impacts on AFDC participation and payments, while failing to increase their impacts on employment and earnings.

• Increasing expenditures on welfare-to-work programs has a positive effect on their impacts on childhood behavioral and emotional problems.