ACF OPRE: Early Head Start Research and Evaluation Project (EHSRE)

VIII. SUMMARY AND RECOMMENDATIONS

Using a rigorous, random-assignment research design, the national Early Head Start Research and Evaluation project documented the impacts of the 17 purposively selected programs on families and children at ages 2 and 3. In this chapter we summarize the key evaluation findings and draw lessons for programs, policymakers, and researchers.

A. KEY FINDINGS FROM THE ANALYSIS OF EARLY HEAD START IMPACTS

Early Head Start is making a difference for low-income families with infants and toddlers. By the time children’s eligibility for Early Head Start ends at age 3, programs stimulated better outcomes along a broad array of dimensions with children, parents, and their home environments. Some of the outcomes that the programs improved are important predictors of later school achievement.

For 3-year-old children, the Early Head Start research programs largely sustained the statistically significant, positive impacts on cognitive and language development that had been found at age 2. Early Head Start children were significantly less likely than control-group children to score in the at-risk range of developmental functioning in these areas. As previous research suggests, by moving children out of the lowest-functioning group, Early Head Start may be reducing their risk of poor cognitive and language outcomes later on.
The programs had favorable impacts on more aspects of social-emotional development at age 3 than at age 2. At age 3, Early Head Start children engaged their parents more, were less negative to their parents, and were more attentive to objects during play than were control children. Early Head Start children also were rated lower in aggressive behavior by their parents than control children.
When children were 3, the Early Head Start programs continued to have significant favorable impacts on a wide range of parenting outcomes. Early Head Start parents were observed to be more emotionally supportive and to provide more support for language and learning than control-group parents (for example, they were more likely to read to their children daily). They were also less likely than control-group parents to engage in negative parenting behaviors. Early Head Start parents were less likely to report that they spanked their child in the past week, and they reported greater knowledge of mild discipline strategies.
Fathers whose children were enrolled in Early Head Start were significantly more likely than fathers and father figures from control-group families to participate in program-related child development activities, such as home visits, parenting classes, and meetings for fathers. Although providing services specifically to fathers is relatively new for Early Head Start programs (in comparison to their history of serving mothers and children), the programs had significant favorable impacts in several areas of fathering and father-child interactions.
The Early Head Start programs had several important impacts on parents’ progress toward self-sufficiency. The positive impacts on participation in education and job training activities continued through 26 months following enrollment, and some impacts on employment began emerging late in the study period in some subgroups. These impacts did not result in significant improvements in income during this period, however.
Early Head Start mothers were somewhat less likely to experience subsequent births during the first two years after they enrolled and may therefore have been less likely to experience the economic and psychological consequences of closely spaced births.
The program impacts on children and parents in some subgroups of programs were larger than those in other subgroups. The subgroups in which the impacts were relatively large (with effect sizes in the 20 to 50 percent range across multiple outcomes) included mixed-approach programs, African American families, mothers who enrolled during pregnancy, and families with a moderately high (vs. a low or very high) number of demographic risk factors. In a few subgroups, the programs produced few significant favorable impacts. Knowledge of these variations in impacts across subgroups can be used to guide program improvement efforts.

The consistent pattern of statistically significant, favorable impacts across a wide range of outcomes when children were 2 and 3 years old, with larger impacts in some subgroups, is promising. Most impacts were modest (with effect sizes in the 10 to 20 percent range), but the wide range of impacts on both children and parents suggests that Early Head Start programs may be improving the balance of risk and protective factors in the lives of the low-income families they serve. Whether this broad range of modest impacts that have been sustained through toddlerhood will continue through childhood is unknown. However, the overall pattern of findings—modest impacts on a wide range of child and parent outcomes that were sustained until age 3—suggests that the programs are building assets in children and families that may well continue to facilitate positive outcomes later on. Also, the program impacts on children and parents in some subgroups of families and programs were considerably larger than the overall impacts (effect sizes ranging from 20 to 50 percent), suggesting that for some children and families, the potential longer-term impacts may be larger. This was true both for program subgroups (mixed-approach programs, especially those that were fully implemented early) and family subgroups (particularly, mothers who enrolled during pregnancy, African American families, and families with a moderate number of demographic risk factors).

Early Head Start programs took both direct (providing services to children directly) and indirect (providing services through parents) pathways to accomplishing their goals. Consistent with many programs’ theories of change, we found evidence that the programs’ impacts on parenting when children were 2 years old were associated with impacts on children when they were 3 years old. For example, higher scores on the cognitive development measure at age 3 were associated with higher levels of parent supportiveness in play and greater support for cognitive and language development when the children were 2; similarly, lower levels of aggressive behavior when children were 3 were related to greater parental warmth and lower levels of spanking when the children were 2 years old.

The programs’ impacts on child and family outcomes were also consistent with the finding that programs substantially increased their families’ receipt of services relative to control families. Given the voluntary nature of the Early Head Start program, participation levels ranged from no participation to intensive participation throughout the evaluation period. Overall participation rates, however, were high during the first 28 months after enrollment. Furthermore, a high percentage of program families received intensive services, a reflection of the substantial efforts of program staff to engage families in ongoing services. On average, program families participated in Early Head Start for 21 months. These high levels of participation are reflected in large impacts on service receipt. Although other services were available in the Early Head Start communities, and although many control group families received some services, Early Head Start families were, during the first 28 months after random assignment, significantly more likely to receive a wide variety of services, much more likely to receive intensive services, and much more likely to receive intensive services that focused on child development and parenting.

Implementing key services in accordance with the Head Start Program Performance Standards for quality and comprehensiveness appears to be important to success. When children were 2, programs that had fully implemented key elements of the Head Start Program Performance Standards early had a stronger pattern of impacts than programs that reached full implementation of the standards later or not at all during the evaluation period. The differences in impacts on children and parenting among programs that fully implemented the standards early, later, or incompletely became less distinct by the 3-year assessment point, when all three groups of programs had some important impacts. Nevertheless, the findings show that:

Programs that were fully implemented (whether early or late) produced a broader range of impacts at age 3 than the incomplete implementers.
Although it is not possible to fully disentangle the effects of program approach and implementation pattern, there is evidence that reaching full implementation contributes to a stronger pattern of impacts. Mixed-approach programs that were fully implemented early demonstrated a stronger pattern of impacts (and some of the largest impacts detected in the study) than those that were implemented later or not at all. Home-based programs that were fully implemented early or later demonstrated impacts on some important outcomes that other home-based programs did not have.

All program approaches for delivering services produced impacts on child and parent outcomes. Programs chose their service approach based on their understanding of local family needs, their philosophies of best practice, and the resources available. This may partially explain findings showing that programs selecting different approaches had different patterns of outcomes:

The center-based programs, which had the greatest impacts on receipt of center-based child care and the amount of child care received, consistently enhanced cognitive development and, by age 3, reduced negative aspects of children’s social-emotional development. The programs also demonstrated favorable impacts on several parenting outcomes, but had few impacts on participation in self-sufficiency-oriented activities.
The home-based programs, which had the greatest impacts on receipt of home visits, case management, and parent-child group activities, had favorable impacts on language development at age 2, but not at age 3. They had a favorable impact on children’s engagement of their parents in semistructured play interactions at age 3. Only a few impacts on parents were significant, but parents in home-based programs reported less parenting stress than their control group did. Implementing home-based programs was challenging. Nevertheless, those that reached full implementation by fall 1999 had a stronger pattern of impacts. When the children were 3, the fully implemented programs had significant favorable impacts on cognitive and language development that have not generally been found in evaluations of home-visiting programs.
Programs that offered both home-based and center-based options in response to local families’ needs (the mixed-approach programs) had more flexibility in serving individual families, were able to keep them engaged in services longer on average, and had a pattern of stronger impacts on children and families. The mixed-approach programs consistently enhanced children’s language development and aspects of social-emotional development. These programs also had consistent significant favorable impacts on a wider range of parenting behavior and participation in self-sufficiency-oriented activities. The mixed-approach programs that became fully implemented early had a particularly strong pattern of impacts. The stronger pattern of impacts among mixed-approach programs may reflect the benefits of families receiving both home-based and center-based services, the value of programs’ flexibility to fit services to family needs, or the fact that these programs were able to keep families enrolled somewhat longer.

The impacts of the Early Head Start research programs were broad. The programs reached all types of families with child development services and provided them with a significantly greater number of services, and services that were more intensive than families would have received in their communities without the benefit of Early Head Start. By age 3, most subgroups of children benefited in some way from participating in Early Head Start. Similarly, most subgroups of parents benefited in some way related to their parenting. The programs also helped parents in most subgroups work toward self-sufficiency.

Analyses of program impacts on subgroups of children and families also suggest:

Earlier intervention is better. The 17 Early Head Start research programs appear to have been more effective in improving child outcomes in families that enrolled before their child was born than in families that enrolled after their child was born (some effect sizes were as large as 50 percent). However, children who were born after enrollment also benefited from the program, and program impacts on parenting were similar across these groups.
Both firstborn and later-born children and their families benefited from participating in Early Head Start, although the pattern of impacts differed between these groups. The programs had significant favorable impacts on child development and parenting in both groups of families. Early Head start consistently increased the participation in education of parents of firstborn children, however, and reduced the proportion who had another baby during the first two years after enrollment.
Early Head Start appears to have provided a foundation of support for children’s development among families in which parents reported symptoms of depression when they enrolled, a group that other studies have found to be difficult to serve. Among parents at risk of depression in the eight research sites that measured depression at baseline, Early Head Start parents reported significantly less depression than control-group parents when children were 3. Early Head Start also demonstrated a favorable pattern of impacts on children’s social-emotional development and parenting outcomes among these families.
Early Head Start also appears to have provided support for children’s development in families of teenage parents. Like other programs designed to increase self-sufficiency among disadvantaged teenage parents, the Early Head Start research programs succeeded in increasing school attendance among teenage parents. Unlike other large-scale programs, however, the programs also enhanced their children’s development.
Families with many demographic risks usually pose difficult challenges for early intervention and family support programs, and this was true for the Early Head Start research programs as well.1 Program impacts on the families with more than 3 risks were unfavorable, although programs did significantly delay subsequent births in the group with more than 3 risks. Previous research suggests that low-income families who have experienced high levels of instability, change, and risk may be overwhelmed by changes that a new program introduces into their lives, even though the program is designed to help. As a result, the program requirements may create unintended negative consequences for these families. Because families with the most risks were more likely to be in home-based or mixed-approach programs that were not fully implemented early, it is possible that the staff turnover and disruptions in staff-family relationships experienced in some of these programs had an adverse effect on the most vulnerable families. Early Head Start had strong impacts, however, on families with a moderate number of demographic risks.
The Early Head Start programs were especially effective in improving child development and parenting outcomes of the African American children and parents who participated, and they also had a favorable pattern of impacts on the Hispanic children and parents who participated. While many impacts on child development and parenting were favorable among white families, virtually none was statistically significant. The more-disadvantaged status of African American control group children and families relative to the control families in other racial/ethnic groups may have set the stage for the Early Head Start programs to make a larger difference in the lives of the African American children and parents they served. Early Head Start brought many of the outcomes of African American children and parents in the program group closer to the levels experienced by the other racial/ethnic groups.

B. RECOMMENDATIONS FOR PROGRAMS, POLICY, AND RESEARCH

The impact findings, taken together with findings from the study of program implementation (see Pathways to Quality), suggest several lessons for programs. Several of the lessons pertain to program implementation:

Fully implementing key elements of the Head Start Program Performance Standards is important for maximizing impacts on children and parents. The research programs that reached full implementation by fall 1999 had a stronger pattern of impacts on child and family outcomes than the programs that did not.
If they offer center-based services, programs should seek ways to place greater emphasis on parenting, parent-child relationships, and family support, areas in which the center-based research programs did not have a strong pattern of impacts. They should also increase efforts to support language development and do even more than they are already doing to foster cognitive development.
If programs offer home-based services, they should strive to deliver a greater intensity of services, including more frequent home visits, while also attending to children’s cognitive development and encouraging and supporting center-based activities for children as they become older toddlers. As documented in the implementation study, delivering home visits at the required intensity was extremely challenging, and the pattern of impacts produced by the home-based research programs suggests that doing so is important.

Several lessons for programs emerge from the evaluation findings related to specific outcomes:

To ensure the safety of infants and toddlers, programs (especially center-based ones) should be more vigilant about parental safety practices. The programs did not increase consistent, correct use of car seats among families.
Greater access to services to address the mental health needs of parents, many of whom reported symptoms of depression and parenting stress, is needed. Although several subgroups demonstrated that favorable impacts on parent mental health outcomes are possible, we found no significant overall impacts on receipt of mental health services or on parent mental health outcomes.

Finally, several recommendations for programs pertain to which families they should seek to enroll and the timing of enrollment:

Programs should enroll parents and children as early as possible, preferably before children are born. Although the programs improved outcomes among children who were enrolled after birth, the strongest pattern of impacts was achieved with children whose families enrolled during pregnancy.
Programs should enroll parents at all stages of childbearing. The research programs had favorable impacts on both firstborn and later-born children and their parents.

The evaluation findings also have implications for policymakers, including Head Start Bureau staff and policymakers concerned with programs and policies serving low-income families with young children:

Early Head Start programs may provide support for children’s development among families who may be struggling with their own needs. While increasing parents’ participation in education and employment-oriented activities, the Early Head Start research programs had significant favorable impacts on children’s development. These improvements occurred despite the fact that average family income did not increase significantly.
Early Head Start programs may provide an effective way of serving some difficult-to-serve families. The research programs achieved favorable significant impacts among teenage parents and parents who reported depressive symptoms when they enrolled, including significant favorable impacts on children as well as parents.
Like other early childhood programs, Early Head Start programs may have the greatest opportunity to improve outcomes among families with a moderate number of demographic risks, but may have greater difficulty improving outcomes among families with young children who have four to five of the risk factors measured.
This study validated the importance of meeting the Head Start Program Performance Standards for achieving impacts on children and parents, and it underscores the value of monitoring programs regularly. The performance standards may be useful as a guide to providing effective services in other early childhood and early intervention programs.
The strong pattern of impacts among mixed-approach programs suggests that flexibility in service options for families may be valuable when community needs assessments show that both home-based and center-based services are needed.

Finally, the national Early Head Start Research and Evaluation project incorporated some innovative features into a large, multi-site evaluation, and the evaluation findings have implications for researchers:

Devoting significant resources to conceptualizing, documenting, and analyzing the implementation process and understanding as fully as possible the approaches (strategies and activities) that programs take in delivering services is critical for understanding program impacts and deriving lessons from them. Pathways to Quality (ACYF 2002) includes information on methods of rating implementation and defining program approaches that may be useful to researchers investigating similar topics in Early Head Start and other programs.
Using multiple methods for measuring outcomes, so that findings are not dependent only on parent reports, child assessments, or any single methodology, increases the confidence that can be placed in the impact findings. The Early Head Start findings are based on a mixture of direct child assessments, direct observations of children’s behavior by trained observers, ratings of videotaped parent-child interactions in standardized ways, ratings of children’s behaviors by their parents, and parents’ self-reports of their own behaviors, attitudes, and circumstances. Details of the measurement process in the Early Head Start evaluation can be found in Volume II, Appendix C.
Identifying subgroups of programs and policy-relevant populations is valuable so that analyses can begin to address questions about what works for whom. Having adequate numbers of programs and adequate sample sizes within sites to make program-control comparisons of outcomes for particular subgroups of sites or subgroups of families can provide important insights into program impacts under particular conditions and for particular groups of families. Researchers do not always have the benefit of the large, multisite sample that was created for the Early Head Start national evaluation, but if questions about multiple approaches across multiple populations are of interest, every effort should be made to increase sample sizes and variability.
Incorporating local perspectives in national evaluation studies enables the voices of programs and local researchers to supplement the cross-site analyses and enhance the interpretation of the national findings. In text boxes throughout this report, and in more in-depth write-ups in Volume III, it is possible to see the diversity of research at the local program level that can be brought to bear on a large number of developmental, programmatic, and policy questions.
Partnerships with local programs were important to the success of the evaluation, and participating in the research enhanced local programs’ continuous program improvement processes.

C. NEXT STEPS

More analyses are available in two special policy reports that provide additional findings related to children’s health and child care. In addition, members of the Early Head Start Research Consortium are continuing to analyze national data, and local research partners are analyzing local data. Reports similar to those presented in Volume III will continue to appear in the coming months and years. Finally, ACF/ACYF are sponsoring a longitudinal follow-up study in which the children in the national sample at the 17 sites are being assessed, and their mothers and fathers interviewed, as they enter kindergarten. The follow-up study, which will be completed by 2004, will provide an opportunity to learn about the experiences of Early Head Start children and families after they leave the program.

1The demographic risk factors considered include (1) being a single parent, (2) receiving welfare cash assistance, (3) being neither employed nor in school or job training, (4) being a teenage parent, and (5) lacking a high school diploma or GED.(back)

Table of Contents | Previous | Next