Table of Contents | Previous | Next |
VI. VARIATIONS IN IMPACTS BY PROGRAM APPROACH AND PATTERN OF IMPLEMENTING KEY FEATURES OF THE PERFORMANCE STANDARDS
Reflecting the diversity of communities and families they served, the 17 Early Head Start research programs varied in the approaches they took to providing services and in the time it took them to reach full implementation of the Head Start Program Performance Standards. As discussed in Chapter IV, program impacts on the services families received varied significantly when programs were grouped by program approach and by pattern of implementation. Because the pattern of impacts on service receipt and intensity differed across these groups, we expected that program impacts on child and family outcomes might also vary on these dimensions.
To briefly summarize the patterns of impacts on service receipt discussed in Chapter IV, home-based programs had the largest impacts on receipt of home visits, weekly home visits during at least one follow-up period and throughout the entire follow-up period, and participation in parent-child group socialization activities. Center-based programs had relatively larger impacts on the use of center-based child care and weekly out-of-pocket costs of child care. Mixed-approach programs tended to have impacts on service use that were between those of home-based and center-based programs but were often closest in magnitude to the impacts that home-based programs had on service use. Similarly, as expected, programs that implemented key performance standards early had somewhat larger impacts on the receipt of any key services (home visits, center-based care, and case management) and larger impacts on the receipt of core child development services and home visits at the required intensity than programs that were not fully implemented until the later period or were incompletely implemented.
Analyses of differences in impacts on child and family outcomes by program approach and implementation pattern show that while all groups of programs had significant impacts on some child and family outcomes, impacts varied across these groups. When children were 3 years old, mixed-approach programs had a stronger pattern of impacts on child and family outcomes than the other programs, but center-based programs also had some important impacts. Home-based programs had fewer significant impacts. With respect to implementation patterns, all three groups of programs had some favorable impacts on child and family outcomes. However, the early and later implementers had significant favorable impacts on a broader range of outcomes than the incomplete implementers. The early implementers had impacts on depression and employment not found among the other programs. Mixed-approach programs that fully implemented key aspects of the Head Start Program Performance Standards early produced some more-favorable impacts (with some of the largest effect sizes detected in the study) and the home-based programs that were fully implemented either early or later produced favorable impacts on some important outcomes, including children’s cognitive and language development.
We also examined some other program- and site-level subgroups to explore whether Early Head Start impacts varied as a function of either urban/rural program location or whether state welfare regulations require parents to engage in work activities while their youngest child is under 1 year old. Neither of these other analyses suggested that they were important ways of classifying programs to examine differences in impacts on services or on children and families. Tables showing the impacts of Early Head Start by these subgroups may be found in Appendix E.VI.
This discussion focuses on several aspects of the subgroup findings. First, we interpret the subgroup impacts in the context of the overall impacts reported in Chapter V. In some cases, although Early Head Start had an overall impact when averaging across all sites, none of the individual subgroup impacts is significant. This may be due, in part, to the substantially smaller sample sizes when examining each subgroup. We interpret such situations to mean that all program approaches contributed to the overall impact. In interpreting these findings, we also take effect sizes into account, and in order to understand patterns of effects, we describe program-control differences as “favorable” when effect sizes are larger. Interpretation of subgroup findings is also aided by the chi-square test, which is statistically significant if the program-control differences differ across the three subgroups. A significant chi square does not always tell us where that difference lies, however, so that is a matter of interpretation. Finally, we consider patterns across outcome variables within clusters of outcomes (child cognitive and language, child social-emotional, parenting, and so forth). Given these considerations, our approach to interpreting subgroup effects is necessarily more complex than to reporting overall impacts as in Chapter V. For example, we note relatively large impacts even when they are not statistically significant so as to identify patterns of findings, and note this in the text so that readers may form their own conclusions. By considering (1) the overall (full-sample) impacts, (2) impacts within each subgroup, (3) the magnitude of the program-control differences, (4) the chi-square statistic, and (5) patterns of differences within clusters of outcomes for a particular subgroup and for a single outcome across subgroups, we draw our interpretations with respect to the meaning of the findings for Early Head Start programs and policy.
The following sections discuss variations in program impacts on child development, parenting, and family well-being by program approach and implementation pattern. The final section draws conclusions from these findings.
A. HOW CHILD DEVELOPMENT, PARENTING, AND FAMILY WELL-BEING IMPACTS VARIED BY PROGRAM APPROACH
As described in Chapter I, the Early Head Start programs adopted three main approaches to providing child development services based on the needs of children and families in their communities.1 Home-based programs provided these services primarily through frequent home visits, as well as through parent-child group socialization activities. Center-based programs provided child development services primarily through child care in Early Head Start centers supplemented by parenting education and family support services. Mixed-approach programs provided home-based services to some families, center-based services to some families, and a mix of home- and center-based services to some families. This mix of services could occur across different families or across time with the same families, depending on how the program designed its services to meet families’ needs (see Chapter I). Regardless of the pattern of services, home visits and child care in Early Head Start centers were the two primary vehicles through which programs delivered child development services.
In 1997, four programs took a center-based approach; seven programs took a home-based approach; and six programs took a mixed approach. By 1999, home-based and center-based programs were beginning to offer a greater mix of services in response to the changing needs of families and children in the program. In particular, some home-based programs began offering some center-based care to families that needed it, either directly or by partnering with local, good-quality infant/toddler care providers. Few research families used the new center-based slots, however. Other home-based programs began working with child care providers to improve the care offered to program children. Because the impacts on service use continued to differ according to programs’ approaches to service delivery in 1997, we examined differences in impacts on child and family outcomes according to the programs’ approaches in 1997.
Program approaches were not randomly determined, but instead, Early Head Start programs chose program approaches and an array of services that were most appropriate for their communities and the families they expected to serve. Family characteristics differed by program approach (as discussed in Chapter II) as did the communities in which the programs operated and the programs’ patterns and levels of implementation. As a result, the pattern of impacts by program approach should not be interpreted as a test of which program approach is most effective but as a test of the effectiveness of each approach among programs that chose that approach.
In this section, we discuss the impacts of Early Head Start by program approach, presenting the impacts in three subsections—child development, parenting, and parents’ physical and mental health and self-sufficiency. In discussing the subgroup findings below, we focus on several different aspects of the findings.
1. Child Development
When children were 3 years old, impacts on children’s cognitive, language, and social-emotional development were favorable and statistically significant overall. For most child development outcomes, the program impacts did not differ significantly by program approach. Mixed-approach programs had a somewhat stronger pattern of favorable impacts on children with significant effect sizes in the 20 to 30 percent range, although center-based and home-based programs also had some important impacts (see Table VI.1). Impacts on the Bayley Mental Development Index (MDI) at age 3 (reported in Chapter V) did not differ significantly by program approach. While the impacts on the proportion of children scoring below 85 on the Bayley MDI were not statistically significant in any of the three groups, center-based programs had a significantly stronger favorable impact on the proportion of children scoring below 85 than the other programs. When children were 2 years old, the Early Head Start impacts on cognitive development were more strongly associated with center-based programs than was true when children were 3 years old.
Impacts on children’s receptive vocabulary scores (PPVT-III) did not differ significantly across program approaches; however, only the impact for mixed-approach programs was large enough to reach statistical significance. Mixed-approach programs also reduced the proportion of children with receptive vocabulary scores below 85 significantly and to a significantly greater extent than did other programs. The stronger impacts on language development among mixed-approach Early Head Start programs are consistent with the interim findings when children were 2 years old.
Among the positive aspects of children’s social-emotional development at age 3, the impacts of Early Head Start on observational measures of behavior were generally in a favorable direction and not significantly different across program approaches. One impact among home-based programs and two impacts among mixed-approach programs reached statistical significance. Early Head Start had a significant positive impact on children’s engagement of the parent in semistructured play in home-based and mixed-approach programs. The impact on this outcome among children in center-based programs was relatively large, but not statistically significant. Early Head Start also led to significantly greater sustained attention with objects in semistructured play among children in mixed-approach programs.
When children were 3 years old, the favorable impacts of Early Head Start on positive aspects of children’s behavior were similar to those found at age 2 among mixed-approach programs. The favorable impact at age 3 on children’s engagement of their parents in play among home-based programs, however, was not found when children were 2.
Among the negative aspects of children’s social-emotional development at age 3, the impacts of center-based Early Head Start programs tended to be consistently favorable. Although the differences in impacts across program approaches were not statistically significant, center-based programs significantly reduced negativity toward the parent in semistructured play. Moreover, the center-based programs tended to reduce parent-reported aggressive behavior and frustration in the puzzle challenge task, but these impacts were not large enough to reach statistical significance.
The pattern of stronger favorable impacts of center-based programs on negative aspects of children’s social-emotional behavior is somewhat different from the pattern we found when children were 2 years old. The impacts of the mixed-approach programs on negative behaviors were more favorable at age 2, and the reduction in aggressive behavior was statistically significant among the mixed-approach programs. At age 2, the impacts of center-based programs on aggressive behavior were favorable but not statistically significant.
These findings suggest that the favorable overall impacts of Early Head Start on children’s cognitive development, language development, aggressive behavior, and behavior in relation to the parent during semistructured play did not differ greatly across program approaches. However, mixed-approach programs appear to have had greater impacts on language development and on positive aspects of social-emotional behavior, while center-based programs tended to have favorable impacts on the cognitive development of children with mild delays and on one negative aspect of children’s social-emotional behavior.
2. Parenting
Early Head Start had favorable impacts on important aspects of parenting when children were 3 years old across all three program approaches, but impacts appeared to be stronger (with effect sizes often in the 20 to 30 percent range) and more consistent across a broad range of parenting behavior for parents in mixed-approach programs (Table VI.2). This finding is consistent with the pattern of impacts reported for parents when children were 2 years old (ACYF 2001).
When children were 3 years old, Early Head Start had a favorable overall impact on the organization, stimulation, and support provided in the home environment, as measured by the total HOME score. For each program approach, the impact of Early Head Start on total HOME scores was favorable, but not statistically significant. In contrast, when children were 2, only home-based and mixed-approach programs had favorable impacts on the total HOME score.
When children were 3 years old, the overall impacts of Early Head Start on emotionally supportive parenting were generally favorable and did not differ significantly across program approaches. Parents in home-based and mixed-approach Early Head Start programs were rated as more supportive toward their child in semistructured play than control-group parents in those sites, and the impacts were statistically significant. When children were 2 years old, favorable impacts on emotional support also occurred within both home-based and mixed-approach programs, and were statistically significant in most cases. Impacts on aspects of stimulation of children’s cognitive and language development were generally more favorable among parents in mixed-approach programs. Several impacts in this area were favorable for parents in center-based programs, but only one reached statistical significance. The home-based programs did not have any impacts on support for children’s cognitive and language development. Among parents in mixed-approach programs, Early Head Start had a significant impact on the quality of assistance provided to the child during the puzzle challenge task, the number and frequency of parent-child play activities, and whether the parent read to the child every day. Center-based Early Head Start programs had a favorable impact on the number and frequency of parent-child play activities.
When children were 3 years old, parents in mixed-approach programs were significantly less detached from the child in semistructured play than control-group parents. In contrast, parents in center-based programs tended to show greater detachment during semistructured play compared with their control-group counterparts, although this difference also was not statistically significant.
Participation in Early Head Start center-based and mixed-approach programs led parents to reduce physical punishment, both the incidence of spanking in the past week as reported by the parent and physical punishment as a reported discipline strategy. The impacts of the mixed-approach programs on these outcomes were statistically significant, and while not statistically significant, the effect sizes for impacts on these outcomes for parents in center-based programs were comparable to those of the mixed-approach programs. This finding suggests that mixed-approach and center-based Early Head Start programs may offer more information or different types of services that help to educate parents and reduce physical punishment.
A perplexing finding emerged with regard to the safe and consistent use of car seats. Although Early Head Start had no overall effect on car seat safety, Early Head Start parents in center-based programs were significantly less likely than their control-group counterparts to report using car seats consistently and safely. This finding could have emerged by chance, but it is consistent with a pattern of unfavorable impacts on safety practices at age 2 and might suggest that center-based programs need to focus on car-seat safety practices.
Thus, when children were 3 years old, Early Head Start had favorable impacts on a wide range of important parenting behaviors for parents in mixed-approach programs, including emotional support, stimulation of language and learning, levels of negative parenting behavior, and punitive discipline strategies. For parents in center-based programs, a pattern emerged in which Early Head Start also enhanced some important aspects of emotional support and support for cognitive and language development and reduced reported use of physical punishment (although, perhaps because the sample size in this subgroup was smaller, many of these impacts were not statistically significant). These results are broadly consistent with the findings when children were 2 years old. However, in contrast to the findings at age 2, when there were several important statistically significant impacts on parents in home-based Early Head Start programs, there was only one significant impact (on supportiveness of the child during semistructured play) for parents in home-based programs when children were 3 years old. Other impacts that were significant at age 2 remained favorable but were no longer statistically significant at age 3.
3. Parents’ Physical and Mental Health and Self-Sufficiency
Although Early Head Start had no overall impact on parents’ mental health or family conflict when children were 3, within subgroups by program approach, the programs did have some impacts (Table VI.3). Parents in home-based programs reported significantly lower levels of parental distress than their control-group counterparts and, although the impacts were not large enough to be statistically significant, Early Head Start also appeared to reduce parental distress among parents in mixed-approach and center-based programs. This finding is broadly consistent with the significant favorable impact on parental distress among mixed-approach programs and the favorable, though not significant, impact found among home-based programs when children were 2 years old.
At the same time, Early Head Start programs had an unfavorable impact on reported feelings of depression among parents in center-based programs. While average levels of depressive symptoms were unchanged, the proportion of parents with severe depressive symptoms was significantly higher among parents in the center-based program group compared with the control group, which had relatively low rates of severe depression when children were 3. Impacts on other aspects of parenting that might also be expected to be unfavorable due to the increase in depressive symptoms were not unfavorably affected (for example, supportiveness and intrusiveness during play). When children were 2 years old, we did not find higher levels of depression among parents in center-based programs using a different measure of depression.2 When children were 3, there were no significant impacts on reported feelings of depression in mixed-approach and home based-programs, where base rates of symptoms of severe depression were about twice as high as those in center-based sites.
All three program approaches had at least some positive impacts on participation in education and training activities during the follow-up period (Table VI.4). Home-based and mixed-approach programs had a significant positive impact on the proportion ever participating in education and training programs. The mixed-approach programs had a significantly larger impact than the other programs. Among parents in home-based programs, most of this activity focused on high school education. Among parents in mixed-approach programs, the activity was a mix of high school and vocational education. The home-based and mixed-approach programs also increased parents’ average hours per week in education and training programs significantly, although the impacts were small in terms of hours.
Program impacts on quarterly participation rates in education and training programs were favorable during several quarters of the follow-up period for all three subgroups by program approach. Impacts were statistically significant among mixed-approach programs in quarters 3 and 4; in quarters 4 through 8, impacts were statistically significant among home-based programs. Impacts among center-based programs were comparable in size to those of the other two program approaches in quarters 3 through 6, but were not statistically significant.
Early Head Start mixed-approach programs had a significant positive impact on the proportion of parents who were ever employed, with most of the difference in employment occurring during the second year after enrollment. Impacts on quarterly employment rates were significant among parents in mixed-approach programs in quarters seven and eight. Early Head Start had no statistically significant impact on employment among parents in either center-based or home-based programs, although the impact of center-based programs on employment was favorable. It is possible that the capacity of mixed-approach programs to match parents with good-quality child care when they were ready to consider working helped to ensure that parents could more successfully make the transition to employment than similar parents in the control group. In contrast, the lack of a significant employment impact among parents in center-based programs may be attributable to a stronger initial attachment to the labor force, as control-group rates of employment were higher among parents in center-based programs than they were for parents in the other two program approaches. The lack of any favorable impact on employment among parents in home-based programs may reflect a greater focus on education activities, as impacts were greatest in this area among home-based programs, particularly in the second year after enrollment.
4. Exploring the Relationships Between Parenting Impacts When Children Were 2 and Child Impacts When Children Were 3 by Program Approach
Early Head Start programs that chose different approaches to service delivery typically also had different theories of change regarding how the program would intervene in children’s lives.3 Center-based programs, which offered center-based child development services as well as parent education, expected changes in children’s development to occur mainly through the direct services, with a lesser impact of the program occurring through changes in parenting. Home-based programs focused child development services on both the child and the parent, and these programs expected changes in children’s development to occur mainly through changes in parenting. Mixed-approach programs, which blended center-based and home-based services in different patterns, varied in terms of the extent to which they expected program effects on children to be mediated by impacts on parents. To explore whether the impacts on parenting when children were 2 years old and on children’s development when they were 3 years old are consistent with the program-specific theories of change, we estimated mediated models by program approach that were similar to those estimated for the full sample (discussed in Chapter V and Appendix D.9).4
The results of estimating the mediated models for center-based programs are consistent with our expectations. The estimates suggest that impacts on parenting behavior when children were 2 are related to the impacts on child outcomes at age 3 in the expected directions, but the implied pathway for program impacts through parenting behavior to children appears to be fairly weak, in part because few of the parenting influences were affected by the program in the earlier period. We were able to estimate models of cognitive and language development and aggressive behavior only for children in center-based programs, because most or all of the parenting mediators were not affected by Early Head Start in the earlier period.
For home-based programs, the estimated relationships between impacts on parenting behavior when children were 2 years old and impacts on children’s outcomes when they were 3 years old were consistently in the expected directions. Although there was only one statistically significant child outcome among home-based programs when children were 3, the impacts that were not statistically significant were favorable and allowed for successful completion of the mediated analyses. Impacts on supportiveness, cognitive stimulation, and language support when children were 2 years old were all positively related to impacts on cognitive and language development and positive aspects of social-emotional development and inversely related to later impacts on negative aspects of social-emotional development when children were 3 years old. Earlier impacts on intrusiveness, detachment, and parental distress were all inversely related to later impacts on positive aspects of social-emotional development and positively related to later impacts on negative aspects of social-emotional development. Overall, the estimates suggest that part of the Early Head Start impacts on the cognitive, language, and socio-emotional development of children at age 3 in home-based programs could have emerged because of earlier impacts on related parenting behavior.
For mixed-approach programs, the estimated relationships between impacts on parenting behavior when children were 2 years old and impacts on child outcomes a year later were nearly all in the expected directions. Overall, the estimates are consistent with the theory that part of the Early Head Start impact on children’s outcomes at age 3 may be mediated by earlier impacts on parenting behavior.
5. Understanding Program Services and Their Impacts
Across all of the program approaches, Early Head Start had favorable impacts on children’s cognitive and language development, on levels of aggression, and on behavior in relation to the parent during semistructured play. Nevertheless, the pattern of impacts on children and parents varied to some degree across program approaches, reflecting in part differences in theories of change and impacts on service use, as well as differences in the characteristics of the populations they served.
Mixed-approach programs appear to have had the broadest pattern of favorable impacts on children and families, with many effect sizes in the 20 to 30 percent range. They had greater impacts on children’s language development and on positive aspects of social-emotional development. The mixed-approach programs also had statistically significant, favorable impacts on a wider range of parenting behaviors when children were 3 years old, including emotional support, support for children’s cognitive and language development, insensitivity, and use of punitive discipline strategies. They also appear to have had larger positive impacts on participation in education and training programs and in the final quarters of follow-up, employment.
Center-based programs appear to have had greater favorable impacts on the cognitive development of children with mild delays and on negative aspects of children’s social-emotional development. Parents in center-based programs tended to be more emotionally supportive, provide more support for children’s cognitive and language development, and use less punitive discipline strategies than similar parents in the control group. These parents reported a higher incidence of severe depressive symptoms than parents in the control group. Perhaps because parents applying to center-based programs were already planning to work or attend school, there were few statistically significant program impacts on participation in education and training activities or on employment, although the pattern of impacts was favorable.
Fewer statistically significant impacts were found for children and families in home-based programs when children were 3 years old, which suggests some fade-out of impacts on children’s language development and parents’ support for language and learning that were found when children were 2 years old. At age 3, children were more engaging of the parent in semistructured play and parents showed more supportiveness during the same parent-child play than control group children, but no other impacts on children or parents were large enough to reach statistical significance. Parents in these programs reported lower levels of parental distress than their control-group counterparts.
The different patterns of impacts by program approach may partly relate to different durations of program participation. Parents in mixed-approach programs tended to continue participating in the program for longer periods than did parents in either center-based or home-based programs, and this may have contributed to the somewhat stronger pattern of impacts found at age 3. The differences in duration of program participation by program approach, in turn, could have been influenced by any number of family characteristics, but could also relate to differences in the programs’ abilities to flexibly respond to the changing needs of families as their children moved through infancy and toddlerhood and the parents’ school or job opportunities changed.
B. HOW CHILD DEVELOPMENT, PARENTING, AND FAMILY WELL-BEING IMPACTS VARIED BY PATTERNS OF IMPLEMENTATION
The 17 programs varied in their patterns of implementing key elements of the Head Start Program Performance Standards pertaining to the quantity and quality of services, based on ratings that were developed for the implementation study.5 As summarized in Chapter I and reported more fully in Pathways to Quality (Administration on Children, Youth, and Families 2002), six programs were rated as fully implemented in fall 1997 (early implementers), six were not rated as fully implemented in fall 1997 but were rated as fully implemented in fall 1999 (later implementers), and five were not rated as fully implemented in either time period (incomplete implementers). The incomplete implementers either emphasized family support (with less emphasis on child development) or faced difficult implementation challenges (such as early staff turnover in leadership positions or partnerships that did not work out well).
We expected early implementers to have stronger and more enduring impacts than later implementers or incomplete implementers. Information about receipt of Early Head Start services (discussed in Chapters III and IV) shows that the impacts on receipt of any core child development services and any home visits were largest for programs that were implemented early and smallest for incomplete implementers. Similarly, the impacts on receipt of core child development services at the required intensity and weekly home visits followed the same pattern.
Because differences in impacts on service receipt correspond to the pattern of implementation in predictable ways, we expected that the program impacts on children and families would also vary according to the pattern of implementation. In particular, we expected that programs that had met the performance standards by a point soon after families enrolled, and sustained full implementation over most of the period that families participated in the program, would have the strongest and most enduring impacts on families and children. Programs that became fully implemented later were expected to have weaker impacts than early implementers, and incomplete implementers were expected to have weaker impacts than later implementers.
When children were 2 years old, the early implementers had a stronger pattern of impacts on child and family outcomes than later and incomplete implementers. By the 3-year assessment point, however, differences in impacts on children’s development and parenting by implementation pattern were less distinct. All three categories of programs had some important impacts when children were 3 years old, but the early and later implementers favorably influenced a broader range of child development and parenting outcomes. This pattern suggests that some experience in a fully-implemented program, even when it occurs later in the families’ enrollment period, is sufficient to provide benefits in terms of child development and parenting outcomes (even in the later implementers the families experienced one year or more of full implementation). At the same time, it is notable that early-implemented programs also favorably influenced parents’ mental health and self-sufficiency.6
Even if the program is not fully implemented overall, fully implementing some key services can make a difference for families and children. Incomplete implementers, many of which had strong family support components, had impacts on self-sufficiency, mental health, and social-emotional aspects of parenting and children’s development. Nevertheless, with child development services that did not meet some key program performance standards, these programs had no significant impacts on children’s cognitive or language development or on parents’ support for children’s cognitive and language development.
It is important to consider that factors other than implementation pattern might also contribute to the differences in impacts for these subgroups. For example, differences in program approaches or family characteristics might be confounded with implementation pattern, as home-based programs seem to have faced more challenges meeting the performance standards than did the other program approaches (ACYF 2002). Within the home-based and mixed-approach programs, it was possible to examine differences in impacts by implementation pattern while holding program approach constant. The results of these analyses provide evidence that fully implementing the performance standards makes a difference.
The following subsections describe the patterns of impacts by pattern of implementation in the areas of child development, parenting, and mental health and self-sufficiency. Then, we present the differences in impacts by implementation pattern when holding program approach constant and discuss the implications of these findings.
1. Child Development
When children were 3, Early Head Start improved a range of child development outcomes; in many cases, these impacts did not differ significantly among the three program groups defined by pattern of implementation (see Table VI.5). Early Head Start had a favorable impact on children’s cognitive development among both early and later implementers. Both early and later implementers increased average Bayley MDI scores significantly. The impacts on the percentage of children who scored below 85 (one standard deviation below the average score) were also favorable for children in the early and later-implemented programs, although they were not statistically significant (but the reduction in the percentage below 85 was significant in the overall analysis—see Chapter V).
Early Head Start had a positive impact on the language development of children overall. Program impacts on children’s average PPVT-III scores were favorable for Early Head Start programs in all three implementation categories and statistically significant among the later implementers. The favorable impacts on the percentage of children with PPVT-III scores below 85 were somewhat larger among early implementers.
Early Head Start programs in all three implementation categories enhanced positive aspects of children’s social-emotional behavior, but the pattern of impacts appears particularly strong among incomplete implementers. Early Head Start children in incompletely implemented programs showed significantly greater levels of engagement of the parent in semistructured play and attention to objects during play compared with their control-group counterparts. Impacts on engagement of the parent during semistructured play were also statistically significant for children in later-implemented programs.
According to the analysis of impacts on the full sample (Chapter V), Early Head Start programs had favorable impacts on children’s aggressive behavior and negativity toward the parent during semistructured play among all three groups of programs defined by the level and timing of implementation, and the differences in impacts across groups were not statistically significant. The pattern of statistically significant impacts within implementation groups was mixed, however. The favorable impact on parent-reported levels of aggressive behavior was statistically significant among children in incompletely implemented programs, but not in the other two groups. The favorable impact on negativity toward the parent during semistructured play was statistically significant among children in early-implemented programs, but not for the other two implementation groups.
When children were 2 years old, the impacts on children’s development were more strongly associated with early-implemented programs. The pattern of impacts across implementation subgroups found when children were 3 years old likely reflects, at least in part, the greater time separation of the implementation measures and the child assessment measures for many families and the fact that most programs in all three groups continued improving services over time.
2. Parenting
When children were 3, Early Head Start impacts on parenting behavior and knowledge were mainly concentrated in early- and later-implemented programs. Very few significant impacts emerged among parents in incompletely implemented programs (Table VI.6). A year earlier, the strongest impacts on parenting behavior and knowledge were concentrated among the early implementers.
At the 3-year-old assessment point, Early Head Start had a favorable impact overall on the cognitive stimulation and emotional support in the home, measured by total HOME scores, but impacts on total HOME scores were statistically significant only among the early implementers. Impacts on the physical environment of the home were not significant for any of the three implementation groups.
When children were 3, Early Head Start had important impacts on aspects of emotional support among parents in all three groups of programs classified by implementation pattern. The Early Head Start impact on parents’ warmth toward the child as rated by the interviewer during the home visit was favorable and statistically significant among parents in early-implemented programs. Impacts on parent supportiveness during semistructured play were statistically significant in later-implemented and incompletely implemented programs. The impacts on supportive presence during the puzzle challenge task were not large enough to reach statistical significance in any of the subgroups.
Early Head Start had positive impacts on several aspects of stimulation of language learning among parents in early-implemented and later-implemented programs, but not among parents in incompletely implemented programs. Early Head Start impacts on parent-child play and reading to children daily were positive and statistically significant among parents in early-implemented programs. Impacts on quality of assistance in the puzzle challenge task and support for language and learning were statistically significant among parents in later-implemented programs. Later-implemented programs also had a favorable impact on parents’ regular reading to the child at bedtime. When children were 2 years old, Early Head Start impacts on parent stimulation of children’s language and learning were concentrated among the early implementers.
Early Head Start programs that were implemented later had statistically significant impacts on several negative parenting behaviors. Parents in later-implemented programs were less likely to be detached during semistructured play and were less likely to be intrusive during the puzzle challenge task compared with their control-group counterparts. However, impacts on hostility and punishment were mixed for parents in later-implemented programs. Compared with control-group parents, Early Head Start parents were more harsh toward the child during the interview, as rated in the interviewer observation, although average levels of harshness were very low for both groups.7 Early Head Start had no impact on negative regard toward the child during semistructured play (and average levels were low for both groups, as scores range from 1 to 7). Significantly fewer Early Head Start parents reported that they spanked the child in the previous week, and parents were more likely to suggest mild, less punitive discipline strategies in response to common parent-child conflict situations compared to their control-group counterparts. It is possible that the later-implemented programs increased knowledge about the adverse effects of punitive parenting practices without making significant changes in behavior.
Early-implemented programs had significant impacts on punishment and discipline strategies. Although Early Head Start programs in all three implementation groups tended to reduce the incidence of physical punishment, parents in early-implemented programs also were significantly more likely than their control-group counterparts to suggest using mild and non-punitive discipline strategies in response to common parent-child conflict situations. Parents in early-implemented programs were significantly less likely than their control-group counterparts to suggest using physical punishment as a discipline strategy.
In summary, both early- and later-implemented programs had favorable impacts across several domains of parenting. In particular, these programs increased emotional support of the child, increased support for child language and cognitive development, and reduced negative parenting behaviors. The impacts across several domains of parenting may partly explain the favorable impacts on children’s cognitive and language development and certain behavioral outcomes among these programs. In addition, Early Head Start programs that were incompletely implemented had a favorable impact on supportive behavior during play and tended to reduce the incidence of physical punishment. These impacts on emotional support and physical punishment could partly explain the favorable impacts on children’s behavioral outcomes among these programs.
3. Parent Mental Health and Self-Sufficiency
At the 3-year-old assessment point, some impacts on parent mental health emerged in the early-implemented and incompletely implemented programs (Table VI.7). Parents in Early Head Start programs that were not completely implemented reported significantly lower levels of parental distress compared with their control-group counterparts. Early-implemented Early Head Start programs significantly lowered average levels of depressive symptoms reported by parents, consistent with the reduction in the probability of depression found among these programs when children were 2 years old.
Impact education, and training) tended to be greatest for parents in early-implemented and incompletely implemented programs (Table VI.8). Impacts on education or training activities were favorable for all three groups of programs classified by implementation status, but the impacts were significantly larger among parents in incompletely implemented programs. Although parents in incompletely implemented programs participated in vocational education programs at higher levels than they participated in high school programs, the incompletely implemented Early Head Start programs had greater impacts on high school attendance, nearly doubling participation. Impacts on quarterly rates of participation in education and training were favorable and significant for parents in incompletely implemented programs from the third through the eighth quarter after enrollment. Impacts on quarterly rates of participation in education and training activities tended to be favorable in the other two implementation groups, but did not reach statistical significance.
Impacts on employment were positive and significant for early-implemented programs, and the impact on the employment rate during the first two years after enrollment was positive among parents in incompletely implemented programs. Impacts on quarterly employment rates among parents in early-implemented programs were statistically significant in the fourth through sixth quarters after enrollment, but they were not significant in any quarter among parents in later- or incompletely implemented programs.
4. The Importance of Implementation
The impacts of Early Head Start on 3-year-old children and their parents suggest that fully implementing the performance standards is important. By the time children reached 3 years of age, however, early implementation of the performance standards appears to have been less important, as families in later-implemented programs received fully implemented services for a year or longer and experienced a stronger pattern of impacts than they did when the children were 2. It appears that some significant experience with a fully-implemented program may be sufficient to generate positive outcomes for children and families. It is also possible that other factors contributed to the pattern of impacts we have described here. Home-based programs were challenging to implement, and as a consequence, only one of the seven was implemented early. Thus, the pattern of impacts by program implementation could be partly attributable to differences in impacts by program approach. To explore the potential confounding of implementation pattern and approach to service delivery, we examined the patterns of impacts by program implementation separately within two of the program approach subgroups. Within the home-based and mixed-approach programs, it was possible to examine differences in impacts by implementation pattern while holding program approach constant.8 The results provide evidence that fully implementing the performance standards makes a difference.
Home-based programs had fewer impacts overall, but the four early/later implementers had significant favorable impacts on children’s cognitive and language development, parental distress, and reported spanking in the past week (Tables VI.9 and VI.10). The three incompletely implemented home-based programs had significant favorable impacts only on two aspects of children’s social-emotional development (sustained attention and engagement of parent in the play task) and parents’ participation in education and training activities. These impacts may reflect the strong family support components of some of the incompletely implemented programs that encountered challenges in implementing the Early Head Start child development requirements.
We also examined impacts within the mixed-approach programs. Six programs could be divided into three that were fully implemented early and three that were implemented later or incompletely. Early Head Start mixed-approach programs that were implemented early had stronger impacts than incompletely implemented programs across a broad range of outcomes, with effect sizes in the 20 to 50 percent range. These early-implemented mixed-approach programs had stronger impacts on children’s cognitive and social-emotional development than late or incompletely implemented programs (Table VI.11). Although the impact of the early-implemented programs on the average PPVT-III score appears to be smaller than that of the later and incompletely implemented programs, the early implementers significantly reduced the proportion of children scoring below 85, while the later and incompletely implemented programs did not.
With the exception of parental detachment during play, impacts on parenting tended to be stronger for the early-implemented programs, including the impacts on supportive presence in the puzzle challenge task and the percentage of parents reading daily and at bedtime to their children. Impacts on parents’ mental health, including symptoms of depression and dysfunctional interaction, tended to be more favorable among early implementers. The only significant impact was an increase in dysfunctional interaction among the late/incomplete implementers. Both groups of programs increased parents’ participation in education programs and in employment activities, although the employment impacts tended to be larger and were statistically significant for parents in the early-implemented programs (Table VI.12).
C. SUMMARY AND CONCLUSIONS
Two program features appear to be important for understanding the impacts of Early Head Start on the services families receive and on the ways in which programs influence children’s development, parenting behavior, parents’ mental health, and self-sufficiency. These features—the program’s approach to serving families and its pattern of implementing key performance standards—were associated with differences in impacts on the receipt of services and on child and family outcomes.
When children were 3 years old, we found that favorable impacts on children’s development, parenting behavior, and self-sufficiency appeared to be more numerous and stronger for mixed-approach programs, but center-based programs also had favorable impacts on a range of child development and parenting outcomes. At the same time, the findings were not completely favorable for parents in center-based programs, as some of those parents experienced symptoms of more-severe depression than their control-group counterparts. Home-based programs had few significant impacts.
These variations could be attributable in part to different durations of program participation. Families continued to participate in mixed-approach programs for a longer period, on average, than was true for families in center-based or home-based programs. Differences in length of participation, which may be attributable to the mixed-approach programs’ greater flexibility in providing services as family needs changed, could have enabled families to make stronger and more sustained progress.
When programs are grouped by pattern of implementation, we found that while all three categories of programs had some important impacts at the 3-year assessment point, the early and later implementers favorably influenced a broader range of outcomes. By the time children were 3 years old, the later-implemented programs appear to have “caught up” with the early implementers in terms of their impacts on a broad range of important child development and parenting outcomes. This pattern suggests that a year or more of experience in a fully-implemented program, even when it occurs later in the families’ enrollment period, provides benefits in terms of child and family outcomes.
Early-implemented programs had some impacts that were not found in the other groups. In addition to impacts on children’s development and parenting, early-implemented programs had favorable impacts on parents’ self-reported symptoms of depression, participation in education activities, and employment, areas that can take time for programs to influence and which later-implemented programs did not change. The findings suggest that early-implemented programs were able to move beyond influencing just child development and parenting support to also have an impact on family development, including self-sufficiency and mental health.
Our findings also suggest that fully implementing some, but not all, key services can make a difference for families and children. Incomplete implementers had favorable impacts on mental health (parental distress) and on participation in education and training programs. Many of these programs had strong family support components but did not meet some key performance standards for child development services. Thus, the fact that the incomplete implementers had an impact on mental health and self-sufficiency that was similar to those of the early implementers is consistent with what we know about features of the programs. Incomplete implementers had little impact on parenting behavior, although supportiveness in play was enhanced. Incomplete implementers reduced aggressive behavior and improved several aspects of child behavior in relation to the parent. Thus, the impacts on parents and children tended to be in the social-emotional, rather than the cognitive domains, which could reflect the programs’ greater focus on family support relative to child development. In contrast, the early-implemented programs had significant impacts on a broad range of outcomes, including child cognitive, language and social-emotional development; parenting behavior; parent mental health and self-sufficiency.
While it is not possible to fully disentangle the effects of program approach and program implementation, analyses of impacts by pattern of implementation within the home-based and mixed-approach programs provide additional evidence that reaching full implementation contributes to a stronger pattern of impacts. Home-based programs that were fully implemented early or later had some favorable impacts on child cognitive and language development, impacts that are not often found in home-based program evaluations. Mixed-approach programs that were fully implemented early produced a stronger pattern of impacts (and some of the largest impacts detected in the study) compared with those that were not fully implemented early.
1As we stated in Chapter I, programs that primarily offer services to families through the home-based option, for purposes of discussion, are called “home-based programs” in this report. Those offering services to families through the center-based option are referred to as “center-based programs” for this report, and those programs that serve families through various combinations of home- and center-based options are referred to as “mixed-approach programs” in this report.(back)
2When children were 2 years old, we measured depression using the Composite International Diagnostic Interview (CIDI) – Short Form - Major Depression (Nelson et al. 1998), from which a probability of clinical depression can be derived. When children were 3 years old, we used the short form of the Center for Epidemiological Studies – Depression (CES-D) scale (Radloff et al. 1977; Ross et al. 1983), which measures depressive symptoms and uses cutoff points to indicate a high probability of clinical depression. Although several of the symptom questions are similar, the reporting period differs (CIDI asks about the past year and CES-D asks about the previous week). The two measures could thus classify the same individual differently.(back)
3See Pathways to Quality (ACYF, 2002) for a full presentation of how Early Head Start research programs’ theories of change were assessed.(back)
4To avoid an overly technical presentation, this section summarizes the results of our analysis of the role of parenting impacts “mediators” when children were 2 years old in relation to the child impacts we observed when children were 3 years old. The methodology of these analyses and the details of the results are presented in Appendix D.9.(back)
5The Head Start Program Performance Standards specify performance criteria that are based on research and consensus from the field about what constitutes high-quality, comprehensive services.(back)
6We also conducted analyses focusing on the programs that achieved strong full implementation of child and family development services. These analyses are discussed in Chapter II and results are presented in Appendix Table E.VI.9. They show that the four strong fully implemented programs had a stronger pattern of impacts on child and parenting outcomes than the other programs. (back)
7As discussed in Chapter V, harshness measures whether the parent scolded the child, physically restrained the child, or slapped or spanked the child during the interview. Scores can range from 0, if no harsh behavior was observed, to 3, if all three types of behavior were observed.(back)
8We were unable to examine differences in implementation within the center-based programs because the sample included only four center-based programs. The analysis of implementation within the home-based and mixed-approach programs required dividing programs differently by implementation pattern in order to form subgroups of sufficient size for the analysis. Thus, within home-based programs, we compared early and later-implemented programs with the incompletely implemented ones; within mixed programs, we compared early-implemented programs with those that were implemented either later or incompletely.(back)
| Table of Contents | Previous | Next |