Table of Contents | Previous | Next |
Chapter 5: Impact of Head Start on Children’s Cognitive Development
Highlights
Head Start was found to have a positive impact on the cognitive development of children in both the 3- and 4-year-old age groups, with children in the Head Start group being more advanced by spring 2003 than non-Head Start children. However, the magnitude of the statistically significant and positive cognitive impact of Head Start, while meaningful in some skill areas, was small to modest, with the magnitude of estimated impacts varying across skill areas. Specifically:
-
The largest impacts were found for direct assessments of pre-reading skills (19 to 24 percent of a standard deviation) and parent perceptions of children’s emergent literacy skills (effect sizes of 29 to 34 percent of a standard deviation).
-
Relatively small impacts were found for the direct assessments of pre-writing skills (13 to 16 percent of a standard deviation) and vocabulary knowledge (effect sizes of 10 to 12 percent of a standard deviation).
-
No overall significant impact was found in the skill areas of oral comprehension, phonological awareness, and early math skills. In the latter areas, however, some demographic subgroups showed significant impacts.
-
The evidence of beneficial cognitive impacts of Head Start was more widespread and consistent among 3-year-old children than among 4-year-olds.1
-
Positive cognitive impacts of Head Start were found for children from English-speaking families and to a more limited degree for children from Spanish-speaking families (i.e., children whose home language is Spanish, excluding study children in Puerto Rico). However, tests of the interaction between language and program impact failed to reach statistical significance.
-
There was substantial evidence that children in the 3-year-old group whose primary caregivers reported high levels of depressive symptoms at baseline showed less benefit of Head Start on their cognitive development than children whose mothers reported lower or no evidence of depressive symptoms.
-
Evidence that Head Start has positive benefits is particularly strong for Hispanic and African American children.
Organization and Presentation of Findings
This chapter focuses on the impact of Head Start on six different constructs that make up the cognitive domain:
- Pre-reading skills;
- Pre-writing skills;
- Vocabulary knowledge;
- Oral comprehension and phonological awareness;
- Early math skills; and
- Parent’s perceptions of their child’s early language and literacy skills.
The discussion that follows focuses on an examination of statistically significant “intent-to-treat” impact estimates. This involves using the complete sample of children who were randomly assigned in 2002 (see Chapter 4), measuring the average impact of access to Head Start. The discussion begins with a review of the overall average impacts for all newly entering children in the 3- and 4-year-old groups respectively. Then it examines impacts for subgroups of children defined by the language used for the child assessment (i.e., children who were assessed in English in both fall 2002 and spring 2003, and children who were assessed in Spanish in fall 2002 and in English in spring 2003). Following the review of overall average effects, the discussion examines the extent to which impacts occurred for key subgroups of Head Start children and how different in size impacts may have been for various subgroups.
The estimated impacts on program participants (i.e., referred to as the “impact on the treated”) are presented in Appendix 5.1, focusing primarily on the extent of any differences from the intent-to-treat estimates. For clarity, the discussion in Appendix 5.1 examines only the combined group of all children (i.e., separate breakdowns by the language of assessment subgroups are not provided).
The statistical results for the discussion in this chapter are presented in a series of tables provided at the end of the chapter, plus additional tables in Appendix 5.2. Exhibits 5.1-A through 5.1-C (for children in the 3-year-old group) and 5.2-A through 5.2-C (for the 4-year-old group), present the overall average impact estimates for the combined sample and for the two separate language groups. The data in these tables are presented for individual measures (e.g., Woodcock-Johnson III Letter-Word Identification) in three ways: (1) as simple mean differences, (2) using regression analyses that include only demographic covariates measured in fall 2002, (3) and using regression analyses that add a measure of the outcome variable assessed in fall 2002.
For the latter two regression-based estimates, the shaded columns in the tables show statistically significant results that are discussed in the text. As discussed in Chapter 4, impact estimates are based on the regression model that includes family demographic characteristics and the fall measure of the parent outcome as covariates only in instances where analyses show the fall measure could not be substantially influenced by early exposure to Head Start. In those instances where such early substantial influence on the fall measure cannot be ruled out, the regression-adjusted impact estimates use only the family demographic characteristics as covariates. However, because inclusion of the fall outcome measure has other benefits as discussed in Chapter 4 (e.g., increased statistical precision, reduction in potential spring 2003 nonresponse bias) that may be considered more important to some than the removal of some of Head Start’s impacts from the spring estimate, the analysis that controls for the fall outcome is provided as well. Conversely, since some risk of a small amount of impact removal exists any time the fall outcome measure enters the analysis, estimates based solely on demographic variables are also presented, although they are not always highlighted in the discussion.
Exhibits 5.3-A and 5.3-B (for the 3- and 4-year-old groups, respectively) summarize all of the statistically significant average impacts both for the overall group and for a set of 11 subgroups identified for examination because of their special program or child development importance as discussed in Chapter 4. Subgroup results are shown first for estimated differences in impacts between subgroups (e.g., boys versus girls) and then as impacts on individual subgroups (e.g., impacts on boys alone). Both perspectives are important, as discussed in Chapter 4: differences in the size of impacts indicate the types of children and families not benefiting as much as others from Head Start participation; impacts on individual subgroups show where any overall gains from the program are occurring and whether those gains are widespread (as opposed to concentrated among certain segments of the Head Start population). These tables present two columns of figures: the estimated impacts and the estimates expressed as “effect sizes” (i.e., the impact estimates divided by the standard deviation of the outcome measure in the population). Effect sizes provide a yardstick for gauging the quantitative importance of a measured impact in relation to the natural variation of the child or family outcome Head Start is seeking to affect.2 Effect sizes are important in interpreting the size of Head Start’s measured impact and, in particular, how much larger that impact may be for the average program participant as opposed to the larger group of children and families accorded access to the program (some of whom do not participate in the program).
Finally, Exhibits 5.4 through 5.25, provided in Appendix 5.2, show the results of all moderator/subgroup analyses, including those that do not produce statistically significant impacts, with a separate table for each individual measure of cognitive outcomes. Again, for clarity, these results are only presented for the full combined sample (i.e., not separately for the English-English and Spanish-English language groups).
Estimated Impact of Access to Head Start
This first section discusses the estimated impact of Head Start on cognitive outcomes using the sample of children randomly assigned to either Head Start or to the non-Head Start group, referred to as “intent-to-treat” impact estimates. These estimates show the effect of Head Start on the average child given access to the program.
Impact on Pre-Reading Skills
Overall, there were significant impacts on the Pre-Reading Skills of children in both the 3- and 4-year-old groups, with the skills of children in the Head Start group being more advanced than those of non-Head Start group children. Significant differences were found both in children’s performances on the Woodcock-Johnson III Letter-Word Identification test and on the Letter Naming task. Both of these tests tap letter recognition skills that are important steps toward becoming a proficient reader and are predictive of how well children are reading at the end of kindergarten and 1st grade.
As shown in Exhibit 5.1-A, among children in the 3-year-old group from all language backgrounds, the IRT scale score on the Woodcock-Johnson III Letter-Word Identification test was 5.65 points higher for the Head Start group than for the non-Head Start group, by the end of the program year. By the same time, the Head Start group children could also identify an average of 1.3 more letters than children in the non-Head Start group (the latter could identify an average of 3.8 letters).
Among children in the 4-year-old group from all language backgrounds (Exhibit 5.2-A), the IRT scale score on the Woodcock-Johnson III Letter-Word Identification test was 5.74 points higher for the Head Start group than for the non-Head Start group. By the same time, the Head Start group children could identify an average of 2.3 more letters than children in the non-Head Start group (the latter could identify an average of 9.2 letters).
The magnitude of the Head Start impact on children’s Pre-Reading Skills was modest but meaningful. The effect size of the impacts on Letter-Word Identification test scores were 24 percent of a standard deviation for children in the 3-year-old group and 22 percent for children in the 4-year-old group (see Exhibit 5.3-A and 5.3-B). The effect sizes of the impacts on Letter Naming task performance were 19 percent for children in the 3-year-old group and 24 percent for the 4-year-old group.
The availability of publisher norming data for the Woodcock-Johnson III Letter-Word Identification test permits comparisons of the skill levels of children in the Head Start Impact Study with those of the general population of 3- and 4-year-olds in the US (including those who were not from low-income families). These comparisons showed that, at the end of the program year, the mean performance of Head Start children was still below average performance levels for all U.S. children, by about one-third of a standard deviation. But, by comparing their performance in spring 2003 to the standard scores of children in the non-Head Start group, it appears that Head Start serves to narrow the gap between the skills of Head Start children and the skills of the general population of young children by about 45 percent.
This was determined by estimating the mean standard score on the Woodcock-Johnson III Letter-Word Identification test for children in the 3-year-old Head Start group, which was calculated to be equal to 96.0 in the spring of the program year.3 The comparable mean standard score for children in the non-Head Start group was estimated to be equal to 92.4. The mean standard score for all U.S. 3-year-olds is 100.0, with a standard deviation of 15. Therefore, the gap between the average score of the 3-year-old Head Start group and the overall national average score was 4 standard-score points; whereas, the gap between the average for the non-Head Start group and the national norm was 7.6 points. Hence, the Head Start group had a deficit that was smaller by 3.6 points, or 47 percent (3.6/7.6 = 0.47). Similarly, for 4-year-olds, the mean standard score on the Letter-Word Identification test for the Head Start group was 95.2 in the spring of the program year; whereas, the mean for 4-year-olds in the non-Head Start group was 91.3. Again, the mean standard score for all U.S. 4-year-olds is 100.0, with a standard deviation of 15. Consequently, the gap between the average score of the 4-year-old Head Start group and the overall national average score was 4.8 standard-score points, and the gap between the average for the non-Head Start group and the national norm was 8.7 points. Hence, the Head Start group had a deficit that was smaller by 3.9 points, or 45 percent (3.9/8.7 = 0.45).
Impact on Pre-Writing Skills
The pre-writing skills of both 3- and 4-year-old children in the Head Start group were slightly more advanced than those of the non-Head Start group children in the same age group. Pre-writing skills were measured by two tests: the McCarthy Draw-A-Design test, which measures perceptual-motor skills involved in seeing and copying basic geometric shapes, and the Woodcock-Johnson III Spelling test, which first measures perceptual-motor skills involved in tracing or copying letter shapes and then measures children’s ability to draw letters on request, without being shown the shape of the letter in question. Significant differences were found both in children’s performances on the Woodcock-Johnson III Spelling test and on the McCarthy Draw-A-Design test. Both of these tests tap skills that are necessary for writing words and sentences and are also predictive of how well children are reading at the end of kindergarten and 1st grade.
Among children in the 3-year-old group, a statistically significant difference in average scores was found on the Draw-A-Design test between the Head Start group and the non-Head Start group children. However, no significant difference was found for children in the 3-year-old group on the (more advanced) Spelling test. On the other hand, for children in the 4-year-old group, a significant difference in average scores was found on the Spelling test between children in the Head Start and non-Head Start groups, but no statistically significant difference was found for these older children on the (more basic) Draw-A-Design test.
As shown in Exhibit 5.1-A, among children in the 3-year-old group from all language backgrounds, the score on the McCarthy Draw-A-Design test was 0.15 points higher for the Head Start group than for the non-Head Start group. However, the IRT scale score on the Woodcock-Johnson III Spelling test for children in the 3-year-old Head Start group was not significantly different from that of children in the non-Head Start group. For children from all language backgrounds in the 4-year-old group (see Exhibit 5.2-A), the IRT scale score on the Woodcock-Johnson III Spelling test was 4.14 points higher for the Head Start group than for the non-Head Start group. Mean scores of the two groups of children in the 4-year-old group on the McCarthy Draw-A-Design test were not significantly different.
The magnitude of the Head Start impact on children’s Pre-Writing Skills, though statistically significant, was relatively small. The effect size of the impact on McCarthy Draw-A-Design test scores of children in the 3-year-old group was 13 percent, and the effect size of the impact on the Woodcock-Johnson III Spelling test scores of children in the 4-year-old group was 16 percent of a standard deviation.
As above, the availability of publisher norming data for the Woodcock-Johnson III Spelling test allows a comparison of the skill levels of children in the Head Start Impact Study with those of the general population of 3- and 4-year-olds in the US (including those who were not from low-income families). These comparisons showed that, at the end of the program year, the mean performance of Head Start 4-year-old children was still below average performance levels for all U.S. 4-year-olds by one-half of a standard deviation. At the same time, based on the mean standard scores of children in the non-Head Start group, it would appear that Head Start narrows the gap between the early writing skills of Head Start-eligible children and the skills of the general population of young children by 28 percent.4
Impacts on Vocabulary Knowledge
Among children in the 3-year-old group, the Vocabulary Knowledge of children in the Head Start group was slightly more advanced than that of the non-Head Start children in the same age group. Among children in the 4-year-old group, only Head Start children from Spanish-speaking families showed vocabulary knowledge significantly greater than that of the non-Head Start children. Vocabulary knowledge was measured by two tests: the Peabody Picture Vocabulary Test, Third Edition (PPVT-III) (adapted), which measures children’s receptive vocabulary by asking them to select one of four pictures that best represents each of a series of words spoken by the examiner, and a Color Naming test that measures children’s ability to name the colors of drawings of bears in 10 different colors. Both of these tests tap skills that are indicative of children’s oral language development. Vocabulary tests are strongly predictive of children’s general knowledge at the end of kindergarten and 1st grade. Vocabulary is also predictive of later reading proficiency as children move from the basic decoding stages of reading to more advanced literacy that involves relating what is read to information already acquired about the outside world.
As shown in Exhibit 5.1-A, among children in the 3-year-old group from all language backgrounds, the IRT scale score on the PPVT-III (adapted) was 4.23 points higher for the Head Start group than for the non-Head Start group, and children in the Head Start group could name about one more color than children in the non-Head Start group (the estimated impact is 0.70). As shown in Exhibit 5.2-A, among children in the 4-year-old group from all home-language backgrounds, the IRT scale score on the PPVT-III (adapted) was not significantly different from that of children in the non-Head Start group, nor was there an overall average impact found for these children on the Color Naming test. The magnitude of the Head Start impact on children’s Vocabulary Knowledge, though statistically significant, was relatively small. The effect size of the impact on PPVT-III (adapted) scores of children in the 3-year-old group was 12 percent of a standard deviation and about 10 percent for the Color Naming task.
Again, using publisher norming data for the PPVT-III shows that Head Start children were 8 percent5 closer to the national norm on vocabulary knowledge than non-Head Start children, but only for children in the 3-year-old group.
Impacts on Oral Comprehension and Phonological Awareness
Two other skill areas that have been shown to relate to children’s emergent literacy and later academic achievement are oral comprehension and phonological awareness. Oral comprehension is the child’s ability to understand and make inferences from spoken phrases and sentences. In the Head Start Impact Study, this skill was measured by the Woodcock-Johnson III Oral Comprehension test. The test consists of a series of incomplete sentences spoken to the child. The child is asked to “fill in the blank” in each sentence based on contextual cues contained in the sentence and his or her prior knowledge of common phrases.
Phonological awareness is a child’s understanding that spoken sentences are made of component words, compound words are made up of simpler words, and that even simple words are made up of component syllables and sounds (phonemes). The skill also involves understanding that when a component sound is added to or deleted from a word, the meaning of the resulting word is often different from that of its unaltered predecessor. In the Head Start Impact Study, phonological awareness was measured by the Elision task from the Comprehensive Test of Print and Phonological Processing – Preschool edition (CTOPPP). In this task, children were asked to identify the word that resulted when part of a word was deleted. The test first uses pictures to assist children in determining the right answer, but then progresses to asking about the results of word, syllable, or phoneme deletion without any picture assistance.
Neither children’s scores on the Woodcock-Johnson III Oral Comprehension test nor their scores on the CTOPPP Elision test showed any significant overall effect of access to Head Start (see Exhibits 5.1 and 5.2), i.e., the scores of children in the Head Start group did not differ significantly at the end of the program year from those of children in the non-Head Start group. This was the case among children both in the 3-year-old and 4 year-old groups, in all language groups combined, as well as among children from English-speaking families and Spanish-speaking families analyzed separately.
Impact on Early Math Skills
For the most part, the Early Math Skills of 3- and 4-year-old children did not show a significant impact resulting from access to Head Start. There were, however, some analytic subgroups that showed significant positive impacts of Head Start on early math achievement.
Math skills were measured with two tests: the Woodcock-Johnson III Applied Problems test and a rating scale that assessors used to evaluate how well children had done at one-to-one counting of a set of drawings of 10 bears (the same ones used in the Color Naming task). The Applied Problems test assesses children’s proficiency at solving simple word problems that involve counting, simple arithmetic, and basic measurement. Both of these tests assess basic skills and understandings that are essential for the development of more advanced quantitative capabilities and are predictive of mathematics achievement in kindergarten and 1st grade. The test scores are also predictive of later reading achievement because they rely on children’s comprehension of spoken questions and instructions and on children’s ability to make one-to-one associations between sounds and pictures or written symbols.
Although the mean scores on the Applied Problems test of children in both the 3- and 4-year-old Head Start groups were generally higher than those of non-Head Start children, these differences did not reach statistical significance. Differences in the ratings of how well children did on one-to-one Counting Bears were also generally small and not statistically reliable.
Impact on Parent Perceptions of Children’s Emerging Literacy Skills
Among children in both age groups, parental reports of children’s emergent literacy skills were higher for children in the Head Start group compared to those in the non-Head Start group. Parent perceptions of their children’s early academic skills were measured by the Parent-Reported Emergent Literacy Scale (PELS). This is a series of questions about how many letters of the alphabet the child knows, how many colors he or she can identify, how high he or she can count, whether the child can write his or her first name, etc. These questions were first developed for the 1993 National Household Education Survey on School Readiness, and resulting scale scores have been shown to correlate with children’s age and disability status, with socioeconomic family characteristics, and with other measures of children’s cognitive and social development.
As shown in Exhibit 5.1-A, among children in the 3-year-old group from all language backgrounds, the PELS scale score for children in the Head Start group was about 0.5 points higher than the scores for children in the non-Head Start group. Among children in the 4-year-old group from all language backgrounds (Exhibit 5.2-A), the PELS score for children in the Head Start group was 0.4 points higher. The magnitude of the Head Start impact on parent perceptions of children’s emerging literacy skills was moderate, with effect sizes of 34 percent of a standard deviation for 3-year-old group children and 29 percent of a standard deviation for 4-year-old group children.
Moderator/Subgroup Differences
The analysis of impacts by subgroups of children and families (detailed in Appendix 5.2 and summarized in Exhibits 5.3-A and 5.3-B for those found to be statistically significant) show some variations in impact for particular types of Head Start participants. The most important and consistent of these findings are discussed below in two sections. The first section looks at instances where a statistically significant difference in impact was found between particular subsets of children that were identified in advance as being of program or child development importance, e.g., larger or smaller impacts for special needs children as compared to children without special needs. The second section looks at statistically significant impacts on particular subsets of children, e.g., the impact on children with special needs. The former tells us whether one type of Head Start child or family is benefiting more than another, while the latter points out where impacts are occurring and whether they are widespread. The former “difference-indifferences” impacts are more difficult to detect (with a given sample size); relatively few such impact variations have been detected.
Differences in Impact
The subgroup factors will at times lead to discernibly different sizes of impact for one subpopulation than another, information that may be helpful for assessing and enhancing the program. In the cognitive domain, a statistically significant relationship was found between the primary caregiver’s reported level of depressive symptoms and the impact of Head Start across several of the cognitive outcome measures, but only for children in the 3-year-old group. Consistently, the impact of Head Start was found to decrease with increasing levels of caregiver’s reported depressive symptoms (see Exhibits 5.3-A and 5.3-B). The areas of development in which such interactions were found were vocabulary knowledge (on both PPVT-III adapted and Color Naming), early math skills (both Woodcock-Johnson III Applied Problems and Counting Bears), phonological awareness (CTOPPP Elision), and parent perceptions of children’s emerging literacy (the PELS measure).
Other statistically significant findings in Exhibits 5.3-A and 5.3-B are not discussed because it is possible they are due to chance alone and do not represent true impacts of the intervention (see discussion of subgroup impact analysis in Chapter 4).6
Impacts on Particular Subgroups
Child Language. This section summarizes information from both the language-group-specific analyses (Exhibits 5.1 and 5.2), and the analysis of subgroup impacts defined by language using the combined sample (Exhibits 5.3-A and 5.3-B). As shown in these tables, significant impacts of Head Start were found for English-speaking children in the areas of pre-reading, pre-writing, and vocabulary skills, but only in the area of vocabulary for Spanish-speaking children.7
With regard to pre-reading, English-speaking children in both age groups exhibited positive impacts on the Woodcock-Johnson III Letter-Word Identification test, and children in the 4-year-old group also had positive impacts on the Letter naming task. In the area of pre-writing skills, English-speaking children in the 3-year-old group had positive impacts on the easier Draw-a-Design test, while children in the 4-year-old group exhibited a positive impact on the more difficult Woodcock-Johnson III Spelling test (this same age group difference was also found in the overall average impacts). Finally, in terms of vocabulary, significant impacts were found for 3 year-old Head Start group children from English-speaking families (on the Color Naming task only) and from Spanish-speaking families (on both the PPVT-III (adapted) and Color Naming). As shown in Exhibit 5.1-C, among children in the 3-year-old group from Spanish-speaking families, the IRT scale score on the PPVT-III (adapted) was about nine points higher for the Head Start group than for the non-Head Start group. The score on the Color Naming test for these children was 2.52 points higher than that of children in the non-Head Start group, meaning that the Spanish-speaking Head Start children could recognize more than two additional colors than the non-Head Start group of Spanish-speaking children. For these Spanish-speaking children, these gains represent a reduction in their deficit from national norms on the PPVT-III of 13 percent8.
Head Start also had an impact on parent perceptions of children’s emerging literacy (the PELS measure), for both English- and Spanish-speaking families in the 3-year-old group. For the 4-year-old group, an impact was only found for the English-speaking families.
Race and Ethnicity. The results for race and ethnicity are somewhat scattered across the cognitive domains, but there is particularly strong evidence that Head Start is having a positive impact on the cognitive development of minority children. Specifically, for Hispanic children in the 3-year-old group, positive impacts are noted in pre-reading (both Woodcock-Johnson III Letter Word Identification and Letter Naming), vocabulary (PPVT-III, adapted), and pre-writing (Woodcock-Johnson III Spelling). For African American children in the 3-year-old group, positive impacts are noted in pre-reading (Woodcock-Johnson III Letter Word Identification), phonological awareness (CTOPPP Elision), and in pre-writing (Draw-a-Design); for African American children in the 4-year-old group, positive impacts are noted in pre-reading skills (Woodcock-Johnson III Letter-Word Identification), and on early writing (Woodcock-Johnson III Spelling).
Additionally, for White children in the 3-year-old group, positive impacts are noted in oral comprehension (Woodcock-Johnson III Oral Comprehension); for White children in the 4-year-old group, positive impacts are noted in pre-reading skills (Letter Naming Task).
Head Start also had an impact on parent perceptions of children’s emerging literacy (the PELS measure), for African American, Hispanic, and White children in the 3-year-old group. For the 4-year-old group an impact was only found for African-American children.
Parent’s Marital Status. Although impacts for these subgroups are noted on several cognitive measures, the estimated impacts are scattered across domains and inconsistent in terms of whether greater impacts are noted for children with married parents or for children with unmarried parents. Consequently, at this time, no conclusions can be drawn from these results, and further analysis is needed.
Special Needs Children. Impacts are found in both pre-reading and pre-writing skills, but in all cases, the impact of Head Start is found for children without special needs.
In total, the subgroup-specific findings indicate that cognitive gains from Head Start participation are widespread across the different demographic groups examined, with all groups except special needs children shown to be benefiting.
| Outcome Measure | Intent-To-Treat Impact Estimates | ||||
|---|---|---|---|---|---|
| Head Start Mean | Non-Head Start Mean | Mean Difference 2 | Regression-Adjusted Impact Estimates (Demographic Covariates Only) | Regression-Adjusted Impact Estimates (With Fall Measure) 3 | |
| (Sample N=2,071): | |||||
| PPVT-IRT/ML | 254.0 | 250.0 | 4.0* | 3.38 | 4.23* (0.12) |
| CTOPPP Elision (English) | 243.4 | 239.7 | 3.6 | 3.51 | 4.10 |
| Woodcock-Johnson: Letter-Word Identification - IRT/ML 1i | 307.0 | 300.5 | 6.5*** | 5.65** (0.24) | 5.53*** |
| Letter Naming Task | 5.5 | 3.9 | 1.6** | 1.22* | 1.30* (0.19) |
| Woodcock-Johnson: Spelling - IRT/ML | 346.6 | 343.6 | 2.9 | 2.05 | 2.40 |
| Woodcock-Johnson: Applied Problems-IRT/ML | 377.3 | 373.6 | 3.7 | 3.62 | 4.04 |
| Woodcock-Johnson: Oral Comprehension-IRT/ML | 435.5 | 435.4 | 0.1 | 0.35 | 0.62 |
| Color Naming/ Identification | 13.9 | 13.0 | 0.9* | 0.87 | 0.70* (0.10) |
| McCarthy Drawing | 3.2 | 3.0 | 0.2** | 0.14* | 0.15* (0.13) |
| Counting Bears | 2.9 | 2.7 | 0.2 | 0.14 | 0.12 |
| Parent Educational Literacy Activities Scale (PELS) 1ii | 2.9 | 2.4 | 0.5*** | 0.47 *** (0.34) | 0.42*** |
|
* = p≤0.05, ** = p≤0.01, *** = p≤0.001. 1 Fall measure used in regression failed statistical test. (back: 1i, 1ii) 2 Differences are rounded to the nearest 0.1. (back) 3 As described in Chapter 4, two regression specifications were estimated for some of the cognitive outcomes in the combined sample. The two models yielded the same results. (back) Note: Numbers in parentheses in shaded boxes are estimated effect sizes. |
| Outcome Measure | Intent-To-Treat Impact Estimates | ||||
|---|---|---|---|---|---|
| Head Start Mean | Non-Head Start Mean | Mean Difference 2 | Regression-Adjusted Impact Estimates (Demographic Covariates Only) | Regression-Adjusted Impact Estimates (With Fall Measure) | |
| (Sample N=1,629): | |||||
| PPVT-IRT/ML | 259.0 | 255.5 | 3.5 | 2.85 | 3.70 |
| CTOPPP Elision (English) | 248.2 | 242.9 | 5.3 | 4.70 | 3.89 |
| Woodcock-Johnson: Letter-Word Identification - IRT/ML | 308.8 | 302.7 | 6.1** | 5.43** | 5.19*** (0.15) |
| Letter Naming Task | 5.7 | 4.3 | 1.4* | 1.13 | 1.22* (0.18) |
| Woodcock-Johnson: Spelling - IRT/ML | 346.1 | 344.4 | 1.7 | 1.12 | 0.95 |
| Woodcock-Johnson: Applied Problems-IRT/ML | 381.3 | 378.2 | 3.0 | 3.04 | 2.94 |
| Woodcock-Johnson: Oral Comprehension-IRT/ML1 | 438.9 | 438.4 | 0.5 | 0.43 | 0.22 |
| Color Naming/ Identification | 14.7 | 14.1 | 0.7 | 0.66 | 0.56 |
| McCarthy Drawing | 3.2 | 3.0 | 0.2* | 0.14* | 0.15* (0.13) |
| Counting Bears | 2.9 | 2.8 | 0.1 | 0.12 | 0.10 |
| Parent Educational Literacy Activities Scale (PELS) | 3.0 | 2.5 | 0.5*** | 0.48*** | 0.44*** (0.19) |
* = p≤0.05, ** = p≤0.01, *** = p≤0.001. 1 Fall measure used in regression failed statistical test. (back) 2 Differences are rounded to the nearest 0.1. (back) Note: Numbers in parentheses in shaded boxes are estimated effect sizes. |
| Outcome Measure | Intent-To-Treat Impact Estimates | ||||
|---|---|---|---|---|---|
| Head Start Mean | Non-Head Start Mean | Mean Difference 2 | Regression-Adjusted Impact Estimates (Demographic Covariates Only) | Regression-Adjusted Impact Estimates (With Fall Measure) | |
| (Sample N=442): | |||||
| PPVT-IRT/ML1 | 234.2 | 225.1 | 9.1* | 9.09* (0.26) | 6.31* |
| TVIP-IRT/ML1 | 253.4 | 247.1 | 6.3 | 3.24 | 4.60 |
| CTOPPP Elision (English)1 | 224.0 | 224.6 | -0.5 | 0.56 | 0.80 |
| Woodcock-Johnson: Letter-Word Identification-IRT/ML1 | 300.1 | 291.3 | 8.8* | 8.75* (0.36) | 6.88 |
| Woodcock-Johnson: Letter-Word Identification-IRT/ML1 | 352.2 | 350.2 | 2.0 | 0.64 | -0.25 |
| Letter Naming Task 1 | 4.6 | 2.2 | 2.3* | 1.77* (0.27) | 1.53 |
| Woodcock-Johnson: Spelling-IRT/ML1 | 348.3 | 340.3 | 8.0* | 6.79 | 4.83 |
| Woodcock-Johnson: Applied Problems-IRT/ML1 | 361.7 | 353.4 | 8.2 | 8.68 | 5.91 |
| Woodcock-Johnson: Oral Comprehension-IRT/ML1 | 422.0 | 421.3 | 0.7 | 1.14 | 1.08 |
| Color Naming/ Identification 1 | 10.9 | 8.7 | 2.2 | 2.52* (0.35) | 1.77* |
| McCarthy Drawing 1 | 3.3 | 3.1 | 0.2 | 0.18 | 0.16 |
| Counting Bears 1 | 2.5 | 2.2 | 0.3 | 0.30 | 0.25 |
| Parent Educational Literacy Activities Scale (PELS) 1 | 2.4 | 1.9 | 0.6** | 0.52** (0.22) | 0.42*** |
* = p≤0.05, ** = p≤0.01, *** = p≤0.001. 1 Fall measure used in regression failed statistical test. 2 Differences are rounded to the nearest 0.1. Note: Numbers in parentheses in shaded boxes are estimated effect sizes. |
| Outcome Measure | Intent-To-Treat Impact Estimates | ||||
|---|---|---|---|---|---|
| Head Start Mean | Non-Head Start Mean | Mean Difference 2 | Regression-Adjusted Impact Estimates (Demographic Covariates Only) | Regression-Adjusted Impact Estimates (With Fall Measure) 3 | |
| (Sample N=1,638): | |||||
| PPVT-IRT/ML | 293.9 | 291.3 | 2.5 | 2.00 | 2.59 |
| CTOPPP Elision (English) | 275.2 | 273.7 | 1.6 | 1.01 | 1.40 |
| Woodcock-Johnson: Letter-Word Identification - IRT/ML 1i | 325.5 | 319.2 | 6.2* | 5.74* (0.22) | 3.48 |
| Letter Naming Task | 11.5 | 9.2 | 2.3** | 2.23** | 2.28**(0.24) |
| Woodcock-Johnson: Spelling - IRT/ML | 371.6 | 367.7 | 3.9* | 4.01* | 4.14*(0.16) |
| Woodcock-Johnson: Applied Problems-IRT/ML | 397.5 | 394.4 | 3.0 | 2.69 | 2.88 |
| Woodcock-Johnson: Oral Comprehension-IRT/ML | 443.4 | 443.7 | -0.2 | -1.05 | -0.90 |
| Color Naming/ Identification 1ii | 17.1 | 16.5 | 0.7 | 0.60 | 0.18 |
| McCarthy Drawing 1iii | 4.5 | 4.4 | 0.2 | 0.22 | 0.13 |
| Counting Bears | 3.8 | 3.6 | 0.2 | 0.16 | 0.12 |
| Parent Educational Literacy Activities Scale (PELS) 1iv | 3.8 | 3.3 | 0.4*** | 0.41*** (0.29) | 0.29** |
|
* = p≤0.05, ** = p≤0.01, *** = p≤0.001. 1 Fall measure used in regression failed statistical test. (back: 1i, 1ii, 1iii, 1iv) 2 Differences are rounded to the nearest 0.1. (back) 3 As described in Chapter 4, two regression specifications were estimated for some of the cognitive outcomes in the combined sample. The two models yielded the same results. Note: Numbers in parentheses in shaded boxes are estimated effect sizes. (back) |
| Outcome Measure | Intent-To-Treat Impact Estimates | ||||
|---|---|---|---|---|---|
| Head Start Mean | Non-Head Start Mean | Mean Difference 2 | Regression-Adjusted Impact Estimates (Demographic Covariates Only) | Regression-Adjusted Impact Estimates (With Fall Measure) | |
| (Sample N=1,130): | |||||
| PPVT-IRT/ML | 304.3 | 303.6 | 0.8 | 0.85 | 1.25 |
| CTOPPP Elision (English) | 284.9 | 282.8 | 2.2 | 1.85 | 0.15 |
| Woodcock-Johnson: Letter-Word Identification - IRT/ML 1 | 330.2 | 322.5 | 7.8* | 6.86* (0.26) | 4.01 |
| Letter Naming Task | 13.1 | 10.1 | 3.1** | 2.88** | 2.99** (0.32) |
| Woodcock-Johnson: Spelling - IRT/ML 1 | 371.8 | 367.2 | 4.6* | 4.07* (0.16) | 3.14 |
| Woodcock-Johnson: Applied Problems-IRT/ML | 402.5 | 400.8 | 1.7 | 1.56 | 1.46 |
| Woodcock-Johnson: Oral Comprehension-IRT/ML | 450.1 | 450.6 | -0.5 | -0.38 | -0.57 |
| Color Naming/ Identification 1 | 17.9 | 17.5 | 0.4 | 0.44 | 0.01 |
| McCarthy Drawing 1 | 4.5 | 4.3 | 0.2 | 0.26 | 0.14 |
| Counting Bears 1 | 3.8 | 3.8 | 0.0 | 0.02 | -0.02 |
| Parent Educational Literacy Activities Scale (PELS) 1 | 4.0 | 3.5 | 0.5*** | 0.44*** (0.13) | 0.23* |
* = p≤0.05, ** = p≤0.01, *** = p≤0.001. 1 Fall measure used in regression failed statistical test. 2 Differences are rounded to the nearest 0.1. Note: Numbers in parentheses in shaded boxes are estimated effect sizes. |
| Outcome Measure | Intent-To-Treat Impact Estimates | ||||
|---|---|---|---|---|---|
| Head Start Mean | Non-Head Start Mean | Mean Difference 2 | Regression-Adjusted Impact Estimates (Demographic Covariates Only) | Regression-Adjusted Impact Estimates (With Fall Measure) | |
| (Sample N=508): | |||||
| PPVT-IRT/ML 1 | 266.9 | 262.5 | 4.4 | 5.92* (0.15) | 8.49* |
| TVIP-IRT/ML 1 | 296.0 | 291.9 | 4.1 | 5.93 | 7.95 |
| CTOPPP Elision (English) 1 | 250.4 | 251.7 | -1.3 | -0.39 | 0.67 |
| Woodcock-Johnson: Letter-Word Identification-IRT/ML 1 | 313.3 | 311.7 | 1.5 | 1.70 | 7.31* |
| Woodcock-Munoz: Letter-Word Identification-IRT/ML 1 | 358.0 | 357.1 | 0.9 | 1.31 | 2.76** |
| Letter Naming Task 1 | 7.5 | 7.3 | 0.1 | 0.32 | 1.27 |
| Woodcock-Johnson: Spelling - IRT/ML 1 | 371.1 | 368.8 | 2.3 | 2.29 | 3.90 |
| Woodcock-Johnson: Applied Problems - IRT/ML 1 | 384.5 | 379.9 | 4.7 | 5.72 | 8.29* |
| Woodcock-Johnson: Oral Comprehension - IRT/ML 1 | 425.2 | 426.6 | -1.4 | -1.43 | -0.73 |
| Color Naming/ Identification | 15.1 | 14.2 | 0.9 | 0.98 | 0.61 |
| McCarthy Drawing 1 | 4.7 | 4.6 | 0.0 | 0.14 | 0.12 |
| Counting Bears 1 | 3.6 | 3.2 | 0.4 | 0.41 | 0.41 |
| Parent Educational Literacy Activities Scale (PELS) 1 | 3.2 | 2.9 | 0.3 | 0.32 | 0.29** |
* = p≤0.05, ** = p≤0.01, *** = p≤0.001. 1 Fall measure used in regression failed statistical test. 2 Differences are rounded to the nearest 0.1. Note: Numbers in parentheses in shaded boxes are estimated effect sizes. |
| Outcome Measures | Estimated Impact of Access to Head Start | Effect Size | |
|---|---|---|---|
| Overall Impact | PPVT-III | 4.23* | 0.12 |
| WJ-III Letter-Word Identification--IRT/ML | 5.65** | 0.24 | |
| Letter Naming Task | 1.30* | 0.19 | |
| Color Naming/Identification | 0.70* | 0.10 | |
| McCarthy Drawing | 0.15* | 0.13 | |
| PELS | 0.47*** | 0.34 | |
| Difference in Impact 1 | PPVT-III: Depression | -0.11* | - 0.00 |
| CTOPPP Elision: Depression | -0.13* | - 0.00 | |
| WJ-III Applied Problems: Depression | -0.09* | - 0.00 | |
| WJ-III Oral Comprehension: Race (White Impact Exceeds African American) | 4.73* | 0.33 | |
| Color Naming/Identification: Depression | -0.02* | - 0.00 | |
| Counting Bears: Depression | -0.00* | - 0.00 | |
| PELS: Depression | -0.00* | - 0.00 | |
| Impact on Subgroup 2 | PPVT-III: Parent Married | 4.51* | 0.13 |
| PPVT-III: Spanish-English Language Group | 7.52* | 0.21 | |
| PPVT-III: Hispanic | 7.26* | 0.21 | |
| CTOPPP Elision: African American | 7.47* | 0.17 | |
| WJ-III Letter-Word Identification: No Special Needs | 5.38** | 0.22 | |
| WJ-III Letter-Word Identification: African American | 5.80** | 0.24 | |
| WJ-III Letter-Word Identification: Hispanic | 6.92* | 0.29 | |
| WJ-III Letter-Word Identification: English-English Language Group | 5.05*** | 0.21 | |
| WJ-III Letter-Word Identification: Parent Married | 6.53** | 0.27 | |
| WJ-III Letter-Word Identification: Parent Not Married | 5.21** | 0.22 | |
| WJ-III Spelling: Hispanic | 5.61* | 0.25 | |
| Letter Naming Task: No Special Needs | 1.24* | 0.19 | |
| Letter Naming Task: Hispanic | 1.45* | 0.22 | |
| Letter Naming Task: Parent Not Married | 1.46* | 0.22 | |
| WJ-III Oral Comprehension: White | 2.82** | 0.20 | |
| WJ-III Oral Comprehension: Parent Married | 2.09* | 0.15 | |
| Color Identification: English-English Language Group | 0.87** | 0.12 | |
| Color Identification: Parent Married | 1.50** | 0.21 | |
| McCarthy Drawing: No Special Needs | 0.16* | 0.14 | |
| McCarthy Drawing: African American | 0.18* | 0.16 | |
| McCarthy Drawing: English-English Language Group | 0.17* | 0.15 | |
| PELS: No Special Needs | 0.50*** | 0.36 | |
| PELS: White | 0.37*** | 0.27 | |
| PELS: African American | 0.53** | 0.38 | |
| PELS: Hispanic | 0.51** | 0.37 | |
| PELS: English-English Language Group | 0.48*** | 0.35 | |
| PELS: Spanish-English Language Group | 0.46* | 0.33 | |
| PELS: Parent Married | 0.52*** | 0.38 | |
| PELS: Parent Not Married | 0.43*** | 0.31 | |
|
* = p≤0.05, ** = p≤0.01, *** = p≤0.001. 1 A total of 82 differences in impacts between subgroups were examined. The complete set of results, including differences not found to be statistically significant, appears in Appendix 5.2. Findings for depression indicate the change in Head Start’s estimated impact that accompanies a 1-point increase in mother’s baseline depression score. Findings for baseline factors other than depression indicate the amount by which Head Start’s estimated impact for the first subset of participants listed in the row label exceeds that for the second subset listed. (back) 2 A total of 99 subgroup impacts were examined. The complete set of results, including differences not found to be statistically significant, appears in Appendix 5.2. (back) |
| Outcome Measures | Estimated Impact of Access to Head Start | Effect Size | |
|---|---|---|---|
| Overall Impact | WJ-III Letter-Word Identification | 5.74* | 0.22 |
| Letter Naming Task | 2.28** | 0.24 | |
| WJ-III-Spelling | 4.14* | 0.16 | |
| Color Identification | 0.60 | 0.11 | |
| PELS | 0.41*** | 0.29 | |
| Difference in Impact 1 | Counting Bears: Race (Hispanic Impact Exceeds White) | 0.52* | 0.38 |
| Impact on Subgroup 2 | PPVT-III: Hispanic | 5.64* | 0.14 |
| WJ-III Letter-Word Identification: No Special Needs | 5.88* | 0.22 | |
| WJ-III Letter-Word Identification: African American | 10.56* | 0.40 | |
| WJ-III Letter-Word Identification: English-English | |||
| Language Group | 7.32* | 0.27 | |
| WJ-III Letter-Word Identification: Parent Not Married | 7.92* | 0.30 | |
| Letter Naming Task: No Special Needs | 2.39** | 0.25 | |
| Letter Naming Task: White | 2.77** | 0.29 | |
| Letter Naming Task: English-English Language Group | 3.05** | 0.32 | |
| Letter Naming Task: Parent Not Married | 2.70* | 0.29 | |
| McCarthy Drawing: Parent Not Married | 0.39* | 0.20 | |
| WJ-III Spelling: No Special Needs | 4.97** | 0.19 | |
| WJ-III Spelling: African American | 9.75** | 0.38 | |
| WJ-IIII Spelling: English-English Language Group | 4.49** | 0.17 | |
| WJ-III Spelling: Parent Not Married | 6.31* | 0.25 | |
| PELS: No Special Needs | 0.43*** | 0.30 | |
| PELS: African American | 0.75** | 0.53 | |
| PELS: English-English Language Group | 0.45*** | 0.32 | |
| PELS: Parent Married | 0.35* | 0.25 | |
| PELS: Parent Not Married | 0.52** | 0.37 | |
|
* = p≤0.05, ** = p≤0.01, *** = p≤0.001. 1 A total of 82 differences in impacts between subgroups were examined. The complete set of results, including differences not found to be statistically significant, appears in Appendix 5.2. Findings for depression indicate the change in Head Start’s estimated impact that accompanies a 1-point increase in mother’s baseline depression score. Findings for baseline factors other than depression indicate the amount by which Head Start’s estimated impact for the first subset of participants listed in the row label exceeds that for the second subset listed. (back) 2 A total of 99 subgroup impacts were examined. The complete set of results, including differences not found to be statistically significant, appears in Appendix 5.2 (back) |
1 Future analyses will test the statistical significance of the difference in impacts across the two age groups. (back)
2 The standard deviation of each outcome measure is derived from data on children/families in the non-Head Start sample, excluding members of the Head Start sample, to ensure that any effect of the intervention on the variation of the outcome is excluded from the calculation. (back)
3 Woodcock-Johnson III provides a Compuscore program that does not convert a raw score of 0 to a standard score. The W ability and standard scores presented in this chapter reflect the Compuscore program and therefore do not include children with a raw score of 0. This has been done so that these scores can be compared to the National norms. (back)
4 The mean standard score on the Spelling test for children in the 4-year-old group randomly assigned to Head Start was 92.5 in the spring of the program year, whereas the mean for children in the non-Head Start group was 89.6. (The mean standard score for all U.S. 4-year-olds is 100.0, with a standard deviation of 15.) The gap between the average score of the 4-year-old Head Start group and the overall national average score was 7.5 standard-score points; whereas, the gap between the average for the non-Head Start group and the national norm was 10.4 points. Hence, the Head Start group had a deficit that was smaller by 2.9 points, or 28 percent (2.9/10.4 = .28). (back)
5 The mean standard score on the PPVT-III for children in the 3-year-old group randomly assigned to Head Start was 82.9 in the spring of the program year and 81.4 for children in the non-Head Start group. (The mean standard score for all U.S. 3-year-olds is 100.0, with a standard deviation of 15.) The gap between the average score of the 3-year-old Head Start group and the overall national average score was 17.1 standard-score points; whereas, the gap between the average for the control group and the national norm was 18.6 points. Hence, the Head Start group had a deficit that was smaller by 1.5 points, or 8 percent (1.5/18.6 = .08). (back)
6 While each of the remaining subgroup findings taken one at a time is structured to limit the probability of a “false positive” to 1 in 20, as a group it is almost inevitable that some of these results will reach that level by chance alone. Only when a substantial share of all the tests of impact conducted for a given subgroup—or of a difference in impact between two subgroups—is statistically significant across all four of the outcome domains considered (not simply the outcomes reported in this chapter) can we be sure that at least some of those findings represent real impacts. (back)
7 As discussed elsewhere, this excludes children from Puerto Rico. (back)
8 The gap between the average score of children in the 3-year-old Head Start subgroup from Spanish-speaking families and the overall national average score was 24.7 standard-score points, whereas the gap between the average for the non-Head Start group and the national norm was 28.3 points. Hence, the Head Start group had a deficit that was smaller by 3.6 points, or 13 percent (3.63/28.3 = .13). (back)
| Table of Contents | Previous | Next |