ACF OPRE: Head Start Impact Study and Follow-up

Appendix 6.1: Social-Emotional Domain Estimated Impact on Program Participants

This appendix focuses on the impact estimates that are adjusted for the fact that some of the children assigned to Head Start failed to take advantage of this opportunity; i.e., they never participated in Head Start (the “no-shows”) and that some children assigned to the non-Head Start group managed to find their way into the program (the “crossovers”). These results are summarized in Exhibit A.6.1.1 and A.6.1.2.

All measures of the impact of participation in Head Start exceeded the corresponding measures of the average impact of access to Head Start, regardless of which adjustment method is used. For example, the first row of Exhibit A.6.1.1 shows an estimated impact of -0.48 on the Total Problem Behavior Scale for the average child in the 3-year-old group granted access to the program and a larger average impact of -0.54 to -0.60 for those children who actually attended Head Start. These same increases are again seen in the relationship between the calculated effect sizes for the estimates of “access” vs. “participation,” with a few instances of the size of the effect moving from “small” (under 0.2) to “modest” (between 0.2 and 0.5).

A comparison of the estimates resulting from the two adjustment methods indicates that there is, as expected, some dampening effect of the crossover children in the non-Head Start group, i.e., reducing the size of any observed difference in outcomes between the Head Start and non-Head Start groups. However, as previously noted, the combined no-show/crossover adjustments should only be used as a rough indication of the likely consequences of the presence of crossovers on the interpretation of the impact of Head Start on children’s school readiness.

Exhibit A.6.1.1: Initial Estimates of the Impact of Head Start on Social-Emotional Outcomes, Intent to Treat, and Impact on the Treated: Stastically Significant Results Only, 3-Year-Old Group, Combined English-English and Spanish-English Group (Weighted Data)
Outcome Measure		Impact Estimates			Effect Sizes
Outcome Measure		Impact of Access to Head Start	Impact of Head Start Participation, No-Show Adjustment	Impact of Head Start Participation, Combined No-Show and Crossover Adjustment	Impact of Access to Head Start	Impact of Head Start Participation, No-Show Adjustment	Impact of Head Start Participation, Combined No-Show and Crossover Adjustment
Overall Impact	Total Problem Behavior Scale	-0.48**	-0.54**	-0.60**	-0.13	-0.14	-0.16
Overall Impact	Hyperactive Behavior Scale	-0.29**	-0.33**	-0.32*	- 0.18	- 0.21	-0.20
Difference in Impact 1	Social Competencies Checklist: Race (White Impact Exceeds African American)	0.49**	0.56***	N/A	0.39	0.45	N/A
Difference in Impact 1	Social Competencies Checklist: Race (Hispanic Impact Exceeds African American)	0.37*	0.42*	N/A	0.30	0.34	N/A
Impact on Subgroup 2	Total Problem Behavior Scale: No Special Needs	-0.52*	-0.59*	N/A	-0.14	-0.16	N/A
	Total Problem Behavior Scale: White	-0.86**	-0.98*	N/A	-0.23	-0.26	N/A
	Total Problem Behavior Scale: Parents Not Separated or Divorced	-0.50**	-0.56**	N/A	-0.13	-0.15	N/A
	Total Problem Behavior Scale: Parent Not Married	-0.47*	-0.52*	N/A	-0.13	-0.14	N/A
	Total Problem Behavior Scale: English-English Language Group	-0.46*	-0.52*	N/A	-0.12	-0.14	N/A
	Aggressive Behavior: White	-0.30*	-0.34*	N/A	-0.17	-0.19	N/A
	Hyperactive Behavior Scale: No Special Needs	-0.30*	-0.34*	N/A	-0.19	-0.22	N/A
	Hyperactive Behavior Scale: White	-0.34*	-0.39*	N/A	-0.22	-0.25	N/A
	Hyperactive Behavior Scale: Hispanic	-0.40*	-0.45**	N/A	-0.25	-0.28	N/A
	Hyperactive Behavior Scale: Male	-0.31*	-0.36*	N/A	-0.19	-0.23	N/A
	Hyperactive Behavior Scale: Parent Not Separated or Divorced	-0.33***	-0.37***	N/A	-0.21	-0.23	N/A
	Hyperactive Behavior Scale: Parent Married	-0.39*	-0.45*	N/A	-0.25	-0.28	N/A
	Hyperactive Behavior Scale: Parent Not Married	-0.25*	-0.28*	N/A	-0.16	-0.18	N/A
	Hyperactive Behavior Scale: English-English Language Group	-0.20*	-0.23*	N/A	-0.13	-0.15	N/A
	Hyperactive Behavior Scale: Spanish-English Language Group	-0.68**	-0.77*	N/A	-0.43	-0.49	N/A
	Social Competencies Checklist: African American	-0.34**	-0.38**	N/A	-0.27	-0.30	N/A

* = p≤0.05, ** = p≤0.01, *** = p≤0.001.

1 A total of 60 differences in impacts between subgroups were examined. The complete set of results, including differences not found to be statistically significant, appears in Appendix 6.2. Findings for depression indicate the change in Head Start’s estimated impact that accompanies a 1-point increase in mother’s baseline depression score. Findings for baseline factors other than depression indicate the amount by which Head Start’s estimated impact for the first subset of participants listed in the row label exceeds that for the second subset listed. (back)

2 A total of 78 subgroup impacts were examined. The complete set of results, including differences not found to be statistically significant, appears in Appendix 6.2. (back)

Exhibit A.6.1.2: Initial Estimates of the Impact of Head Start on Social Emotional, Intent to Treat and Impact on the Treated: Statistically Significant Results Only, 4-Year-Old Group, Combined English-English Spanish-English Group (Weighted Data)
Outcome Measure		Impact Estimates			Effect Sizes
Outcome Measure		Impact of Access to Head Start	Impact of Head Start Participation, No-Show Adjustment	Impact of Head Start Participation, Combined No-Show and Crossover Adjustment	Impact of Access to Head Start	Impact of Head Start Participation, No-Show Adjustment	Impact of Head Start Participation, Combined No-Show and Crossover Adjustment
Overall Impact	No statistically significant impacts	N/A	N/A	N/A	N/A	N/A	N/A
Difference in Impact 1	Social Competencies Checklist: Depression	-0.00*	-0.00*	N/A	- 0.00	- 0.00	N/A
Difference in Impact 1	Aggressive Behavior: (African American Impact Exceeds Hispanic)	0.81**	0.97**	N/A	0.51	0.61	N/A
Impact on Subgroup 2	Total Problem Behavior Scale: African American	-0.92**	-1.20**	N/A	-0.27	-0.36	N/A
	Aggressive Behavior Scale: African American	-0.61**	-0.80**	N/A	-0.38	-0.50	N/A
	Aggressive Behavior Scale: Female	-0.30*	-0.37*	N/A	-0.19	-0.23	N/A
	Aggressive Behavior Scale: English-English Language Group	-0.24*	-0.30*	N/A	-0.15	-0.19	N/A

* = p≤0.05, ** = p≤0.01, *** = p≤0.001.

2 A total of 78 subgroup impacts were examined. The complete set of results, including differences not found to be statistically significant, appears in Appendix 6.2. (back)

Table of Contents | Previous | Next