ACF OPRE: Early Head Start Research and Evaluation Project (EHSRE)

D.8 RESULTS FROM RERUN OF 24-MONTH CHILD AND FAMILY OUTCOMES

Data on child and family outcomes based on the 24-Month Parent Interview were not complete at the time the Early Head Start interim report was written. Approximately 100 records from a number of sites were added to the sample after publication of the interim report. To ensure consistency of results, we reanalyzed the impact of Early Head Start programs on child and family outcomes at 24 months using this slightly augmented sample. The results of those analyses are consistent with the findings presented in the interim report, as summarized here.

1. Early Head Start Impacts for the Entire Sample

There were no dramatic changes in the impact of Early Head Start programs on the entire sample. Results are detailed in Tables D.8A through D.8H. Some of the smaller changes are:

Some effects sizes for child cognitive outcomes (the Bayley MDI and MacArthur vocabulary and sentence complexity) became slightly larger and more statistically significant (Table D.8A). Early Head Start children scored higher than control children on these measures.
The effect size for parent supportiveness in parent-child semistructured play became slightly smaller, and the statistical significance dropped to the .10 level (Table D.8C).
The effect sizes for several “discipline strategy” variables became somewhat larger, and many became significant at the .05 and .01 levels (Tables D.8E and D.8F). This reinforces the pattern shown in the interim report, which showed that Early Head Start parents were more likely to suggest mild discipline and less likely to suggest severe discipline strategies.

2. Early Head Start Program Impacts by Program Approach

Tables D.8I through D.8L show the results of the analyses by program approach. Once again, there were no major differences compared with the results reported in the interim report. Some of the smaller changes are:

Some effect sizes for child cognitive outcomes (the Bayley MDI and MacArthur vocabulary and sentence complexity) became slightly larger and more statistically significant. The impact of Early Head Start on these cognitive and language development measures was still found largely in center-based and mixed-approach programs (Table D.8I).
Early Head Start programs had positive impacts on a number of child language and social-emotional development outcomes in home-based and mixed-approach programs, just as reported in the interim report (Table D.8I).
In this reanalysis, Early Head Start programs still showed positive impacts on many parenting behaviors in home-based and mixed-approach programs (Table D.8J). The positive impacts on the HOME language environment and the reduction of parent detachment both were slightly more robust with the full sample (statistical significance rises to the .05 level) in the home-based programs.
There were minor changes in the statistical significance of the impact of Early Head Start on parenting behavior in mixed-approach programs, but effect sizes remained the same (Table D.8J).
The impact of Early Head Start on suggesting mild discipline strategies in center-based programs became larger and statistically significant at the .05 level. The impact on knowledge of infant development in home-based programs also became more robust (Table D.8K).
There were generally no changes in the magnitude of Early Head Start impacts on family health and functioning. However, the reduction in parental distress and family conflict in home-based programs became significant at the .10 level (Table D.8L).

3. Early Head Start Program Impacts by Programs’ Implementation Pattern

Tables D.8M through D.8P show the results of the analyses by the implementation pattern of programs. Once again, there were no major changes from the results reported in the interim report. Some of the smaller changes were:

The only change in child cognitive and language development was that the impact on the MacArthur combining word score became more robust, increasing to the .05 level for early implemented programs (Table D.8M).
The most notable change in child social-emotional development was the negative impact of the Bayley Emotional Regulation measure; it became statistically significant at the .05 level for incompletely implemented programs (Table D.8M).
There were no notable changes in the magnitude of the impacts of Early Head Start on parenting behavior, although some impacts became slightly more and some slightly less robust (Table D.8N).

TABLE D.8A

IMPACTS ON COGNITIVE AND LANGUAGE DEVELOPMENT
Outcome	Program Group Participantsa	Control Groupb	Estimated Impact Per Participantc	Effect Sized
Cognitive Development
Bayley Mental Development Index (MDI)	90.1	88	2.1***	15.6
Percent with Bayley MDI Below 100	75.2	79.7	-4.5**	-11
Percent with Bayley MDI Below 85	33.8	40.8	-6.9***	-14.2
Language Development
MacArthur Communicative Development Inventory (CDI): Vocabulary Production Score	56.5	53.9	2.6**	11.7
MacArthur CDI: Percent Combining Words	81	77.9	3.1*	7.5
MacArthur CDI: Sentence Complexity Score	8.8	7.7	1.1**	13.4
Sample Size
Parent Interview	1,118	1,048	2,166
Bayley	931	850	1,781

SOURCE: Parent interview and child assessments conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

aA participant is defined as a program group member who received more than one Early Head Start home visit, met with an Early Head Start case manager more than once, received at least two weeks of Early Head Start center-based care, and/or participated in Early Head Start group parent-child activities.(back)

bThe control group mean is the mean for the control group members who would have participated in Early Head Start if they had instead been assigned to the program group. This unobserved mean was estimated as the difference between the program group mean for participants and the impact per participant.(back)

cThe estimated impact per participant is measured as the estimated impact per eligible applicant divided by the proportion of program group members who participated in Early Head Start services (which varied by site). The estimated impact per eligible applicant is measured as the difference between the regression-adjusted means for all program and control group members.(back)

dThe effect size was calculated by dividing the estimated impact per participant by the standard deviation of the outcome measure for the control group times 100 (that is, it is the impact expressed as a percentage of the standard deviation).(back)

*Significantly different from zero at the .10 level, two-tailed test
**Significantly different from zero at the .05 level, two-tailed test.
***Significantly different from zero at the .01 level, two-tailed test.

TABLE D.8B

IMPACTS ON SOCIAL-EMOTIONAL DEVELOPMENT FOR THE FULL SAMPLE
Outcome	Program Group Participantsa	Control Groupb	Estimated Impact Per Participantc	Effect Sized
Parent-Child Structured Play: Engagement of Parente	4.3	4.2	0.1*	8.7
Parent-Child Structured Play: Negativity toward Parente	1.7	1.8	-0.1	-7.1
Parent-Child Structured Play: Sustained Attention with Objectse	5	5	0.1	7.1
Bayley Behavior Rating Scale (BRS): Emotional Regulation in a Cognitive Task (average score)f	3.6	3.6	0	-1.6
Bayley Behavior Rating Scale (BRS): Orientation/ Engagement in a Cognitive Task (average score)f	3.7	3.7	0	0
Child Behavior Checklist: Aggressive Behavior Problems (average score)	9.9	10.4	-0.5*	-9
Sample Size	1,118	1,048	2,166

SOURCE: Parent interviews, child assessments, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

eBehaviors are observed during the videotaped Parent-Child Structured Play task and coded on a seven-point scale.(back)

fBehaviors are observed during the Bayley assessment and rated on a five-point scale by the Interviewer/Assessor.(back)

TABLE D.8C

IMPACTS ON EMOTIONAL SUPPORT
Outcome	Program Group Participantsa	Control Groupb	Estimated Impact Per Participantc	Effect Sized
Home Observation for Measurement of the Environment (HOME): Emotional Responsivitye	6.2	6.1	0.1*	8.6
Parent-Child Structured Play: Supportivenessf	4.1	4	0.1*	8.9
Sample Size
Parent Interview	1,118	1,048	2,166
Parent-Child Interactions	941	855	1,796

SOURCE: Parent interviews and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

eBehaviors are observed during the HOME assessment and rated on a yes/no scale by the Interviewer/Assessor.(back)

fBehaviors are observed during the videotaped parent-child structured play task and coded on a seven-point scale.(back)

*Significantly different from zero at the .10 level, two-tailed test.
**Significantly different from zero at the .05 level, two-tailed test.
***Significantly different from zero at the .01 level, two-tailed test.

TABLE D.8D

IMPACTS ON THE HOME ENVIRONMENT AND STIMULATION OF LANGUAGE AND LEARNING
Outcome	Program Group Participantsa	Control Groupb	Estimated Impact Per Participantc	Effect Sized
Home Observation for Measurement of the Environment (HOME) - Total Score	26.5	26.1	0.4***	11.1
Structuring the Environment
HOME: Support of Cognitive, Language, and Literacy Environment	10.3	10.1	0.2***	12.7
Percentage of Parents Who Set a Regular Bedtime for Child	61.5	55.6	5.8**	11.6
Percentage of Parents and Children Who Have Regular Bedtime Routines	68.8	66.6	2.3	4.8
Parent-Child Activities
Parent-Child Activities	4.6	4.5	0.1**	10.6
Percentage of Parents Who Read to Child Every Day	58	52	5.9**	11.9
Percentage of Parents Who Read to Child at Bedtime	29	22.5	6.5***	15.3
Parent's Verbal-Social Skills
HOME: Maternal Verbal-Social Skillse	2.8	2.7	0	6.5
Sample Size
Parent Interview	1,118	1,048	2,166
Parent-Child Interactions	941	855	1,796

SOURCE: Parent interviews, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

eBehaviors are observed during the HOME assessment and rated on a yes/no scale by the Interviewer/Assessor.(back)

TABLE D.8E

IMPACTS ON NEGATIVE PARENTING BEHAVIOR IN STRUCTURED PLAY AND INTERACTION
Outcome	Program Group Participantsa	Control Groupb	Estimated Impact Per Participantc	Effect Sized
Insensitivity
Parent-Child Structured Play: Detachmente	1.4	1.5	-0.1**	-10.2
Parent-Child Structured Play: Intrusivenesse	1.9	1.9	0	-4.3
Hostility and Punishment
Parent-Child Structured Play: Negative Regarde	1.5	1.5	0	1.8
Home Observation for Measurement of the Environment (HOME):
Absence of Punitive Interactionsf	4.4	4.4	-0.1	-4
Percentage of Parents who Spanked the Child in the Previous Week	47.2	52.8	-5.6**	-11.2
Sample Size
Parent Interview	1,118	1,048	2,168
Parent-Child Interactions	941	855	1,796

SOURCE: Parent interviews, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

eBehaviors are observed during the videotaped parent-child structured play task and coded on a seven-point scale.(back)

fBehaviors are observed during the HOME assessment and rated on a yes/no scale by the Interviewer/Assessor.(back)

TABLE D.8F
IMPACTS ON PARENTING KNOWLEDGE: CHILD DEVELOPMENT AND DISCIPLINE STRATEGIES
Outcome	Program Group Participantsa	Control Groupb	Estimated Impact Per Participantc	Effect Sized
Knowledge of Child Development
Knowledge of Infant Development Inventory (KIDI)	3.4	3.3	0.1***	12.7
Discipline Strategies
Percentage of Parents Who Suggested Responses to Hypothetical Situations with Child:
Prevent or Distract	72.8	67.4	5.5**	11.7
Remove Child or Object	80.4	81.7	-1.4	-3.5
Talk and Explain	37.8	31.1	6.7***	14.3
Threaten or Command	32.1	34.3	-2.2	-4.6
Shout	5.8	5.1	0.7	3.4
Physical Punishment	27.1	30.5	-3.4*	-7.4
Percentage of Parents Suggesting Only Mild Responses to the Hypothetical Situationse	43.1	38.2	4.9**	10
Index of Severity of Discipline Strategies Suggestedf	2.7	2.8	-0.2**	-9
Sample Size	1,118	1,048	2,166


SOURCE: Parent interviews conducted when children were approximately 24 months old. NOTE: All impact estimates were calculated using regression models, where each site was weighted equally. aA participant is defined as a program group member who received more than one Early Head Start home visit, met with an Early Head Start case manager more than once, received at least two weeks of Early Head Start center-based care, and/or participated in Early Head Start group parent-child activities.(back) bThe control group mean is the mean for the control group members who would have participated in Early Head Start if they had instead been assigned to the program group. This unobserved mean was estimated as the difference between the program group mean for participants and the impact per participant.(back) cThe estimated impact per participant is measured as the estimated impact per eligible applicant divided by the proportion of program group members who participated in Early Head Start services (which varied by site). The estimated impact per eligible applicant is measured as the difference between the regression-adjusted means for all program and control group members.(back) dThe effect size was calculated by dividing the estimated impact per participant by the standard deviation of the outcome measure for the control group times 100 (that is, it is the impact expressed as a percentage of the standard deviation).(back) eParents were classified as suggesting only mild discipline if their responses to the three discipline situations include only the following: prevent or distract, remove child or object, or talk and explain.(back) fThe Index of Severity of Discipline Strategies is based on a hierarchy of discipline practices from talk and explain or prevent/distract (1) through physical punishment (5). The most severe approach suggested is used to code this scale.(back) Significantly different from zero at the .10 level, two-tailed test Significantly different from zero at the .05 level, two-tailed test. **Significantly different from zero at the .01 level, two-tailed test.

TABLE D.8G

IMPACTS ON SAFETY PRACTICES
(Percentages)
Outcome	Program Group Participantsa	Control Groupb	Estimated Impact Per Participantc	Effect Sized
Family Has Syrup of Ipecac in the House in Case of a Poison Emergency	29.9	29.6	0.3	0.6
Parent/Guardian Has or Knows How to Find the Telephone Number For the Poison Control	38.2	36	2.2	4.5
Family Uses a Gate or Door at the Top of Stairs	79	80.9	-1.8	-3.8
Family Uses Guards or Gates For Windows	63	64.8	-1.8	-3.8
Family Has Covers on Electrical Outlets That Child Can Reach	60.6	60.7	-0.1	-0.2
Family's Homes Has Working Smoke Alarms	87.1	84.8	2.3	6.2
Family Uses a Car Seat For Child and it is in the Back Seat of the Car	80.8	82	-1.2	-3.1
Interviewer Observed That Child's Play Area is Safe	69.2	68.8	0.3	0.7
Sample Size	1,118	1,048	2,166

SOURCE: Parent interviews and interviewer observations conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

TABLE D.8H

IMPACTS ON PARENT HEALTH AND FAMILY FUNCTIONING
Outcome	Program Group Participantsa	Control Groupb	Estimated Impact Per Participantc	Effect Sized
Parent's Physical Health
Parent's Health Status	3.5	3.5	0	2.7
Parent's Mental Health
Parental Distress	24.8	26	-1.2**	-12.2
Parent-Child Dysfunctional Interaction	16.9	17.4	-0.5*	-8.7
CIDI-Depression-Average Probability	12.1	12.1	0	1
Family Functioning
Family Environment Scale- Family Conflict (Average Score)	1.7	1.7	-0.1**	-10.3
Sample Size	1,118	1,048	2,166

SOURCE: Parent interviews conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

TABLE D.8I

IMPACTS ON CHILD OUTCOMES AT AGE 2, BY PROGRAM APPROACH IN 1997
Outcome	Program Group	Control Groupa	Impact Estimate Per Participantb	Effect Sizec	Program Group	Control Groupa	Impact Estimate Per Participantb	Effect Sizec	Program Group	Control Groupa	Impact Estimate Per Participantb	Effect Sizec
	Center-Based				Home-Based Programs				Mixed-Approach Programs
Child Cognitive Development
Average Bayley Mental Development Index (MDI)	90.1	87.1	3.1**	22.9	91.6	90.4	1.1	8.4	88.2	86.9	1.4	10.1
Percentage with MDI < 85***d	31.4	43.6	-12.2*	-25	31.4	32.7	-1.3	-2.7	37.9	44.8	-6.9	-14.1
Percentage with MDI < 100	75.4	84.1	-8.2*	-20.1	73.1	72.2	1	2.4	77.3	79.8	-2.6	-6.3
Child Language Development
Average MacArthur CDI-Vocabulary Production	55	55	0	-0.2	56.5	53.3	3.1*	13.9	57.5	53.4	4.1**	18.3
Percentage with Vocabulary Production < 25***	11	12.7	-1.7	-5.2	11.4	11.2	0.3	0.8	5.4	8.4	-3.1	-9.6
Percent MacArthur CDI-Combining Words***	84.1	83.6	0.5	1.2	76.9	75.6	1.2	3	83.8	75.3	8.5***	20.2
Average MacArthur CDI-Sentence Complexity*	8.7	8.5	0.2	2.8	8.5	7.7	0.7	9	9.2	6.9	2.3***	28.5
Percentage with Sentence Complexity < 2***	29.1	25.7	3.4	7.4	28.7	30.4	-1.8	-3.9	22.7	31.4	-8.7**	-19
Child Social-Emotional Development
Bayley BRS-Emotional Regulation	3.7	3.7	0.1	7.3	3.6	3.6	-0.1	-5.7	3.6	3.7	0	-4.9
Bayley BRS-Orientation/Engagement	3.7	3.7	-0.1	-7.7	3.6	3.6	0	1.2	3.7	3.7	0	-2.9
Child Behavior Checklist-Aggression	9.3	9.9	-0.6	-11	10.4	10.5	-0.2	-2.7	9.7	10.6	-0.9*	-16
Parent-Child Structured Play: Child Sustained Attention with Objects	5	5.1	-0.1	-8.1	5.1	5	0	4.2	5.1	4.9	0.2*	17.6
Parent-Child Structured Play: Child Negativity Toward Parent	1.8	1.7	0.1	-7.1	1.7	1.7	0	-4.3	1.8	2	-0.2*	-18.8
Parent-Child Structured Play: Child Engagement	4.4	4.4	-0.1	-3.9	4.3	4.3	0	3.7	4.3	4	0.3**	21.5
Sample Size
Parent Interview	240	203	443		500	466	966		352	352	704
Bayley	217	181	398		432	387	819		282	282	564
Parent-Child Interactions	236	195	431		429	374	803		276	286	562

SOURCE: Parent interviews, child assessments, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All estimates were calculated using regression models, where each site was weighted equally. Only sites with at least 10 program group members and 10 control group members in the subgroup are included in the estimates for each subgroup.

aThe control group mean is the mean for the control group members who would have participated in Early Head Start if they had instead been assigned to the program group. This unobserved mean is estimated as the difference between the program group mean for participants and the impact per participant. A participant is defined as a program group member who received more than one Early Head Start home visit, met with an Early Head Start case manager more than once, received at least two weeks of Early Head Start center-based services, and/or participated in Early Head Start parent-child group activities.(back)

bThe estimated impact per participant is measured as the estimated impact per eligible applicant divided by the proportion of program group members who participated in Early Head Start services (which varied by site). The estimated impact per eligible applicant is measured as the difference between the regression-adjusted means for program and control group members.(back)

cThe effect size is calculated by dividing the estimated impact per participant by the standard deviation of the outcome measure for the control group times 100 (that is, it is the impact per participant expressed as a percentage of a standard deviation).(back)

dAsterisks next to variable names indicate significance levels for statistical tests of differences in impacts across the subgroups.(back)

TABLE D.8J

IMPACTS ON PARENTING BEHAVIOR AT AGE 2, BY PROGRAM APPROACH IN 1997
Outcome	Program Group	Control Groupa	Impact Estimate Per Participantb	Effect Sizec	Program Group	Control Groupa	Impact Estimate Per Participantb	Effect Sizec	Program Group	Control Groupa	Impact Estimate Per Participantb	Effect Sizec
	Center-Based Programs				Home-Based Programs				Mixed-Approach Programs
Quality of the Home Environment and Parenting: Emotional Support
Home Observation for Measurement of the Environment (HOME) Emotional Responsivity	5.9	5.9	0	-2.4	6.5	6.4	0.1*	9.5	6	5.9	0.2	10.6
Parent-Child Structured Play: Parent Supportiveness	4	4	0	-4	4.1	4	0.1*	9.3	4.1	3.9	0.2**	19.3
Quality of the Home Environment and Parenting: Stimulation of Language and Learning
HOME Total Score	26.1	26.2	-0.1	-3.1	26.9	26.4	0.5**	13.2	26.3	25.7	0.6**	17.4
HOME Support of Cognitive, Language, and Literacy Environment*d	10.2	10.2	0	-1.6	10.3	10.1	0.2**	10.7	10.4	10	0.6***	21.3
Percentage of Parents who set a Regular Bedtime for Child***	65.9	57.6	8.3*	16.8	59.1	54	5.2	10.4	59.7	55	4.7	9.5
Percentage of Parents and Children Who have Regular Bedtime Routines***	70.1	67.5	2.6	5.5	69.6	65.4	4.2	9	66.4	65.9	0.5	1
Percentage of Parents Who Read to Child Daily***	56.7	49.2	7.5	15	55.8	54.3	1.5	3.1	60.7	48.3	12.4***	24.9
Percentage of Parents Who Read to Child as Part of Bedtime Routine***	29.3	21.1	8.2*	19.5	26.6	19.4	7.2**	17.1	30	25.5	4.5	10.7
Reading Frequency**	4.6	4.5	0.1	8.8	4.6	4.6	0	-1.1	4.7	4.3	0.4***	28.3
Parent-Child Activities to Stimulate Cognitive and Language Development	4.5	4.5	0	5.1	4.6	4.5	0	5	4.6	4.4