Skip Navigation
acfbanner  
ACF
Department of Health and Human Services 		  
		  Administration for Children and Families
          
ACF Home   |   Services   |   Working with ACF   |   Policy/Planning   |   About ACF   |   ACF News   |   HHS Home

  Questions?  |  Privacy  |  Site Index  |  Contact Us  |  Download Reader™Download Reader  |  Print Print      

Office of Planning, Research & Evaluation (OPRE) skip to primary page content
Advanced
Search

 Table of Contents | Previous | Next

D.6 ESTIMATING IMPACTS PER ELIGIBLE APPLICANT

In the body of the report, we focus on impacts per applicant for the child and family outcomes, because these impact estimates are more policy relevant and differ very little from the impacts per eligible applicant. Tables D.6A through D.6N show the impacts per eligible applicant for key outcome variables in order to illustrate how similar the impact findings are to those based on applicants.

TABLE D.6A
IMPACTS ON COGNITIVE AND LANGUAGE DEVELOPMENT
Outcome
Program
Group
Control
Group
Estimated Impact Per Eligible Applicanta
Effect
Sizeb
COGNITIVE DEVELOPMENT
Bayley Mental Development Index (MDI) 89.9 88.1 1.8*** 13.3
Percent with Bayley MDI Below 100 75.4 79.8 -4.4** 10.7
Percent with Bayley MDI Below 85 34.6 40.2 -5.6** 11.5
LANGUAGE DEVELOPMENT
MacArthur Communicative Development Inventories (CDI): Vocabulary Production Score 56.2 53.9 2.4** 10.5
MacArthur CDI: Percent Combining Words 80.5 77.5 2.9* 7.0
MacArthur CDI: Sentence Complexity Score 8.6 7.7 0.9** 11.3
Sample Size 1,021 1,092 2,113  


SOURCE: Parent interviews, child assessments, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

aThe estimated impact per eligible applicant is measured as the difference between the regression-adjusted means for all program and control group members.(back)

bThe effect size was calculated by dividing the estimated impact per eligible applicant by the standard deviation of the outcome measure for the control group times 100 (that is, it is the impact expressed as a percentage of the standard deviation).(back)

*Significantly different from zero at the .10 level, two-tailed test
**Significantly different from zero at the .05 level, two-tailed test.
***Significantly different from zero at the .01 level, two-tailed test.

 

TABLE D.6.B
IMPACTS ON SOCIAL AND EMOTIONAL DEVELOPMENT
Outcome Program
Group
Control
Group
Estimated Impact Per Eligible Applicantc Effect
Sized
Parent-Child Structured Play: Engagement of Parente 4.3 4.2 0.1 7.8
Parent-Child Structured Play: Negativity toward Parente 1.7 1.8 -0.1 -7.3
Parent-Child Structured Play: Sustained Attention with Objectse 5.0 5.0 0.1 6.9
Bayley Behavioral Rating Scale (BRS): Emotional Regulation in a Cognitive Taskf 3.6 3.6 0 -1.5
Bayley BRS: Orientation / Engagement in a Cognitive Taskf 3.6 3.6 0 0.1
Child Behavior Checklist: Aggressive 9.9 10.5 -0.5** -9.5
Sample Size 1,092 1,021 2,113  


SOURCE: Parent interviews, child assessments, and interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

cThe estimated impact per eligible applicant is measured as the difference between the regression-adjusted means for all program and control group members.(back)

dThe effect size was calculated by dividing the estimated impact per eligible applicant by the standard deviation of the outcome measure for the control group times 100 (that is, it is the impact expressed as a percentage of the standard deviation).(back)

eBehaviors are observed during the videotaped Parent-Child Structured Play task and coded on a seven-point scale.(back)

fBehaviors are observed during the Bayley assessment and rated on a five-point scale by the Interviewer/Assessor.(back)

*Significantly different from zero at the .10 level, two-tailed test
**Significantly different from zero at the .05 level, two-tailed test.
***Significantly different from zero at the .01 level, two-tailed test.

 

TABLE D.6C
IMPACTS ON EMOTIONAL SUPPORT FOR CHILDREN
Outcome Program
Group
Control
Group
Estimated
Impact Per Eligible Applicantg
Effect
Sizeh
Home Observation for Measurement of the Environment (HOME): Emotional Responsivityi 6.2 6.1 0.1* 7.4
Parent-Child Structured Play: Supportivenessj 4.0 3.9 0.1*** 13.3
Sample Size 1,092 1,021 2,113  


SOURCE: Parent interviews, child assessments, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

gThe estimated impact per eligible applicant is measured as the difference between the regression-adjusted means for all program and control group members.(back)

hThe effect size was calculated by dividing the estimated impact per eligible applicant by the standard deviation of the outcome measure for the control group times 100 (that is, it is the impact expressed as a percentage of the standard deviation).(back)

iBehaviors are observed during the HOME assessment and rated on a yes/no scale by the Interviewer/Assessor.(back)

jBehaviors are observed during the videotaped Parent-Child Structured Play task and coded on a seven-point scale.(back)

*Significantly different from zero at the .10 level, two-tailed test
**Significantly different from zero at the .05 level, two-tailed test.
***Significantly different from zero at the .01 level, two-tailed test.

 

TABLE D.6D
IMPACTS ON THE HOME ENVIRONMENT AND STIMULATION
OF LANGUAGE AND LEARNING
Outcome Program
Group
Control
Group
Estimated
Impact Per Eligible Applicantk
Effect
Sizel
Home Observation for Measurement of the Environment (HOME): Total Score 26.5 26.1 0.4** 9.8
STRUCTURING THE ENVIRONMENT
HOME: Support of Cognitive, Language, and Literacy Environment 10.3 10.1 0.2*** 11.2
Percentage of Parents Who Set a Regular Bedtime for Child 61.0 56.2 4.8** 9.7
Percentage of Parents and Children Who Have Regular Bedtime Routines 69.1 67.0 2.0 4.4
PARENT-CHILD ACTIVITIES
Parent-Child Activities 4.6 4.5 0.1** 10.4
Percentage of Parents Who Read to Child Every Day 57.4 52.3 5.1** 10.2
Percentage of Parents Who Read to Child at Bedtime 28.4 22.6 5.8*** 13.7
PARENT'S VERBAL-SOCIAL SKILLS
HOME: Maternal Verbal-Social Skillsm 2.8 2.7 0.0 6.4
Sample Size 1,092 1,021 2,113  


SOURCE: Parent interviews, child assessments, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

kThe estimated impact per eligible applicant is measured as the difference between the regression-adjusted means for all program and control group members.(back)

LThe effect size was calculated by dividing the estimated impact per eligible applicant by the standard deviation of the outcome measure for the control group times 100 (that is, it is the impact expressed as a percentage of the standard deviation).(back)

mBehaviors are observed during the HOME assessment and rated on a yes/no scale by the Interviewer/Assessor.(back)

*Significantly different from zero at the .10 level, two-tailed test
**Significantly different from zero at the .05 level, two-tailed test.
***Significantly different from zero at the .01 level, two-tailed test.

 

TABLE D.6E
IMPACTS ON NEGATIVE PARENTING BEHAVIOR IN STRUCTURED PLAY AND INTERACTION

Outcome

Program
Group

Control
Group

Estimated Impact Per Eligible Applicantn

Effect
Sizeo

INSENSITIVITY

Parent-Child Structured Play: Detachmentp

1.4

1.5

-0.1**

-10.2

Parent-Child Structured Play: Intrusivenessp

1.9

1.9

0

-3.3

HOSTILITY AND PUNISHMENT

Parent-Child Structured Play: Negative Regardp

1.5

1.4

0

2.1

Home Observation of Measurement of the Environment (HOME): Absence of Punitive Interactionsq

4.4

4.4

0

-3.7

Percentage of Parents who Spanked the Child in the Previous Week

48.1

52.5

-4.4**

-8.9

Sample Size

1,092

1,021

2,113

 


SOURCE: Parent interviews, child assessments, interviewer observations, and assessments of parent-child interactions during videotaped, semi-structured tasks conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

nThe estimated impact per eligible applicant is measured as the difference between the regression-adjusted means for all program and control group members.(back)
oThe effect size was calculated by dividing the estimated impact per eligible applicant by the standard deviation of the outcome measure for the control group times 100 (that is, it is the impact expressed as a percentage of the standard deviation).(back)
pBehaviors are observed during the videotaped Parent-Child Structured Play task and coded on a seven-point scale.(back)
qBehaviors are observed during the HOME assessment and rated on a yes/no scale by the Interviewer/Assessor(back)

*Significantly different from zero at the .10 level, two-tailed test
**Significantly different from zero at the .05 level, two-tailed test.
***Significantly different from zero at the .01 level, two-tailed test.

 

TABLE D.6F
IMPACTS ON PARENTING KNOWLEDGE: CHILD DEVELOPMENT AND DISCIPLINE STRATEGIES
Outcome Program
Group
Control
Group
Estimated Impact Per Eligible Applicantr Effect
Sizes
KNOWLEDGE OF CHILD DEVELOPMENT
Knowledge of Infant Development Inventory 3.4 3.3 0.1*** 11.1
DISCIPLINE STRATEGIES
Percentage of Parents Who Suggested Responses to Hypothetical Situations with Child:        
Prevent or Distract 72.1 66.7 5.3*** 11.3
Remove Child or Object 80.5 81.3 -0.8 -2.0
Talk and Explain 37.4 31.5 5.9*** 12.6
Threaten or Command 31.8 34.1 -2.3 -4.8
Shout 5.5 4.7 0.8 4.0
Physical Punishment 27.4 29.9 -2.5 -5.5
Percentage of Parents Suggesting Only Mild Responses to the Hypothetical Situationsr 43.0 39.0 4.0* 8.1
Index of Severity of Discipline Strategies Suggesteds 2.7 2.8 -0.1* -6.8
Sample Size 1,092 1,021 2,113  


SOURCE: Parent interviews, child assessments, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

 

TABLE D.6G
IMPACTS ON SAFETY PRACTICES
Outcome Program
Group
Control
Group
Estimated Impact Per Eligible Applicantr

Effect
Sizes

Family Has Syrup of Ipecac in the House in Case of a Poison Emergency 29.6 29.5 0.1

0.3

Parent/Guardian Has or Knows How to Find the Telephone Number For the Poison Control Center 37.3 35.7 1.6

3.2

Family Uses a Gate or Door at the Top Of Stairs 79.7 81.2 -1.5

-3.8

Family Uses Guards or Gates For Windows 62.7 64.7 -1.9

-4.0

Family Has Covers on Electrical Outlets That Child Can Reach 60.4 60.8 -0.3

 

-0.7

Family's Home Has Working Smoke Alarms 86.6 84.8 1.8

5.1

Family Uses a Car Seat For Child and it is in the Back Seat of the Car 81.0 82.0 -1.0

 

-2.7

Interviewer Observed That Child's Play Area is Safe 68.5 68.5 0.0

0.0

Sample Size 1,092 1,021 2,113  


SOURCE: Parent interviews, child assessments, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

 

TABLE D.6H
IMPACTS ON SELF-SUFFICIENCY ACTIVITIES DURING FIRST 15 MONTHS

Outcome

Program
Group

Control
Group

Estimated Impact Per Eligible Applicantr

Effect
Sizes

ANY SELF-SUFFICIENCY ACTIVITIES

Percentage of Parents Ever Employed or in an Education or Job Training Program

85.0

82.5

2.5*

6.5

Average Hours Per Week Employed at All Jobs and in Any Education or Training

20.1

19.7

0.4

2.6

EMPLOYMENT ACTIVITIES

Percentage of Parents Ever Employed

72.5

71.9

0.6

1.3

Average Hours Per Week Employed at All Jobs

14.7

15.4

-0.7

-4.2

EDUCATION ACTIVITIES

Percentage of Parents Who Ever Participated in an Education or Training Program

48.1

43.7

4.4**

8.9

Average Hours Per Week in an Education Program

5.2

4.1

1.0***

13.4

Sample Size

1,139

1,097

2,236

 


SOURCE: Parent Services Follow-Up Interviews completed an average of 7 and 16 months after random assignment.

 

TABLE D.6I
IMPACTS ON EDUCATION ACTIVITIES AND CREDENTIALS BY THE SECOND FOLLOWUP

Outcome

Program
Group

Control
Group

Estimated Impact Per Eligible Applicantr

Effect
Sizes

TYPES OF EDUCATION ACTIVITIES

High School

12.7

9.6

3.0***

10.4

High School or Alternative

13.2

11.2

2.0*

6.4

Adult Basic Education

3.0

2.9

0.2

0.9

English as a Second Language

2.6

1.3

1.3**

11.1

GED Preparation

7.2

6.8

0.3

1.3

Any Vocational Education

14.9

12.9

2.0

6.0

2-Year College

7.4

6.8

0.6

2.4

4-Year College

4.3

4.8

-0.5

-2.4

DEGREES AND CREDENTIALS RECEIVED

Highest Grade Completed

11.4

11.5

-0.1

-2.4

GED Certificate

9.8

9.7

0.1

0.3

High School Diploma

46.1

45.3

0.8

1.6

Received a High School Degree or GED Between Enrollment and Second Follow-Up

 

24.7

 

24.4

 

0.3

 

0.7

Vocational, Business, or Secretarial Diploma

15.4

14.8

0.5

1.5

Associate's Degree

3.1

3.6

-0.6

-3.0

Bachelor's Degree

3.1

4.0

-0.9

-4.5

Sample Size

1,139

1,097

2,236

 


SOURCE: Parent Services Follow-Up Interviews completed an average of 7 and 16 months after random assignment.

 

TABLE D.6J
IMPACTS ON WELFARE PROGRAM PARTICIPATION DURING THE FIRST 15 MONTHS
Outcome Program
Group
Control
Group
Estimated Impact Per Eligible Applicantr Effect
Sizes
Percentage of Parents Who Received Any Welfare Benefits 65.7 64.8 0.9 1.8
Total Welfare Benefits Received ($) 3,652.1 3,431.9 220.2 5.1
Percentage of Parents Who Received AFDC or TANF Benefits 45.0 43.0 2.0 4.0
Total AFDC Or TANF Benefits Received ($) 1,524.3 1,465.3 59.0 2.5
Average Total Food Stamp Benefits Received ($) 1,308.5 1,288.5 20.0 1.3
Sample Size 1,139 1,097 2,236  


SOURCE: Parent Services Follow-Up Interviews completed an average of 7 and 16 months after random assignment.

 

TABLE D.6K
IMPACTS ON FAMILY INCOME AND RESOURCES
Outcome Program
Group
Control
Group
Estimated Impact Per Eligible Applicantr Effect
Sizes
Percentage of Families With Income Above the Poverty Line at Second Follow-Up 33.9 36.1 -2.3 -4.7
Total Family Resources Scale        
First follow-up 150.2 149.1 1.2 5.4
Second follow-up 153.1 152.3 0.9 4.4
Sample Size 1,139 1,097 2,236  


SOURCE: Parent Services Follow-Up Interviews completed an average of 7 and 16 months after random assignment.

 

TABLE D.6L
IMPACTS ON PARENT HEALTH AND FAMILY FUNCTIONING
Outcome Program
Group
Control
Group
Estimated Impact Per Eligible Applicantr Effect
Sizes
PARENT'S PHYSICAL HEALTH
Parent's Health Status 3.5 3.5 0 2.4
PARENT'S MENTAL HEALTH
Parenting Stress Index (PSI): Parental Distress 25.0 25.9 -0.9** -9.6
PSI: Parent-Child Dysfunctional Interaction 17.0 17.4 -0.5* -7.5
Composite International Diagnostic Interview Short Form (CIDI): Major Depression (average probability) 15.6 15.7 -0.1 -0.2
FAMILY FUNCTIONING
Family Environment Scale: Conflict 1.7 1.7 -0.1* -9.2
Sample Size 1,092 1,021 2,113  


SOURCE: Parent interviews, child assessments, interviewer observations, and assessments of semi-structured parent-child interactions conducted when children were approximately 24 months old.

NOTE: All impact estimates were calculated using regression models, where each site was weighted equally.

rThe estimated impact per eligible applicant is measured as the difference between the regression-adjusted means for all program and control group members.(back)
sThe effect size was calculated by dividing the estimated impact per eligible applicant by the standard deviation of the outcome measure for the control group times 100 (that is, it is the impact expressed as a percentage of the standard deviation).(back)

*Significantly different from zero at the .10 level, two-tailed test
**Significantly different from zero at the .05 level, two-tailed test.
***Significantly different from zero at the .01 level, two-tailed test.

 

APPENDIX TABLE D.6M
IMPACTS ON PARENTING BEHAVIOR AT AGE 2, BY PROGRAM APPROACH IN 1997
(Eligible Applicants)
  Center-Based Programs   Home-Based Programs   Mixed-Approach Programs

Outcome

Program Group Control Group Impact Estimate Per Eligible Applicantt Effect Sizeu   Program Group Control Group Impact Estimate Per Eligible Applicantt Effect Sizeu   Program Group Control Group Impact Estimate Per Eligible Appklicantt Effect
Sizeu

CHILDREN'S COGNITIVE AND LANGUAGE DEVELOPMENT

Bayley Mental Development Index (MDI)

89.0 86.6 2.5* 18.3   91.4 90.4 1.0 7.3   88.2 86.8 1.4 10.5

Percentage with Bayley MDI below 85***v

36.3 45.5 -9.2* -18.9   31.6 32.4 -0.9 -1.8   38.2 44.8 -6.6 -13.6

MacArthur Communicative Development Inventories (CDI): Vocabulary Production

54.2 55.2 -1.0 -4.4   56.1 53.3 2.8* 12.6   57.8 53.6 4.2** 18.7

MacArthur CDI: Sentence Complexity*

8.4 8.7 -0.3 -3.2   8.3 7.7 0.6 7.9   9.2 7.0 2.2*** 26.8

CHILDREN'S SOCIAL-EMOTIONAL DEVELOPMENT

Child Behavior Checklist: Aggressive Behavior Problems

9.4 10.3 -0.9 -16.9   10.4 10.4 -0.1 -1.3   9.8 10.7 -0.8* -15.1

Parent-Child Structured Play: Engagement of Parent

4.4 4.4 0.0 -3.7   4.3 4.3 0.0 1.8   4.3 4.1 0.3** 22.2

Parent-Child Structured Play: Negativity Toward Parent

1.8 1.8 0.0 -1.5   1.7 1.7 0.0 -2.2   1.8 2.0 -0.2* -17.0

Parent-Child Structured Play: Child Sustained Attention with Objects

5.0 5.1 -0.1 -10.7   5.1 5.0 0.0 4.4   5.1 4.9 0.2* 17.5

CHILDREN'S HEALTH

Percentage of Children with Poor or Fair Health***

12.2 10.1 2.1 6.2   11.8 14.6 -2.8 -8.2   11.8 13.8 -2.0 -5.9

QUALITY OF THE HOME ENVIRONMENT AND PARENTING

Parent-Child Structured Play: Parent Supportiveness

4.0 4.0 0.0 -1.6   4.0 3.9 0.1* 12.4   4.1 3.9 0.2** 21.8

Home Observation for Measurement of the Environment (HOME): Support of Cognitive, Language, and Literacy Environment*

10.2 10.3 -0.1 -5.0   10.2 10.0 0.2* 9.8   10.4 10.1 0.4*** 20.6

Parent-Child Structured Play: Parent Detachment

1.4 1.4 0.1 5.7   1.4 1.5 -0.1* -13.8   1.4 1.5 -0.2** -16.6

Knowledge of Infant Development Inventory (KIDI)

3.3 3.4 0.0 -3.7   3.4 3.3 0.1** 15.1   3.4 3.4 0.1** 14.8

PARENT'S MENTAL HEALTH AND FAMILY FUNCTIONING

Parenting Stress Index: Parental Distress

25.3 24.9 0.4 4.3   25.2 26.2 -1.0 -10.1   24.4 26.4 -2.0*** -21.5

Family Environment Scale: Conflict

1.7 1.8 0.0 -4.9   1.7 1.7 -0.1 -11.0   1.7 1.7 0.0 -6.7

PARENT'S PROGRESS TOWARD SELF-SUFFICIENCY

Percentage of Parents Ever in and Education or Training Program During the 15 Months after Random Assignment***

53.0 52.3 0.8 1.5   44.9 39.4 5.6** 11.2   48.4 43.4 5.0 10.2

Average Hours per Week Parents Participated in Education or Training During the 15 Months after Random Assignment

6.6 5.5 1.1 14.6   4.8 3.7 1.1** 14.7   4.7 3.7 1.0* 12.6

Sample Size

                           

   Parent Interview

240 203 443     500 466 966     352 352 704  

   Bayley

203 165 368     428 386 814     279 278 557  

   Parent-Child Interactions

223 172 395     421 373 794     269 274 543  

   Parent Services Interview

234 204 438     537 522 1,059     368 371 739  

 

APPENDIX TABLE D.6N
IMPACTS ON PARENTING BEHAVIOR AT AGE 2, BY PATTERN OF IMPLEMENTATION
(Eligible Applicants)
  Early Implementers   Later Implementers   Incomplete Implementers
Outcome Program Group Control Group Impact Estimate Per Eligible Applicantt Effect Sizeu   Program Group Control Group Impact Estimate Per Eligible Applicantt Effect Sizeu   Program Group Control Group Impact Estimate Per Eligible Appklicantt Effect
Sizeu
CHILDREN'S COGNITIVE AND LANGUAGE DEVELOPMENT
Bayley Mental Development Index (MDI) 91.5 89.4 2.1* 15.5   86.0 84.1 1.8* 13.6   92.2

91.5

0.7

4.8

Percentage with Bayley MDI below 85***c 30.9 37.0 -6.1 -12.5   45.6 51.3 -5.7 -11.6   26.4

30.1

-3.7

-7.6

MacArthur Communicative Development Inventories (CDI): Vocabulary Production 60.2 56.2 4.0** 17.7   52.4 51.3 1.2 5.2   56.0

54.1

1.9

8.4

MacArthur CDI: Sentence Complexity 10.0 8.6 1.5** 18.1   7.1 6.2 0.9 11.2   8.7

8.5

0.2

2.0

CHILDREN'S SOCIAL-EMOTIONAL DEVELOPMENT
Child Behavior Checklist: Aggressive Behavior Problems 9.2 10.5 -1.3*** -22.6   10.5 10.6 0.0 -0.3   9.9

10.5

-0.5

-9.6

Parent-Child Structured Play: Engagement of Parent** 4.6 4.4 0.2** 18.0   4.2 4.1 0.2* 13.4   4.1

4.2

-0.2

-13.6

Parent-Child Structured Play: Negativity Toward Parent 1.6 1.7 -0.1 -13.0   1.7 1.8 -0.1 -5.2