ACF OPRE: Early Head Start Research and Evaluation Project (EHSRE)

D.2 ASSESSING AND CORRECTING FOR THE EFFECTS OF NONRESPONSE TO THE EARLY HEAD START INTERVIEWS AND ASSESSMENTS

In the previous section, we examined the baseline characteristics of program and control group members in the full analysis sample and concluded that they were similar. However, as discussed in Chapter II, not all sample members completed the follow-up interviews and assessments. The response rate was about 70 percent to the 26-month parent services interview (PSI), 70 percent to the 36-month birthday-related parent interview (PI), and 55 percent to the Bayley and video assessments. Furthermore, response rates differed somewhat across sites and subgroups defined by site and family characteristics at baseline. Thus, it was important to test whether program group members who responded to the interviews are fully representative of all program group members, and whether control group members who responded to the interviews are fully representative of all control group members. Furthermore, it was important to test whether the baseline characteristics of respondents in the two research groups differ from each other. If not corrected, the effects of interview nonresponse could lead to two problems:

The impact estimates could be biased. This would occur if the differences in the average baseline characteristics of respondents in the program and control groups were correlated with the outcome variables, and hence, the impact estimates.
The impact estimates might not be generalizable to the study population of eligible families. This would occur if the differences between interview respondents and nonrespondents were correlated with the outcome variables (regardless of whether or not the average characteristics of program group and control group respondents were similar).

In this appendix, we assess the effects of nonresponse and discuss procedures that we used to adjust for potential nonresponse effects.

1. Assessing the Effects of Nonresponse

Our basic approach for assessing the effects of nonresponse to key data sources was to compare the baseline characteristics of (1) respondents in the program and control groups, and (2) respondents to the full sample of respondents and nonrespondents in each research group. We conducted this analysis using data from the HSFIS application and enrollment forms, and with the same methods that we used to compare the baseline characteristics of the full program and control groups (see Appendix D.1). To keep the presentation manageable, we focus our analysis on the 26-month PSI and the 36-month birthday-related interviews and assessments.1 Tables D.2A to D.2D display the following results from the nonresponse analysis, with separate tables displayed for each data source:

Variable distributions for interview respondents, by research status
Significance levels for tests of differences between the characteristics of respondents in the program and control groups
Variable distributions for the full sample of respondents and nonrespondents, by research status
Significance levels for tests of differences between respondents and the full sample of respondents and nonrespondents, by research status

We find some differences in the characteristics of respondents and the full sample of respondents and nonrespondents for each research group and data source. Response rates for the program group were higher in center-based programs than in home-based or mixed-approach programs, and response rates for both research groups were higher in “fully implemented” programs than in programs that were not fully implemented. Response rates increased with the education level of the primary caregiver.

TABLE D.2A

COMPARISON OF THE BASELINE CHARACTERISTICS OF RESPONDENTS AND THE FULL SAMPLE OF RESPONDENTS AND NONRESPONDENTS TO THE 26-MONTH PSI, BY RESEARCH STATUS
Variable	Respondents		Respondents and Nonrespondents
Variable	Program Group	Control Groupa	Program Groupb	Control Groupc
Site Characteristics
Program Approach
Center-based	21.4	20.2	20.2	20.6 *
Home-based	45.4	44.8	46.7	45.6
Mixed	33.3	35	33	33.9
Overall Implementation Pattern
Early implementers	34	36.3	34.5 *	34.8 *
Later implementers	38.1	37	35	35.1
Incomplete implementers	27.9	26.7	30.5	30
Family and Parent Characteristics
Age of Mother at Birth of Focus Child
Younger than 20	38.7	38.9	39	39.5
20 to 25	32.6	33.4	33.2	32
25 or older	28.6	27.8	27.9	28.5
Mother Was Younger than 19 at First Birth	42.1	40.4	42.9	41.2
Highest Grade Completed
Less than 12	45.8	46.2	47.7 *	47.8 *
12 or earned a GED	28.2	29.2	27.3	29.8
More than 12	26	24.6	24.9	22.4
Race and Ethnicity
White non-Hispanic	37.2	38.2	37.3 *	37.1
Black non-Hispanic	35.3	34.1	34.2	35
Hispanic	23.7	22.8	23.8	23.4
Other (Asian or Pacific Islander, American Indian, Eskimo, Aleut)	3.9	4.9	3.9	4.9
Primary Occupation
Employed	24.3	23.2	37.2	23.8
In school or a training program	22.2	21	35.3	21.4
Other	53.5	55.8	23.7	54.7
English Language Ability
Primary language is English	79.7	78.6	79.9	78.1
Primary language is not English but the applicant speaks English well	9.1	10.5	9.6	10.3
Primary language is not English and the applicant does not speak English well	11.2	10.9	10.5	11.6
Living Arrangements
Living with a spouse	25.7	26.9	24.9	25.4 *
Living with other adults	38.5	40.4	38.3	39.1
Living with no other adults	35.9	32.7	36.8	5.5
Adult Male Present in the Household	39.6	40.8	38.1 *	39.1 *
Number of Adults in the Householdd
1	36.8	33.5	37.8	36.6 *
2	50.1	53	49.8	50.8
3 or more	13	13.4	12.4	12.6
Number of Children Less than 5 Years Old in the Household Other than the Focus Child
0	64.8	63.3	64.3	65.1 *
1	26.3	27.9	27	26.8
2 or more	8.9	8.9	8.7	8.1
Number of Children Between 6 and 17 in the Household
0	64.1	66.1	64.3	66.4
1	23	21.1	23.1	21.3
2 or more	12.8	12.8	12.6	12.3
Number of Moves in the Past Year
0	51.1	51.2	49.5 *	49.8
1	28.8	27.1	28.9	28.1
2 or more	20.1	21.7	21.6	22.1
Owns Home	12	12	11.0 *	11.1
Household Income as a Percent of Poverty Level (Percent)
Less than 33	29.5	28.2	30.2	30.0 *
33 to 67	31.7	31.7	32.5	29.2
67 to 99	24.2	26.7	24	26.5
100 or more	14.6	13.5	13.3	14.3
Welfare Receipt
AFDC/TANFe	33.1	33.2	35.6 *	34.7
Food Stamps	45.7	46.8	48.0 *	47.8
Medicaid	75.4	75	76.6 *	74.7
SSI	6.4	7.1	7	7
WIC	87.1	86.6	87.5	85.9
Public housing	9.6	9	9.5	8.9
Has Inadequate Resources
Food	4.3	7.4 *	4.9 *	6.3 *
Housing	11.8	12.5	12.3	13.3
Money to buy necessities	20.3	21.5	20.8	21.7
Medical care	12.7	14.5	14.0 *	14.7
Transportation	21.1	23	20.9	22.4
Child care	34.2	34.8	34.4	34.6
Money for supplies	25.5	30.9 *	27.1 *	29.4
Support from friends	12.1	12.8	12.9	14.0 *
Parent information	12.4	15.2 *	12.5	16.3
Maternal Risk Indexf
0 or 1 (low risk)	21	17.8	19.0 *	17.4
2 or 3 (moderate risk)	53.5	56.9	54.2	56.5
4 or 5 (high risk)	25.5	25.2	26.8	26
Random Assignment Date
Before 10/96	35.7	35.4	36	36.5
10/96 to 6/97	30.3	32.3	30.2	30.8
After 6/97	34	32.2	33.8	32.7
Previously Enrolled in Head Start or
Another Childhood Development Programe	12.3	14.1	12.8	13.4
Characteristics of Focus Child
Age (Months)
Unborn	25	27.5	24.2	26.5
Less than 5	34.8	34.2	36.1	34.7
5 or more	40.2	38.3	39.7	38.7
Male	50.7	50.2	51.7	50.4
First Born	62.6	60.6	62.3	62.8 *
Birthweight Less than 2,500 Gramse	8.9	7.8	9.9	8.4
Born more than 3 Weeks Earlye	14.8	11.5 *	15.8	12
Stayed in Hospital After Birthe	17	15.8	18.3 *	16
People Concerned About the Child's Overall Health and Developmente	12.3	14.6	13	13.3 *
Received an Evaluation Because of Concerns About the Child's Overall Health and Development or Because of Suspected Developmental Delaye	5.3	6.4	6	6.9
Risk Categories
Has established risks(e)	11	10.5	11.6	10.6
Has biological or medical riskse	17.6	16.5	18.3	16.8
Has environmental riskse	31.8	38.1 *	32.5	36.4 *
Covered by Health Insurancee	91.3	91.4	90.1 *	89.6 *
Sample Size	1,076	1,011	1,513	1,488

SOURCE: HSFIS application and enrollment forms and 26-month PSI data.

aSignificance levels are from tests comparing program and control group respondents.(back)

bSignificance levels are from tests comparing respondents and the full sample of respondents and nonrespondents in the program group.(back)

cSignificance levels are from tests comparing respondents and the full sample of respondents and nonrespondents in the control group.(back)

dThe primary caregiver is considered to be an adult regardless of her age.(back)

eThese variables pertain to families with focus children who were born at baseline.(back)

fThis index was constructed by summing the number of the following risk factors that the mother faced: (1) being a teenage mother; (2) having no high school credential; (3) receiving public assistance; (4) not being employed or in school or training, and (5) being a single mother.(back)

*Significantly different from zero at the .10 level, two-tailed test.

TABLE D.2B

COMPARISON OF THE BASELINE CHARACTERISTICS OF RESPONDENTS AND THE FULL SAMPLE OF RESPONDENTS AND NONRESPONDENTS TO THE 36-MONTH PI, BY RESEARCH STATUS
Variable	Respondents		Respondents and Nonrespondents
Variable	Program Group	Control Groupa	Program Groupb	Control Groupc
Site Characteristics
Program Approach
Center-based	22.9	21	20.2 *	20.6
Home-based	45.3	44.7	46.7	45.6
Mixed	31.7	34.3	33	33.9
Overall Implementation Pattern
Early implementers	35	35.7	34.5 *	34.8 *
Later implementers	37.8	36.1	35	35.1
Incomplete implementers	27.2	28.2	30.5	30
Family and Parent Characteristics
Age of Mother at Birth of Focus Child
Younger than 20	37.9	38.7	39.0 *	39.5
20 to 25	33	32.6	33.2	32
25 or older	29.1	28.7	27.9	28.5
Mother Was Younger than 19 at First Birth	42.3	40.6	42.9 *	41.2
Highest Grade Completed
Less than 12	45.3	45.5	47.7 *	47.8 *
12 or earned a GED	29.2	29.1	27.3	29.8
More than 12	25.5	25.4	24.9	22.4
Race and Ethnicity
White non-Hispanic	39.6	39.9	37.3 *	37.1 *
Black non-Hispanic	32.5	33.9	34.2	35
Hispanic	23.8	21.6	23.8	23.4
Other (Asian or Pacific Islander, American Indian, Eskimo, Aleut)	4.1	4.6	4.7	4.5
Primary Occupation
Employed	25.2	23.8	22.9 *	23.8
In school or a training program	21.8	20.8	22	21.4
Other	52.9	55.4	55	54.7
English Language Ability
Primary language is English	80.2	80.1	79.9	78.1 *
Primary language is not English but the applicant speaks English well	8.9	10.2	9.6	10.3
Primary language is not English and the applicant does not speak English well	10.9	9.7	10.5	11.6
Living Arrangements
Living with a spouse	26	27	24.9	25.4
Living with other adults	38.4	38.9	38.3	39.1
Living with no other adults	35.6	34.2	36.8	35.5
Adult Male Present in the Household	39.8	40.8	38.1 *	39.1 *
Number of Adults in the Householdd
1	36.8	35.1	37.8	36.6
2	50.3	51.5	49.8	50.8
3 or more	12.9	13.4	12.4	12.6
Number of Children Less than 5 Years Old in the Household Other than the Focus Child
0	64.7	63.3	64.3	65.1
1	26.8	28.4	27	26.8
2 or more	8.5	8.3	8.7	8.1
Number of Children Between 6 and 17 in the Household
0	63.1	65.6	64.3	66.4
1	24.4	21.3	23.1	21.3
2 or more	12.6	13	12.6	12.3
Number of Moves in the Past Year
0	50.8	51.3	49.5	49.8
1	28.1	27.8	28.9	28.1
2 or more	21.1	20.8	21.6	22.1
Owns Home	12.2	12.3	11.0 *	11.1 *
Household Income as a Percent of the Poverty Level (Percent)
Less than 33	29.2	28	30.2 *	30
33 to 67	31.3	30.4	32.5	29.2
67 to 99	25.1	27.4	24	26.5
100 or more	14.3	14.1	13.3	14.3
Welfare Receipt
AFDC/TANFe	32.7	33.4	35.6 *	34.7
Food Stamps	45.9	46.4	48.0 *	47.8
Medicaid	75.8	74.7	76.6	74.7
SSI	6.8	7.2	7	7
WIC	87.5	86	87.5	85.9
Public housing	10	8.6	9.5	8.9
Has Inadequate Resources
Food	4.6	6.9 *	4.9	6.3
Housing	11.9	12.1	12.3	13.3 *
Money to buy necessities	19.7	20.4	20.8 *	21.7 *
Medical care	13.4	14	14	14.7
Transportation	20.7	22.1	20.9	22.4
Child care	33.3	34.2	34.4	34.6
Money for supplies	25.1	30.2 *	27.1 *	29.4
Support from friends	12.2	11.8	12.9	14.0 *
Parent information	12.9	15.4 *	12.5	16.3
Maternal Risk Indexf
0 or 1 (low risk)	21.2	19.2	19.0 *	17.4 *
2 or 3 (moderate risk)	54.3	55.9	54.2	56.5
4 or 5 (high risk)	24.5	25	26.8	26
Random Assignment Date
Before 10/96	35.3	35.6	36.0 *	36.5
10/96 to 6/97	28.5	31.3	30.2	30.8
After 6/97	36.2	33.1	33.8	32.7
Previously Enrolled in Head Start or Another Childhood Development Programe	13.1	14	12.8	13.4
Characteristics of Focus Child
Age (Months)
Unborn	23.9	25.6	24.2	26.5
Less than 5	35.2	35.1	36.1	34.7
5 or more	40.8	39.3	39.7	38.7
Male	51	50.1	51.7	50.4
First Born	61.7	60.9	62.3	62.8 *
Birthweight Less than 2,500 Gramse	9.3	7.3	9.9	8.4 *
Born more than 3 Weeks Earlye	14.8	11.5 *	15.8	12
Stayed in Hospital After Birthe	17.4	16.2	18.3	16
People Concerned About the Child's Overall
Health and Developmente	12.7	14.4	13	13.3
Received an Evaluation Because of Concerns About the Child's Overall Health and Development or Because of Suspected Developmental Delaye	5.5	6.4	6	6.9
Risk Categories
Has established riskse	11.6	10.2	11.6	10.6
Has biological or medical riskse	17.4	16.9	18.3	16.8
Has environmental riskse	31.9	36.6 *	32.5	36.4
Covered by Health Insurancee	91.4	92.3	90.1 *	89.6 *
Sample Size	1,107	1,003	1,513	1,488

SOURCE: HSFIS application and enrollment forms and 36-month PI data.