Table of Contents | Previous | Next |
B.2 SOURCES OF NONRESPONSE
All multisite evaluations of the size and complexity of Early Head Start face a variety of data collection and analytic challenges that affect the overall and site-level response rates. This study is no different. Overall response rates, response rates by site and by data source, and response rates by evaluation subgroups are presented and discussed in Chapter II. Here we describe the nature of the nonresponse.
The primary sources of nonresponse were refusals to participate and inability to locate the families. Overall for the 15-month PSI, 36 percent of the families who did not respond refused to participate, and 40 percent moved or could not be located (the remaining 24 percent included families for whom the interview window closed before the interview was completed or the interview was conducted after our cutoff for inclusion in this report).1 For the 24-month PI, 43 percent of the families who did not respond refused to participate, and 36 percent moved or could not be located (the remaining 21 percent included families for whom the interview window closed before the interview was completed or the interview was conducted after our cutoff for inclusion in this report).
Site coordinators reported that the data collection was very challenging. From the beginning of the project, some site coordinators reported that some families had not understood what they were signing up for (related to the program, the research activities, or both), and some site coordinators reported that control group families refused participation in the study after they learned that they were not going to receive Early Head Start services.
Analysis of the categories of nonresponse by site showed that the center-based sites were more successful in completing interviews and assessments with Early Head Start families than they were with the control group families. One explanation for this is that the Early Head Start families were using center-based services and may have been easier for research and program staff members to contact. To some degree, the same pattern might have been expected across all the programs--if the local research team used all available leads, they may have been able to contact and successfully complete interviews with a larger proportion of the Early Head Start group than the control group. This was not true across all sites, and in a number of sites research teams completed a larger proportion of the interviews with control group families. The national team is continuing to work with the local research teams and the program directors to better understand this variation across sites and to provide a description of the challenges the local research teams faced in completing the interviews and assessments.
In general, the PI response rate establishes the maximum for the Bayley and parent-child structured play assessment response rates. This is because if an interview was not done, it was generally the case that the other assessments also were not done. In some sites, IAs completed the PI by telephone if the interview window was about to close or if the family moved away, rather than lose the entire data collection wave for the family. In those cases it was impossible to conduct the Bayley and the parent-child structured play assessment. Sites reported other data collection-related reasons for nonresponse on the Bayley and the parent-child structured play assessment, including child illness on the interview date, child refusal to participate in the Bayley assessment or the play assessment, parental refusal to participate in the play assessment, and insufficient time during the visit to complete the assessments.
Some of the data that were collected could not be used because of technical problems or errors in administration of the assessment. Between three and seven percent of the 1,807 24-month videotapes sent to Columbia for coding could not be coded because of incorrect administration of the parent-child structured play assessment, lack of video or sound, or other technical problems. Nine percent of the 1,903 24-month Bayley assessments conducted could not be scored because of errors in administration of the test or the lack of a basal. Appendix D includes information about how we adjusted for nonresponse in our analyses.
1 A small number of 24-month birthday-related interviews and assessments, as well as 15-month PSIs, were received by MPR after the cutoff date for inclusion in the analysis files. These data will be added to the data files and included in the longitudinal analyses to be reported in the final report, June 2002.(back)
| Table of Contents | Previous | Next |