Skip Navigation
Administration for Children and Families  
ACF
ACF Home   |   Services   |   Working with ACF   |   Policy/Planning   |   About ACF   |   ACF News   |   HHS Home

  Questions?  |  Privacy  |  Site Index  |  Contact Us  |  Download Reader™  |  Print      

Office of Planning, Research & Evaluation (OPRE) skip to primary page content
Advanced
Search

 Table of Contents | Previous | Next

Executive Summary

Introduction

The Head Start program, which provides comprehensive early childhood development services to low-income children and their families, has experienced significant growth over the last decade, especially as greater attention has been paid to the need for early intervention. During this period, the U.S. General Accounting Office (GAO) released two reports underlining the lack of rigorous research on Head Start’s effectiveness, i.e., ”. . .the body of research on current Head Start is insufficient to draw conclusions about the impact of the national program,”1 and, as a consequence, “. . .the Federal government’s significant financial investment in the Head Start program, including plans to increase the number of children served and enhance the quality of the program, warrants definitive research studies, even though they may be costly.”2

Based upon the GAO recommendation, and the testimony of research methodologists and early childhood experts, Congress mandated through the 1998 reauthorization of Head Start3 that the Department of Health and Human Services determine, on a national level, the impact of Head Start on the children it serves.4 In October 2000, DHHS awarded a contract to Westat in collaboration with The Urban Institute, the American Institutes for Research, and Decision Information Resources to conduct the Head Start Impact Study.

The design of the study is built upon the report of the Advisory Committee on Head Start Research, Evaluating Head Start: A Recommended Framework for Studying the Impact of the Head Start Program. This report set forth a framework for research on the impact of Head Start that is both scientifically credible and feasible. The report also noted that the legislative mandate clearly requires that the impact study must address two main questions. First, what difference does Head Start make to the key outcomes of development and learning (and, in particular, the multiple domains of school readiness) of the nation's low-income children? Second, under what circumstances does Head Start work best and for which children? In addition, the Committee acknowledged that the legislative language recommended the use of a rigorous methodology, including random assignment of children to Head Start and non-Head Start groups at a diverse group of sites, selected nationally and reflecting the range of Head Start quality across the country. The end result is that the Head Start Impact Study is one of a small handful of randomized, nationally representative studies ever conducted.

Study Overview

The Head Start Impact Study is a longitudinal study involving approximately 5,000 3- and 4-year-old newly entering children who were eligible for Head Start and who applied in fall 2002 to 383 centers across 84 nationally representative grantee/delegate agencies.5 The participating children have been randomly assigned to either a Head Start group (that receives Head Start services) or to a control group (that does not receive Head Start services but may enroll in other available services selected by their parents or may be cared for at home). Every effort was made to minimize the burden on individual programs and not to significantly change typical enrollment and recruitment procedures.

Children enrolled in Early Head Start, Migrant Head Start and programs operated by Tribal Organizations, those considered extremely new (i.e., in operation less than 2 years), and those considered severely out of compliance were not included in the study.

Great care was taken to include only programs that were not currently serving all of the eligible children in their community. This was to ensure the availability of a sufficient number of unserved, eligible children available who could be randomly assigned to a control group, without causing the program to serve any fewer children than would otherwise be the case. These "saturation" determinations were based on grantee/delegate agencies' own reports of enrollment levels in fall 2001, along with other available information.

Data Collection

Data collection began in fall 2002 and is scheduled to continue through 2006, following children through the spring of their 1(st) grade year. It includes in-person interviews with parents, in-person child assessments conducted twice during the first year of the study and annually afterwards, annual surveys with care providers and teachers, direct observations of the quality of different care settings, and teacher ratings of individual study children. The data to be collected include the following:

  • Individual child data in areas related to school readiness, such as approaches to learning, language usage and emergent literacy, cognition and general knowledge, physical well-being and motor development, and social and emotional development.
  • Information pertaining to: parenting practices, including parents' descriptions of the types of literacy activities they engage in with children at home; family resources and risk factors; and family demographic and socio-economic characteristics.
  • Information on the structure, process, and quality of Head Start, child care, and school settings through first grade, including teachers' reports on their credentials and experience and structured observations of the quality of different care settings, including assessments of classroom resources and instructional practices.
  • Community-level data relating to the availability of formal and informal family support services.

Status Update and Reports

The Head Start Impact Study began in October 2000 with initial activities focused on developing the study design, recruiting Head Start grantee/delegate agencies, and field testing study measures and procedures. The selection and recruitment of participating Head Start centers, families, and children continued through spring 2002, culminating with the collection of child, family, and program data in fall 2002 and spring 2003. Activities are on schedule for the planned final report of findings in December 2006. Accomplishments to date include:

  • During the 2000-01 Head Start year, conducted an advance field test involving eight Head Start grantees, 21 Head Start centers, and approximately 400 children to test the feasibility of all study procedures.
  • During the 2001-02 Head Start year, completed the selection and recruitment of a national probability sample of 84 grantee/delegate agencies and 378 Head Start centers to participate in the national study.
  • During summer 2002, completed random assignment, in all recruited centers, resulting in 4,750 children being randomly assigned to either a Head Start treatment group or a non-Head Start control group.
  • Completed fall 2002 data collection, with high overall response rates for both child assessments and parent interviews with a combined rate of 80 percent for fall 2002.
  • Completion of data entry and checking for errors, as well as statistical weighting of the fall 2002 data. The initial assessment of the psychometric properties of the child assessments is currently underway.
  • Completion of spring 2003 data collection with approximately 83 percent response rates for both child assessments and parent interviews.
  • Have started data entry and checking for errors, as well as statistical weighting. Initial assessment of psychometric properties for the spring 2003 data collection will continue on an ongoing basis following the completion of this process for the fall 2002 data.
  • A meeting was held of the Advisory Committee on Head Start Research on June 16 and 17, 2003, to update the committee on the progress of the study activities and to seek additional input and recommendations on the remaining work of the study. A summary of the Advisory Committee meeting can be found at http://www.acf.hhs.gov/programs/hsb/research/hsreac/index.htm

Overall, Advisory Committee members commented positively on the research design of the Impact Study and the success of implementation to date, especially given the complexity of the Congressional mandate and the ethical concerns of randomly assigning low-income children to a control group. Several members noted that when completed, the Impact Study will provide a rich array of data that will not only meet the Congressional requirements, but will also allow for an extensive range of secondary data analyses.

A final report of the study findings is scheduled for December 2006.



1U.S. General Accounting Office (1997). Head Start: Research Provides Little Information on Impact of Current Program. Washington DC: Author.(back)


2U.S. General Accounting Office (1998). Head Start: Challenges in Monitoring Program Quality and Demonstrating Results. Washington DC: Author.(back)

3Section 649(g) of the Head Start Act, as amended by the Coats Human Services Reauthorization Act of 1998 (P.L. 105-285).(back)

4See Appendix A for the research-related amendments to the Head Start Act included in the 1998 reauthorization.(back)

5 The study sample includes both Head Start grantees and their delegate agencies. Grantees are organizations that have fiscal and administrative responsibility for programs in their jurisdiction. In some cases, they can subcontract with agencies to handle administrative oversight over some or all of these programs. Throughout this report we use the term grantee/delegate agencies to refer to both types of agencies.(back)

 

 Table of Contents | Previous | Next