US Department of Health and Human Services US Department of Health and Human Services US Department of Health and Human Services US Department of Health and Human Services US Department of Health and Human Services US Department of Health and Human Services
Department of Health and Human Services logo US Department of Health and Human Services Skip ACF banner navigation
US Department of Health and Human Services Questions?  
US Department of Health and Human Services Privacy  
US Department of Health and Human Services Site Index  
US Department of Health and Human Services Contact Us  
US Department of Health and Human Services Download Acrobat® Reader™  
US Department of Health and Human Services   ACF Home   |   Services   |   Working with ACF   |   Policy/Planning   |   About ACF   |   ACF News Search  
US Department of Health and Human Services US Department of Health and Human Services US Department of Health and Human Services
Administration for Children and Families US Department of Health and Human Services
Office of Planning, Research & Evaluation (OPRE) skip to primary page content
Advanced
Search

     

Unique Features of the Early Head Start Research and Evaluation Project for Assessing Impacts of an Infant-Toddler Intervention

Early Head Start research and evaluation has benefited from earlier generations of intervention studies, including the evaluation of the Comprehensive Child Development Program (CCDP), and has been able to incorporate many methodological lessons from those studies. Several unique features will enable the Early Head Start study to answer questions many previous evaluations have been unable to address. Seven unique features are described here:

  • Identification of Effects as a Function of Program Model. Early Head Start program approaches vary in important ways. For example, some programs deliver child development services through child care and others use a home visitation approach. These are appropriate alternatives in the context of different community needs, populations, and geography. The early intervention literature suggests that these variations will result in different patterns and timings of effects.

  • Assessment of Implementation. Programs were ranked according to how well they implement the Head Start Performance Standards, using a consensus-based rating system in which a trained panel of experts reviews implementation data obtained from evaluation site visits. (The Advisory Committee on Services for Families with Infants and Toddlers recommended taking level of program implementation into account.) Only by systematically assessing program implementation against the Performance Standards is it possible to determine whether the program is being carried out in each site as it was designed.

  • Assessment of the Quality and Quantity of Child Development Services. This study assessed both the quality and the quantity of key services for children, including child care and home visits. In an age where child care is noted to be largely poor in quality, it is important to know the quality (as well as the quantity) of child care received by both the program and comparison group families. It is also important to measure the degree of child development emphasis in home visits and to create an understanding about the quality and quantity of home visits. Low-quality, infrequent visits that lack a child development focus would not be expected to have an impact on children, but high-quality, frequent visits oriented toward child development would be expected to have a positive impact on child development. Similarly, large doses of poor-quality care or small doses of high-quality care would not be expected to have an impact on children, but moderate to large amounts of high-quality child care would be expected to have a positive impact. In general, previous evaluations have not examined in detail the effects on children in light of different configurations of these child development service variables.

  • Assessment of Services Received by Both the Comparison Group and the Program Group. The availability of services varies from community to community. For example, some communities are strong in health services but lack employment opportunities. Because different services are available to and actually used by the program and comparison groups, different patterns of outcomes would be expected in specific sites. The team assessed community services available to the comparison group to make predictions on a site-by-site basis about the availability of resources within the community. The evaluation team also directly measured the service use by both comparison and program groups, which provided family-level data on service use for the impact analysis.

  • Analysis of Outcomes by Actual Program Participation. Program directors often state that they do not expect the program to affect the families who drop out. They expect impacts on those families with whom they had consistent and continuous contact over a period of several years. The evaluation team conducted a rigorous impact analyses comparing all families randomly assigned to program and comparison groups; then, the team made statistical adjustments for the different levels of program participation. For example, team members estimated the program impacts on those children and families who received the full program dosage.

  • Implementation and Impact Study by the Same Research Team. In many program evaluations, the implementation and the impact studies are conducted by two different research organizations. This approach makes it difficult for the impact researchers to interpret site-level effects or to effectively use program implementation data in the impact analysis. In the Early Head Start study, these functions were carried out by the same organization; which will facilitate the use of program process data in the impact analysis.

  • Partnerships with Local Researchers. Too often, local program interpretations are missing from national evaluations. A researcher with knowledge of the local program and community can add interpretations to local findings that might not be apparent at the national level and provide local researchers' further interpretation of site-specific findings in a way that is useful to the programs.

  • Timing of the Early Head Start Evaluation. One might argue that the Early Head Start evaluation was conducted too soon to detect strong program impacts. However, there is much to be learned about the early implementation of new programs as they face multiple challenges in moving to new approaches for serving low-income families with infants and toddlers and in adapting program models in response to welfare reform and the changing needs of families within Early Head Start communities. The dynamic state of new Early Head Start programs may be a factor in their early effectiveness, especially in the context of welfare reform. However, a strong, multifaceted evaluation with both implementation and impact analyses is the correct choice for facilitating continuous improvement from the lessons learned.