Abstract

This document contains resources to help Head Start programs that serve pregnant women and families with infants and toddlers develop a performance measurement plan and carry out data collection that will support their continuous program improvement efforts. These performance measures activities should build upon existing screening and assessment activities required by the Head Start Program Performance Standards. This document discusses the importance and development of a comprehensive plan and presents profiles of instruments that may be useful to programs. Because we anticipate that it will be used under different circumstances for different purposes, we present the background information in a book format, and the entire document on a compact disk, to enable users to search for the sections and measures that apply to them. This format will also support the continued evolution of the document, which is intended to be a “living” document to which information on new instruments can be added, through which new tools and approaches can be shared, and in which other resources that individual programs find useful can be compiled. These materials will be most useful when used in consultation with an assessment expert.

Head Start, as a national laboratory for early childhood education, has been a leader in developing outcomes-oriented accountability. It began developing performance measures in 1995, partly to respond to the Government Performance and Results Act (GPRA) requirements. In 1997, the Head Start Bureau launched the Family and Child Experiences Survey (FACES) to collect data on child and family outcomes, as well as program services and management systems, for a large nationally representative sample of children and families in Head Start programs (Administration on Children, Youth and Families 2001b). This study links the development of children and families with their experiences in Head Start. Following the reauthorization of Head Start in 1998, Head Start programs were required to include child outcomes in their self-assessment process by 2003.

Although Early Head Start programs have not been required to report child outcomes, many have started to try to define and measure outcomes, for several reasons. Some programs are doing so in conjunction with performance measurement in Head Start because they operate within Head Start programs. Some are responding to other funders’ requirements. Finally, some simply want to improve their services to families with infants and toddlers. In Spring 2001, the Early Head Start Technical Work Group recommended that the Head Start Bureau move forward to develop performance measures for Head Start programs serving infants and toddlers (Early Head Start and Migrant Head Start programs) to support programs’ efforts.

The development of new performance measures for Head Start programs serving pregnant women and families with infants and toddlers has two purposes. These are (1) to create performance measures sensitive to the developmental stages from pregnancy to age 3 that can be used at both national and local levels for learning about child and family experiences along with program services and management systems; and (2) to provide guidance to local programs in defining and measuring outcomes and using this information for continuous program improvement.

To fulfill these purposes, the Head Start pyramid, representing the Head Start performance measures framework, was expanded to reflect the unique features of programs serving infants and toddlers (see Figure 1 and Appendix C). The ultimate goal of Head Start programs that serve pregnant women and families with infants and toddlers is the same as that of Head Start programs that serve preschool children—enhancing children’s competence.1 The main outcomes supporting this ultimate goal—the blocks in the pyramid—have been expanded to reflect the central role of relationships in supporting attainment of this goal in families with younger children (U.S. Department of Health and Human Services [DHHS] 1994). Enhancing children’s growth and development, enhancing parent-child relationships, and strengthening families as primary nurturers of their children are outcomes that support the ultimate goal of enhancing children’s competence. Supporting these outcomes are objectives related to program services, including the objectives of providing children with individualized services, developing relationships with parents and children, and linking children and families to needed community services. The objective of ensuring well-managed programs that meet standards for high quality in supporting staff, providing early childhood environments, involving parents, and developing community partnerships serves as the foundation of the pyramid. The pyramid rests on the four cornerstones recommended by the Advisory Committee on Services for Families with Infants and Toddlers—staff development, child development, family development, and community building (DHHS 1994).

The national Early Head Start Research and Evaluation Project (EHSRE) studied early program implementation and impacts on children and families (see Box 1). From this study, we learned that each program operates with its own theory of change—that is, a theory explaining how the services they provide will improve the child and family outcomes they are focusing on to meet the needs of children and families in their community. Although all programs must implement all aspects of the pyramid, the ways that programs configure their management systems and program services (the bottom levels of the pyramid) vary widely. The specific child and family outcomes they focus on (the upper levels of the pyramid) also vary. As the program staff learns from continuous program improvement activities or as families’ needs change, programs’ theories of change may evolve. The evaluation showed that in general, patterns of program impacts reflected differences in theories of change. Home-based programs, which emphasized improving parenting and the home environment as an important path to improved child outcomes, had significant favorable impacts on parenting and child outcomes. Center-based programs, which emphasized direct services to children to improve outcomes, had favorable impacts on children and fewer significant impacts on parenting. The study also showed that programs that fully implemented key elements of the Head Start Program Performance Standards (elements related to all levels of the pyramid) had more favorable impacts on a wide range of outcomes than programs that were incompletely implemented (Administration for Children and Families 2002).

The pyramid provides a framework to help programs identify their own theory of change. Together, the pyramid and theory of change provide guidance for examining links between program services and outcomes to inform continuous program improvement activities. The performance measures framework is general and does not identify specific program services and outcomes that should be measured. Individual programs must identify specific services and outcomes based on their own theory of change and select appropriate instruments to fit their available resources.

We do not recommend that programs collect data related to every element of the pyramid (at least to start with). We recommend that programs collect data related to each level of the pyramid (management systems, services, and outcomes), because having such information allows staff to link information on program characteristics and services with outcomes for children and families and to learn about both how well children and families are doing and how services might be improved to promote better outcomes.

Selecting appropriate measures is an important and complex process. This document is intended to be a resource for programs undertaking this process. It includes both screening instruments and assessment instruments that may be useful to programs as they explore how to approach performance measurement. We attempted to identify a wide range of instruments and select those likely to be most useful to Head Start programs serving pregnant women and families with infants and toddlers, but there are useful tools that are not included in this review. For example, some kinds of assessment that are useful for individualizing services, such as portfolio assessment, are not included (Martin 1999). In addition, new measures are constantly emerging, and very recent measures may not be found here. Some programs may wish to use this resource document in consultation with an expert on screening and assessment to help them develop a plan and select instruments. (For more explanation of how we selected instruments for this resource document, see page 17.)

This resource document will be most useful to programs if it is used in conjunction with a comprehensive plan for performance measurement. In the next section, we discuss the importance of developing a plan and the elements that make up such a plan. In the final section, we describe how instruments were selected for this resource document, explain the information provided for each instrument, and present profiles of instruments that may be useful to Head Start programs that serve pregnant women and families with infants and toddlers.

Box 1: MAKING A DIFFERENCE IN THE LIVES OF INFANTS AND TODDLERS AND THEIR FAMILIES: THE IMPACTS OF EARLY HEAD START

A rigorous evaluation of Early Head Start services in 17 programs selected from the first groups of programs funded showed they had significant favorable impacts on a wide range of parent and child outcomes, some with implications for children’s later school success. Findings from the study (Making a Difference in the Lives of Infants and Toddlers and Their Families: The Impacts of Early Head Start), using data gathered when children were age 3 and had completed the program, show that the programs sustained and broadened the pattern of impacts reported when children were age 2 (Building Their Futures: How Early Head Start Programs Are Enhancing the Lives of Infants and Toddlers in Low-Income Families, 2001). All Early Head Start evaluation reports are available online at [http://www.acf.hhs.gov/programs/opre/ehs/ehs_resrch/index.html].

Early Head Start Improved Outcomes. The national evaluation conducted by Mathematica Policy Research, Inc. and Columbia University’s Center for Children and Families at Teachers College, in collaboration with the Early Head Start Research Consortium, reported that 3-year-old Early Head Start children performed significantly better on measures of cognitive, language, and social-emotional development than a randomly assigned control group. While children who participated in Early Head Start performed better then their peers who did not receive Early Head Start on all aspects of development that were assessed, both groups lagged behind 2- and 3-year-old children nationally. This may, in part, be due to the fact that Early Head Start programs must reserve at least 10 percent of their slots for children with disabilities, including those with developmental delays who score at the lower end of the distribution. Parents in Early Head Start scored significantly better than control group parents on measures of many aspects of the home environment and parenting behavior. Furthermore, Early Head Start programs enhanced parents’ progress toward self-sufficiency years. Early Head Start fathers benefited as well. Although the overall impacts of Early Head Start were generally modest, the pattern of favorable findings across outcomes in a wide range of key domains is promising.

Full Implementation Was Important. The impacts on children and parents were consistent with the substantial difference the programs made in families’ receipt of services. Early Head Start families were significantly more likely than control families to receive a wide variety of services, but especially to receive intensive services, and to receive intensive services that focused on child development and parenting.

Implementing the Head Start Performance Standards early and well is important for maximizing impacts on children and families. The research programs were systematically rated according to the extent to which they implemented key elements of the Performance Standards. Those that fully implemented the standards demonstrated a broader pattern of significant impacts than did the programs that did not reach full implementation during the evaluation period.

Patterns of Impacts Were Consistent with Theories of Change. Analyses that compared the contribution of impacts on parenting when children were age 2 to impacts on child outcomes at age 3 in programs providing home-based or a mix of home- and center-based services generally provided support for the theories of change that staff in those programs described, with some of the effects of the program on 3-year-old children being associated with the effects on parenting when they were age 2.