Search Type Title Author/Investigator Search Advanced Search

Return to Previous page

PDF Version, B&W

Executive Summary

In March 2001, the Head Start Bureau and the Commissioner's Office of Research and Evaluation in the Administration on Children, Youth and Families1 awarded eight cooperative agreements under the Head Start Quality Research Center (HSQRC) Consortium to promote the school readiness of preschool children in Head Start. The mission of the HSQRC Consortium is to support the continuous improvement of Head Start by developing, testing, refining, and disseminating interventions to enhance child outcomes in Head Start. To achieve the Consortium’s goal, the eight research organizations have been awarded five-year grants to work in partnership with local Head Start programs.

The HSQRCs were awarded grants to improve child outcomes in the areas of literacy, social-emotional development, and other domains of school readiness. Their interventions include enhancements to curriculum, teacher training and mentoring, parent involvement, and assessment practices. In the first year of the research, each partnership of researchers and Head Start programs launched studies to test individual interventions and the feasibility of using a common set of measures across sites in a pre-post design. These interventions were continued in the programs during the second year of the study while the effectiveness of the individual interventions was tested using common measures in a treatment/control design. In some cases, projects initiated experimental designs starting in year one. Therefore, for some interventions, the outcome data for treatment and control groups are available for two program years.

While these eight interventions vary in many ways, some of them have a similar focus, such as enhancing literacy development or providing curriculum training for teachers. Based on the respective classifications made by each HSQRC member for their own intervention, the eight interventions were classified into five “intervention types” for the purpose of the analyses presented in this interim report. Two of the interventions (Columbia University and Quality Counts, Inc.) were included in multiple categories:

1. Literacy-focused (Education Development Center and State University of New York at Stony Brook)
2. Socio-emotional-focused (University of North Carolina at Chapel Hill and University of Oregon)
3. Teacher training-focused (Columbia University, High/Scope Educational Research Foundation, and Quality Counts, Inc.)
4. Individualizing assessment-focused (Columbia University and Quality Counts, Inc.)
5. Parent Involvement-focused (University of South Carolina)

This report presents interim findings on data from the first two cohorts of data collected across all eight projects by the Data Coordinating Center (DCC). The DCC data were collected by contractors trained in the administration of the direct child assessment measures. In addition to the DCC data, each HSQRC collected data from an additional sample of Head Start children at their site (“non-DCC” sample). The same measures (e.g., direct child assessment, parent interview, teacher-child report) were used to collect data from the non-DCC sample. However, the procedures for collecting those data varied considerably across the eight sites. For example, in some sites HSQRC research staff were trained by the contractors on the direct child assessment procedures to collect data on the non-DCC sample. In other sites, a “train-the-trainers” approach was used. HSQRC staff were trained by the contractors to train other staff to administer the direct child assessment to the non-DCC sample of children. In both cases, HSQRC research staff, rather than contractor staff, collected the direct child assessment data. Due to this issue, the findings presented in this report are limited to analyses with the DCC sample rather than the larger combined sample.

Analyses examined differences in child and family demographics, child school readiness skills, classroom quality, and parenting skills between the children, families, and classrooms in the HSQRC sample and the nationally representative sample from FACES 2000. The analyses then examined differences between the respective treatment and control groups, as well as between the respective treatment groups and data from the nationally representative sample from FACES 2000.

The analyses presented in this report are designed to examine the impact of these types of interventions on the outcomes of children and families participating in Head Start. Fall to spring growth of the treatment and control group children is compared to estimate the relative impacts of the interventions, addressing the primary research question of the HSQRC effort. Further, analyses are also conducted to explore the generalizability of these findings to Head Start programs across the nation (as opposed to being applicable only to the HSQRC sample). Knowing how the HSQRC samples compare with the overall Head Start population is important for evaluating the generalizability of the findings.

Unless otherwise specified, results were examined for the whole sample together and then separately for the five intervention type categories. The questions are as follows:

1. Demographics:
1. What are the demographic characteristics of the children and families participating in the HSQRC studies?
2. How do they compare with those from the national sample in the Head Start FACES study?
2. School readiness skills:
1. What are the school readiness skills of children participating in the HSQRC studies at the time they enter the respective studies?
2. How do they compare with those from the national sample in the Head Start FACES study?
3. For each of the intervention types, do children who were exposed to an intervention have greater growth from fall to spring in their school readiness skills compared to their control-group peers?
4. How does this growth from fall to spring compare with that from the national sample in the Head Start FACES study?
3. Classroom quality:
1. What are the program and staff characteristics and observed quality of the classrooms of the children participating in the HSQRC studies?
2. How do they compare with those from the national sample in the Head Start FACES study?
3. Are there any differences between the control and the intervention classrooms in classroom quality and teacher backgrounds?
4. Parenting skills:
1. What are the parenting behaviors skills that parents participating in the HSQRC studies have?
2. How do they compare with those from the national sample in the Head Start FACES study?
3. In the parent-involvement intervention, do parents who participated in the intervention have greater growth from fall to spring in their parenting skills than the control group parents?
4. How does this growth from fall to spring compare with that from the national sample in the Head Start FACES study?

Child and family demographics. Overall, for each intervention type there were few significant differences in the characteristics of children and families in the treatment groups and their respective control group peers. These similarities tended to be true for both cohorts. The few significant differences that were found between treatment and control groups were not systematic, tending to exist only for one of the cohorts, not both.

In general, the characteristics of the HSQRC children and families receiving an intervention were similar to those of the children and families in the FACES 2000 sample. Typically, only small differences were found in the demographic characteristics between the HSQRC sample and FACES 2000. However, the HSQRC sample had a larger proportion of African American children than the FACES sample.

Child school readiness skills. Within each of the intervention types there were few significant differences between the fall scores of the intervention group children and the control group children, indicating that they enter the QRC studies with similar skills. The few significant differences that did exist only existed for one year, not both years. For most measures, fall to spring gains by the intervention group were not significantly greater than those of their respective comparison groups. There were, however, significant treatment effects for social-emotional, teacher training, and individualizing assessment intervention types on some measures.

In the 2001-2002 program year, the design of the studies examining the literacy-focused interventions did not have a control group so no treatment-control comparisons could be made. However, children participating in the intervention did show significant fall to spring growth on each of the child assessment tasks.

In the 2002-2003 program year, a control group was included in the study design for literacy-focused interventions, allowing for such comparisons to be made. Children participating in the intervention showed significant growth in vocabulary, letter recognition, design copying, and book knowledge. However, these gains were not significantly different than the gains demonstrated by the control group.

For the socio-emotional focused interventions in 2001-2002, children receiving an intervention had a significantly larger growth score in parent ratings of positive approaches to learning than the control group. Children receiving an intervention also demonstrated significant fall to spring declines in parent ratings of total behavior problems, aggression, and hyperactivity; and significant gains in teacher ratings of cooperative classroom behavior. However, these fall to spring differences were not significantly different than those demonstrated by their control group peers. Counter intuitively, intervention children showed a significantly smaller decline in teacher ratings of aggression compared to the control group.

In 2002-2003, children receiving a socio-emotional focused intervention showed significant increases in parent ratings of positive approaches to learning and teacher ratings of cooperative classroom behavior. However, the gains in the parent ratings of positive approaches to learning were not significantly different than those demonstrated by the control group. Counter intuitively, the gains in teacher ratings of cooperative classroom behavior were significantly smaller than those demonstrated by the control group. These counterintuitive findings may be a result of the teacher training involved in the intervention, resulting in teachers in the intervention being more strict in their ratings than teachers who did not receive such training.

In the teacher training focused interventions in 2001-2002, children participating in the interventions showed significantly greater progress than their control group peers in PPVT scores, book knowledge, and early math. Children in the intervention also showed significant progress in letter recognition, early writing, design copying, and understanding of print conventions; however, these gains were not significantly different than those demonstrated by their control group peers.

In 2002-2003, teacher training focused intervention-group children showed significantly greater declines in parent ratings of hyperactive behavior than control group children. The intervention children showed significant growth in PPVT-III scores, early writing, book knowledge, design copying, and letter naming; however, these gains were not significantly different than those demonstrated by their control group peers.

In 2001-2002, children in the individualizing-assessment focused interventions showed significantly greater progress than their control group peers in PPVT scores, book knowledge, and knowledge of print conventions. Children in the intervention group also had significant gains were also found in letter recognition, design copying, and early writing; however these gains were not significantly different than those demonstrated by the children in the control group.

In 2002-2003, individualizing assessment intervention-group children showed significantly greater declines in parent ratings of total problem behavior than control group children. Children in the intervention group also showed significant growth in PPVT-III scores, early writing, book knowledge, design copying, and letter naming. However, these gains were not significantly different than those shown by the control group.

Many of the fall to spring gains for the intervention groups were larger than those reported in FACES. In 2001-2002, children in the literacy intervention group had significantly larger gains in WJ-R Letter-Word ID and WJ-R Dictation than children in FACES 2000. In 2001-2002, children in the teacher training intervention group had significantly larger gains in PPVT, WJ-R Letter-Word ID, and WJ-R Dictation than children in FACES 2000. In addition, children in the individualizing assessment interventions had significantly larger gains in WJ-R Letter-Word ID and WJ-R Dictation than children in FACES 2000.

Classroom quality. While the initial (fall) quality of both the control and intervention classrooms in the HSQRC sites appeared to have good overall quality, the initial (fall) quality of the intervention classrooms was higher. These differences were found in overall process quality, teacher sensitivity and, perhaps most important for many of the HSQRC intervention, language-related activities and materials in classrooms. Intervention classrooms were rated higher in quality on these factors compared with control classrooms over both years of the intervention but especially in the second year.

None of the comparisons of the specific intervention categories of classrooms (e.g., literacy-based, socio-emotional focused, teacher-training or individualizing) with their controls showed significant differences in classroom quality. The lack of significant differences for specific interventions may be due, in some cases, to relatively small sample sizes.

When comparing the intervention with the control classrooms, there were few significant differences for the first year of the intervention, possibly due to the low sample sizes and particularly the relatively reduced samples of control classrooms, since most sites did not have control classrooms in the first year. In the second year of the intervention, intervention classrooms were shown to have higher quality on a number of indicators including overall quality and the quality of language activities. Looking at comparisons of the control and intervention classroom teachers, teachers in the intervention classrooms reported higher annual salaries, and, in fall 2002, had significantly higher levels of education compared with teachers in the control classrooms.

Comparisons by the types of intervention yielded few differences between intervention and control classrooms. In fall 2001 the socio-emotional focused interventions had higher child:adult ratios, usually indicative of lower quality, but the low sample size makes this finding difficult to interpret. In the second year of the intervention, with larger samples, there were no significant differences between intervention and control classrooms except for the parent involvement intervention. The parent involvement intervention classrooms were rated higher in overall quality, the quality of the learning environment, and the Arnett in the fall of 2002.

The findings revealed that the HSQRC classrooms are providing high quality care that fits the definition of best practices for early childhood education. Both intervention and control HSQRC classrooms showed high process quality for both years, compared with FACES classrooms, on the ECERS-R, the Assessment Profile, and the Arnett Caregiver Interaction Scale. This occurred despite higher child adult ratios (indicating lower quality) than those reported for FACES classrooms.

Despite no differences in years of experience and age and only slight differences in education levels, teachers in the HSQRC sites received higher salaries compared with teachers in the national FACES sample. Overall, the backgrounds and experiences of teachers in the HSQRC sites (combining control and intervention classrooms) were similar to the national sample of Head Start teachers, in both the fall 2001 and fall 2002 cohorts.

When breaking out the control and intervention classrooms and comparing each separately to the national FACES sample, teachers in the control classrooms were somewhat better educated (a higher percentage had BA’s or higher) but were paid less than teachers in the FACES sample. Conversely, teachers in the intervention classrooms were better paid than their counterparts in the national sample of FACES classrooms, for both years of the intervention.

Parent outcomes. Across both program years, parents of sampled children in the HSQRC studies were active in the educational experiences of their children. Almost all parents read to their children at least once or twice a week, with 40 percent reading to them daily. Parents were active in weekly literacy activities and monthly outings, and more than 90 percent of parents participated in activities teaching their children words, letters, and numbers.

Parents were also active in setting rules and routines for their children. Parents rated themselves as using approaches that encourage independence in the child, while setting limits on behavior. Spanking was not frequently reported as a form of discipline. Most parents reported having a warm relationship with their children.

Examining the data from the parent involvement intervention, parents in the intervention group showed significantly greater declines in their categorical depression scores, compared to the control group, and the percentage of parents in the intervention group who were categorized as severely depressed significantly decreased over the course of the program year. In addition, parents in the intervention group showed significantly greater increases in their self-reported ability and energy to raise and discipline their children, compared to the control group.

From fall to spring, parents participating in the parent involvement intervention demonstrated significant increases in their participation in educational activities with their children, including weekly activities and monthly outings. They also significantly increased the number of rules and routines in their household, although in both cases the control group parents showed similar improvements.

The parenting behavior and styles of parents in the HSQRC sites were generally similar to those of parents in the FACES sample. There were no significant differences in parents’ frequency of reading to their children, frequency of weekly outings, the number of rules or routines in the home. However, parents in the HSQRC sites did participate in more monthly outings with their children than parents in the FACES sample. Parents in the HSQRC sites used spanking as a means of discipline less frequently than those in parents in FACES.

Conclusions. Some results presented in this interim report suggest that the interventions look promising. However, it is difficult to draw strong conclusions due to small sample sizes, different sizes of the intervention and control groups, and lack of information on fidelity to the interventions. To help address the sample size and imbalanced assignment issues, these studies are continuing with up to two more years of treatment/control designs in additional program sites. Also, measures are being held constant across multiple years in order to maximize the potential combined sample.

In regards to the issue of fidelity to the intervention, this report is limited to the set of measures common among all 8 HSQRC sites. However, each individual HSQRC also administers unique measures that are tailored to their specific intervention. Reports from each HSQRC site may include data on staff fidelity to curriculum procedures. Please refer to these individual reports for information on curriculum fidelity and the respective site’s methods of addressing this issue.

Finally, conducting research in partnership with the participating Head Start programs provided many benefits to the HSQRC researchers. Feedback from directors, managers, and teachers at the respective HSQRC sites improved the quality of the intervention, and provided an understanding of the intensity and type of support local staff needed to successfully implement the intervention. Further, staff feedback facilitated the integration of the intervention with current practice, and facilitated staff buy-in to the intervention. Partnerships between the HSQRC researchers and their partner programs also facilitated immediate feedback to programs for their own quality improvement. However, developing successful partnerships required a significant investment of time. Open communication among all parties, including administrators, teachers, support staff, and parents, was critical to the success of the partnerships and countered potential challenges such as confusion about the research design and methods. Clear communication and information sharing was also important to ensure the cooperation of all staff–those in the intervention and control groups--with the project. In addition, it was important that the HSQRC researchers remained flexible and able to adapt to unforeseen circumstances–staff turnover, strikes, and floods–as well as new national initiatives. The recognition of these challenges, and the strategies developed by the HSQRCs to overcome them, have been critical to the success of the work of the individual studies and the Consortium as a whole.

---

1 The lead office directing the consortium has been restructured as part of the Office of Planning, Research and Evaluation (OPRE) in the Administration for Children and Families (ACF). (back)

 

Return to Previous page