Summary of Design and Findings

Project Upgrade was a two-year experimental test of the effectiveness of three different language and literacy interventions, implemented in child care centers in Miami-Dade County that served children from low-income families. One hundred and sixty-two centers were randomly assigned to one of three research-based curricula or to a control group that continued with its existing program. The curricula, while grounded in a common set of research findings, differed in intensity, pedagogic strategies and use of technology. In each center, one classroom that served four-year-old children was selected for the study. Teachers and aides assigned to the three treatment groups received initial and follow-up training as well as ongoing mentoring over a period of approximately 18 months, from Fall 2003 to Spring 2005. All classrooms in the study, whether treatment or control, received an initial package of literacy materials (paper, crayons, books, tape recorders, books on tape etc.). To reduce staff turnover, teachers in all four groups who remained in centers received $500 in July, at the end of each year of the study.

The hypotheses tested by the study stipulated two kinds of outcomes: teacher behavior and interactions with children, and aspects of the classroom environment that support children’s language and literacy development, measured through direct observation; and children’s language and pre-literacy skills, measured by their performance on a standardized assessment. Study staff conducted classroom observations in Fall 2003, Spring 2004 and Spring 2005. Four-year-old children in the study classrooms were assessed in Spring 2005.

Key findings are summarized below and in Exhibit 1. Here, and in the body of the paper, impacts are described in terms of effect sizes. Effect sizes are standardized measures of the magnitude (size) of treatment effects. For each outcome measure, the effect size is equal to the estimated impact of the treatment, divided by the control group standard deviation (a measure of the variation in scores within the group). The standardization makes possible a comparison of the size of treatment effects across studies and, within limits, across outcome measures.1 For example, if the effect sizes of a treatment on outcome measures A and B are 0.50, and 0.25, respectively, then the size of the treatment impact on A is considered to be twice the size of the impact on B. For each outcome reported, tables showing more detailed statistical data are provided in Attachment A.

Findings

• The initial observations, conducted before the interventions, showed that, across all groups, teachers engaged in few of the behaviors and interactions that have been shown to support children’s development of language and literacy skills.

• Within six months of training, in Spring 2004, all three language/literacy interventions produced significant impacts on teacher behaviors and interactions with children that supported their language and literacy development; by Spring 2005, these impacts were generally more pronounced, and there were significant impacts on the number of classroom activities that involved literacy, and on literacy resources in the classroom.

• The interventions had significant positive impacts on teacher behavior. These impacts were generally stronger for teachers whose primary language was Spanish than for their English-speaking counterparts.

• Two of the three interventions, Ready, Set, Leap and Breakthrough to Literacy, had significant impacts on all four measures of emergent literacy outcomes for children: definitional vocabulary; phonological awareness; knowledge and understanding of print; and the overall index of early literacy. The impact of the two effective interventions was much greater for children in classrooms with Spanish-speaking teachers than for children in classrooms with English-speaking teachers.

• The two interventions that had impacts on child outcomes brought children close to or above the national norms on three of the four outcomes. On the fourth, although children in the two treatment groups had significantly higher scores, they still lagged considerably behind the national norms. The impacts represent between four and nine months of developmental growth, depending on the outcome. The effects of the interventions are substantially larger than those found on similar measures in the Head Start Impact Study and more closely resemble the effects of school-based prekindergarten programs.

• The interventions resulted in a substantial increase in the time spent on language and literacy activities, both teacher-directed and child-initiated. This did not eliminate other important developmental activities. Rather, time spent on each of the other activities was reduced slightly.

• There was a small but significant relationship between teachers’ educational attainment and some aspects of their behavior with children before the interventions. The effect of the training and on-going mentoring provided as an integral part of the interventions was to eliminate this effect. That is, as a result of the training and mentoring, less-educated teachers looked remarkably similar to their better-educated counterparts in the extent to which they provided activities that supported literacy. Consequently, the impacts of the interventions on child outcomes were not affected by teachers’ educational achievement.