Table of Contents | Previous | Next |
CHAPTER VII
Predictive Validity of Cognitive and Behavioral Measures: Relationships Within and Across Cognitive and Social Developmental Domains
The child assessment battery for the Family and Child Experiences Survey (FACES) was designed to provide a comprehensive description of Head Start children’s development in both the cognitive and socio-emotional domains. As an indicator of the validity of the FACES instruments, criterion-related validity, that is the degree to which the test or questionnaire correlates with one or more outcome criteria, was assessed. Systematic evidence for criterion validity is often described in terms of predictive validity. Predictive validity is assessed when the outcome criterion is measured at a later time point after the evaluated test (e.g., later school year). This chapter evaluates how well the FACES measures predict children’s achievement and school adjustment at the end of their kindergarten year.24
FINDINGS
A. Children’s Scores on the FACES Instruments at the End of Head Start Predict Kindergarten Outcomes
The FACES cognitive measures were designed to assess preliteracy skills, as well as general school readiness. In order to determine the predictive validity of these measures, relationships between the FACES assessment scale scores obtained during the Head Start year and the Reading and General Knowledge scale scores obtained at the end of kindergarten were examined by two approaches. In the first approach, Reading and General Knowledge scale scores from the end of the kindergarten year were correlated with the scores from each of the FACES subtests from the end of the Head Start year. The second approach assessed the ability of the FACES scale scores from the end of the Head Start year to predict Reading and General Knowledge scale scores at the end of the kindergarten year in two multiple regression analysis.
Children’s scores on each of the component tasks in the FACES battery at the end of Head Start correlated significantly with their Reading scale scores at the end of kindergarten. Bivariate correlations with the Reading scale ranged from .55 (for the Woodcock-Johnson-R Letter-Word Identification) to .23 (for the Social Awareness). These moderate to high correlations indicate that the FACES measures have predictive power on outcome criteria at later time points. When the subtest scores were combined in a multiple regression model, the model did quite well at predicting children’s early reading skills at the end of kindergarten, accounting for 46 percent of the variance in Reading scale scores. The best predictor of Reading scale scores was the Woodcock-Johnson-R Letter-Word Identification task, which also showed the highest bivariate correlation with the Reading scale (Figure 7.1).
[D] |
Similarly, children’s scores on each of the component tasks in the FACES battery at the end of Head Start also correlated significantly with their scores on the General Knowledge scale at the end of kindergarten. Bivariate correlations with the General Knowledge scale ranged from .77 (for the Peabody Picture Vocabulary Test-III or PPVT-III) to .30 (for the Draw-a-Design and Social Awareness), again indicating that the FACES measures have predictive power on outcome criteria at later time points. When the sub-test scores were combined in a multiple regression model, the model accounted for 65 percent of the variance in General Knowledge scale scores. The best predictor in the multiple regression was the PPVT-III (beta = .62), which also showed the highest bivariate correlation with the General Knowledge scale (Figure 7.2).
[D] |
Neither the PPVT-III nor the Book Knowledge score were significant predictors of the Reading scale score in its multiple regression model, though scores on both of these tasks correlated significantly with the Reading score (rs = .42 and .39, respectively). It is noteworthy to also mention that the best predictor of the Reading score, namely, Woodcock-Johnson-R Letter-Word Identification, was not a significant predictor of General Knowledge in its multiple regression analysis, despite the fact that the Letter-Word Identification task did correlate significantly with the General Knowledge task (r = .40). Conversely, the best predictor of General Knowledge, namely, the PPVT-III, did not have a significant role in the multivariate prediction of children’s Reading scores.
It appears that the Reading and General Knowledge assessments may be tapping two distinct clusters of skills, both of which have been shown to be important for children’s future reading proficiency and academic achievement. The skills tapped by the Reading scale during kindergarten were primarily what Whitehurst and Lonigan (1998) have called “inside-out” skills such as letter recognition, letter-sound association, and word decoding. The skills tapped by the General Knowledge assessment were “outside-in” skills, such as general information, word knowledge, and conceptual understanding that children need to help them comprehend and evaluate what they read and relate it to facts and concepts they have previously acquired. The FACES battery showed its validity by predicting well to children’s later learning in both skill domains. In addition, these results suggest that development in both skills domains should receive attention in preschool curricula and practice, in order to foster both types of school-age abilities.
[D] |
Although both domains were predicted well, the combination of subtests that produced the best forecasts differed across the two skill clusters. The Letter-Word Identification test was the best predictor of inside-out skills, with Applied Problems, Dictation, One-to-One Counting, and McCarthy Draw-a-Design contributing additional predictive power. The PPVT-III was by far the best predictor of outside-in skills, with Applied Problems, Book Knowledge, Dictation, and the McCarthy Draw-a-Design showing much smaller but significant regression coefficients as well. It is noteworthy that seven of the eight FACES subtests contributed significantly to either the Reading or General Knowledge regression model or both.
Predictive Validity of Abbreviated
FACES Battery
In order to further explore the validity of the FACES battery, multivariate
regression analyses were carried out with the set of norm-referenced
tests (i.e., PPVT-III and the Woodcock-Johnson-R subtests) at the
end of Head Start, predicting the Reading and General Knowledge scale
at the end of kindergarten. These analyses were repeated with the
set of criterion-referenced measures (i.e., Social Awareness, McCarthy
Draw-a-Design, Color Names, One-to-One Counting, and Book Knowledge)
as the predictors.
The four norm-referenced tests at the end of Head Start did almost as well as the full battery at predicting the Reading scores at the end of kindergarten (Figure 7.3), predicting 43 percent of the variance. The Letter-Word Identification was most closely associated with the Reading scores. Applied Problems and Dictation had significant coefficients as well. But, as before, the PPVT-III did not.
[D] |
[D] |
The four norm-referenced tests at the end of Head Start also did as well as the full battery at predicting the General Knowledge scores at the end of kindergarten, predicting 65 percent of the variance. The PPVT-III was most closely associated with the General Knowledge scores. Dictation and Applied Problems had significant regression coefficients, but Letter-Word Identification did not (Figure 7.4).
The five criterion-referenced FACES tasks significantly predicted the Reading and General Knowledge assessments at the end of kindergarten, though notably less well than either the full battery or the abbreviated battery composed of norm-referenced tests, predicting 30 percent and 36 percent of the respective variances. All five subtests were significant predictors of the Reading score (Figure 7.5). As with the full battery, tasks that tapped perceptual-motor skills and knowledge of print conventions were more closely related to kindergarten Reading scores than were tasks that tapped vocabulary or general information.
All five sub-tests were significant predictors of General Knowledge as well, but the pattern of relative magnitudes was different than that for Reading. Book Knowledge was most closely related with General Knowledge, while the other four sub-tests had relatively lower, but significant regression coefficients ranging from .15 to .13 (Figure 7.6).
The combination of the four norm-referenced tests at the end of Head Start that were more extended (and hence more reliable in terms of internal consistency) did a better job of predicting to children’s kindergarten achievement than the combination of the five FACES criterion-referenced tasks. However, further analysis revealed that the five criterion-referenced measures provide significant unique contributions to the prediction of the Reading scale score, over and above the contributions of the norm-referenced tests, increasing the predictive power of the assessment battery by almost 3 percent. The criterion-referenced measures also provided unique contributions to the prediction of the General Knowledge scale score increasing the predictive power by .6 percent, an association at the trend level (p > .10). These results indicate that the criterion-referenced measures significantly contribute to the assessment battery, by picking up variance not accounted for by the norm-referenced tests.
[D] |
[D] |
B. FACES Behavioral Ratings at the End of Head Start Predict Children’s Social Competence in Kindergarten
Social competence is an important developmental domain measured by the FACES battery. Two sources of information are tapped for assessing children’s social competencies by collecting ratings of the children’s behavior from their teachers and parents. These analyses examine the ability of the teacher and parent ratings of children’s social competencies during Head Start to predict children’s school adjustment at the end of kindergarten. Analysis of the predictive validity of the behavior ratings mirror those for the cognitive measures. First, teacher ratings of cooperative classroom behavior and total problem behaviors from the end of the kindergarten year were correlated with the behavior ratings from teachers and parents at the end of the Head Start year. Then, the teacher and parent ratings were combined in multiple regressions predicting teacher ratings of cooperative classroom behavior and total problem behaviors at the end of the kindergarten year in two multiple regression analyses.25
[D] |
Parent and teacher ratings of behavior at the end of Head Start were moderately correlated with teacher ratings of cooperative classroom behavior at the end of kindergarten, indicating that the FACES measures have predictive power on kindergarten outcomes (Figure 7.7). Correlations were significant in the expected directions. Problem behaviors as rated by both parents and teachers at the end of Head Start had significantly negative correlations with teacher ratings at the end of kindergarten, ranging from -.36 (for teacher reported ratings of total behavior problems and aggressive behavior) to -.14 (for parent reported ratings of hyperactive behavior and withdrawal behavior). Parent ratings of positive approaches to learning and teacher ratings of cooperative classroom behavior were both positively correlated with teacher ratings of cooperative classroom behavior at the end of kindergarten (rs = .08 and .33, respectively). In general, teacher ratings showed stronger relationships than the parent ratings did with the kindergarten outcomes.
[D] |
When the ratings were combined in a multiple regression, the model accounted for 18 percent of the variance in teacher ratings of cooperative classroom behavior. The best predictor in the multiple regression was teacher reported ratings of aggressive behavior (beta = -.25), which also had the strongest bivariate correlation (Figure 7.8).
Similarly, parent and teacher ratings of behavior at the end of Head Start were moderately correlated with teacher ratings of total problem behaviors at the end of kindergarten, again indicating that the FACES measures have predictive power on kindergarten outcomes. Correlations were significant in the expected directions. Problem behaviors as rated by both parents and teachers at the end of Head Start had significantly positive correlations with teacher ratings of total problem behaviors at the end of kindergarten, ranging from .37 (for teacher reported ratings of aggressive behavior) to .23 (for parent reported ratings of aggressive and withdrawal behavior). Parent ratings of positive approaches to learning and teacher ratings of cooperative classroom behavior were both negatively correlated with teacher ratings of total problem behavior at the end of kindergarten (rs = -.15 and -.36 respectively). In general, teacher ratings showed stronger relationships than the parent ratings did with the kindergarten outcomes.
When the ratings were combined in a multiple regression, the model accounted for 24 percent of the variance in teacher ratings of total problem behavior. The best predictor in the multiple regression was teacher reported ratings of aggressive behavior (beta = .23), which also had the strongest bivariate correlation (Figure 7.9).
Predictive Validity of Abbreviated
Sets of Behavior Ratings
In order to further explore the validity of the behavior ratings,
multivariate regression analyses were carried out with the set of
teacher reported ratings at the end of Head Start predicting the teacher
reported ratings of cooperative classroom behavior and total problem
behavior at the end of kindergarten.
[D] |
[D] |
The teacher reported ratings at the end of Head Start did almost as well as the full set of behavior ratings at predicting the teacher reported ratings of cooperative classroom behavior at the end of kindergarten. The model explained 16 percent of the variance in kindergarten cooperative classroom behavior ratings, and the rating of aggressive behavior was most closely associated with the cooperative classroom behavior ratings (beta = -.26). The rating of cooperative classroom behavior had a significant regression coefficient as well (beta = .14). But, as before, the ratings of hyperactive and withdrawal behavior did not.
The four parent reported behavior ratings significantly predicted the teacher ratings of cooperative classroom behavior at the end of kindergarten, though notably less well than either the full set of behavior ratings or the set of teacher reported ratings. The model explained 6 percent of the variance in teacher reported cooperative classroom behavior, and only parent reported ratings of aggressive behavior had a significant regression coefficient (beta = -.19).
The combination of the four teacher reported behavior ratings at the end of Head Start did a better job of predicting children’s kindergarten behavior than the combination of the four parent reported behavior ratings. However, the four parent ratings provided significant unique contributions to the prediction of the cooperative classroom behavior ratings, over and above the contributions of the teacher ratings, increasing the predictive power of the assessment battery by almost 2 percent.These results indicate that the parent ratings significantly contribute to the assessment battery, by picking up variance not accounted for by the teacher reported ratings.
Similarly, the teacher reported ratings at the end of Head Start did almost as well as the full set of behavior ratings at predicting the teacher reported ratings of total problem behavior at the end of kindergarten. The model explained 18 percent of the variance in kindergarten total problem behavior ratings, and the rating of aggressive behavior was most closely associated with the total problem behavior ratings (beta = .22). The rating of cooperative classroom behavior had a significant regression coefficient as well (beta = -.17). But the ratings of hyperactive and withdrawal behavior did not.
The four parent reported behavior ratings also significantly predicted the teacher ratings of total problem behavior at the end of kindergarten, though notably less well than either the full set of behavior ratings or the set of teacher reported ratings. The model explained 11 percent of the variance in teacher reported total problem behavior, and parent reported ratings of hyperactive behavior had the largest regression coefficient (beta = .18).
The combination of the four teacher reported behavior ratings at the end of Head Start did a better job of predicting children’s kindergarten behavior than the combination of the four parent reported behavior ratings. However, the four parent ratings provided significant unique contributions to the prediction of the cooperative classroom behavior ratings, over and above the contributions of the teacher ratings, increasing the predictive power of the assessment battery by almost 6 percent.These results indicate that the parent ratings significantly contribute to the assessment battery, by picking up variance not accounted for by the teacher reported ratings.
C. Behavior Ratings at the End of Head Start Predict Reading Skills and General Knowledge at the End of Kindergarten
Given that certain positive behaviors may foster learning, while other negative behaviors may impede learning, the ability of these behavior ratings obtained during the Head Start year to predict Reading and General Knowledge scale scores obtained at the end of kindergarten was also examined.
Almost all of the parent and teacher ratings of behavior at the end of Head Start correlated significantly with Reading scale scores at the end of kindergarten, indicating that the FACES measures have predictive power on outcome criteria at later time points. Only parent reported ratings of positive approaches to learning were not significantly related with Reading scores at the end of kindergarten. Correlations were significant in the expected directions. Ratings of problem behaviors were negatively correlated with kindergarten Reading scale scores, suggesting that behaviors that may impede learning are associated with lower reading skills in kindergarten (Figure 7.10).
[D] |
When the behavior ratings were combined in a multiple regression model, the model accounted for 8 percent of the variance in Reading scale scores. The best predictor in the multiple regression was teacher reported ratings of cooperative classroom behavior (beta = .19), followed by parent reported ratings of hyperactive behavior (beta = -.10), followed by teacher reported ratings of withdrawal behavior (beta = -.10) (Figure 7.11).
Significant correlations were also found between children’s scores on each of the parent and teacher reported behavior ratings at the end of Head Start and General Knowledge scale scores at the end of kindergarten, but these relationships were weaker than those found with Reading scale scores reported above. Significant bivariate correlations with General Knowledge scale scores ranged in absolute value from .23 (for teacher ratings of cooperative classroom behavior) to .09 (for parent reports of aggression), again indicating that the FACES measures have predictive power on outcome criteria at later time points. Significant correlations were in the expected directions. Ratings of problem behaviors were negatively correlated with kindergarten General Knowledge scale scores, suggesting that behaviors that may impede learning are associated with lower skills in the natural sciences and social studies in kindergarten (Figure 7.12).
[D] |
[D] |
When the behavior ratings were combined in a multiple linear regression model, the model accounted for 7 percent of the variance in General Knowledge scale scores. The strongest predictor in the multiple regression was teacher reported ratings of cooperative classroom behavior (beta = .26), which also showed the strongest bivariate correlation with General Knowledge scale scores, followed by parent reported ratings of hyperactive behavior (beta = -.12). Surprisingly, teacher reported ratings of aggressive behavior had a significant positive regression coefficient (beta = .11). None of the other behavior ratings were significant predictors in the model (Figure 7.13).
FACES Behavior Ratings Contribute
to the Prediction of Reading at the End of Kindergarten
In order to further examine the usefulness of the behavior ratings,
the unique contribution of the set of behavior rating measures to
the prediction of kindergarten outcomes was assessed. Neither for
reading nor for general knowledge did behavior ratings contribute
significantly over and above the cognitive assessments (p
> .05). However, the eight behavior ratings did provide unique
contributions at the trend level (p >
.10) to the prediction of the Reading scale score, over and above
the contributions of the cognitive tests, increasing the predictive
power of the assessment battery by 1.3 percent. This indicates that
the behavior ratings provide some unique contributions to the prediction
of kindergarten outcomes.
D. Head Start Fall to Spring Gain Scores From the FACES Battery Predict Kindergarten Outcomes
As a further assessment of the predictive validity of the cognitive measures from the FACES battery, the predictive validity of their fall to spring gain scores was assessed.To determine the children’s gains during their Head Start year, differences between the fall 1997 and spring 1998 mean scores were calculated for each of the measures by subtracting the fall score from the spring score for each child in the study.
The ability of the fall to spring gains scores from the cognitive measures of the FACES battery to predict later school readiness was assessed by two approaches. In the first approach, Reading and General Knowledge scale scores from the end of the kindergarten year were correlated with the gain scores from each of the FACES sub-tests from the end of the Head Start year. In these analyses, partial correlations were calculated, controlling for individual difference in fall 1997 baseline scores.26 The second approach assessed the ability of the fall to spring gain scores from the FACES instruments to predict Reading and General Knowledge scale scores at the end of the kindergarten year in a multiple linear regression analysis.27
Children’s fall to spring gain scores on each of the component tasks in the FACES battery correlated significantly with their Reading scale scores at the end of kindergarten. Bivariate correlations with the Reading scale ranged from .39 (for the Woodcock-Johnson-R Letter-Word Identification gain score) to .11 (for the Social Awareness gain score). These significant correlations indicate that the gain scores from the FACES measures are predictive of kindergarten outcomes. When the gain scores were combined in a multiple regression model, the model did quite well at predicting children’s early reading skills at the end of kindergarten, accounting for 31 percent of the variance in Reading scale scores. The best predictor of Reading scale scores was the gain score for Woodcock-Johnson-R Letter-Word Identification task (beta = .27), which also showed the highest bivariate correlation with the Reading scale (Figure 7.14).
Similarly, children’s fall to spring gain scores on all but one of the component tasks (Color Naming; r = .03) in the FACES battery correlated significantly with their General Knowledge scale scores at the end of kindergarten. Significant bivariate correlations with the General Knowledge scale ranged from .33 (for the Book Knowledge gain score) to .10 (for the Social Awareness gain score). These significant correlations indicate that the gain scores from the FACES measures are predictive of kindergarten outcomes. When the gain scores were combined in a multiple regression model, the model did quite well at predicting children’s general knowledge at the end of kindergarten, accounting for 28 percent of the variance in General Knowledge scale scores. The best predictor in the model was the gain score for the PPVT-III (beta = .24), followed by the Book Knowledge gain score (beta = .19), which had the highest bivariate correlation with the General Knowledge scores (Figure 7.15).
Similar to the analysis with the FACES battery scores at the end of Head Start, the Book Knowledge gain score was not a significant predictor of the Reading scale score in its multiple regression model, though it correlated significantly with the Reading score (r = .26). Another similarity to the analysis with the scores at the end of Head Start is that the best predictor of the Reading score, namely the Woodcock-Johnson-R Letter-Word Identification gain score, was not a significant predictor of General Knowledge in its multiple regression analysis, despite the fact that its gain score correlated significantly with the General Knowledge task (r = .18). Conversely, the best predictor of General Knowledge, namely, the PPVT-III, was the weakest significant predictor in the multivariate prediction of children’s Reading scores (beta = .09).
[D] |
The findings of the gain scores analysis support the conclusions of the analysis with the scores at the end of Head Start that the Reading and General Knowledge assessments may be tapping two distinct clusters of skills: “inside-out” skills and “outside-in” skills. The gain scores from the FACES battery again showed its validity by predicting well to children’s later learning in both skill domains.
Although both domains were predicted well, the combination of sub-tests that produced the best forecasts differed across the two skill clusters. The gain score from the Letter-Word Identification test was the best predictor of inside-out skills, with gain scores from One-to-One Counting, Dictation, McCarthy Draw-a-Design, Applied Problems, and PPVT-III contributing additional predictive power. The PPVT-III gain score was by far the best predictor of outside-in skills, with gain scores from Book Knowledge, McCarthy Draw-a-Design, Dictation, One-to-One Counting, and Applied Problems showing much smaller but significant regression coefficients as well. It is noteworthy that seven of the eight FACES sub-tests contributed significantly to either the Reading or General Knowledge regression model or both.
E. Head Start Fall to Spring Gain Scores From the Behavior Ratings Predict Social Competence in Kindergarten
The fall to spring gain scores from the parent and teacher ratings of behavior were moderately correlated with teacher ratings of cooperative classroom behavior at the end of kindergarten, indicating that the FACES measures have predictive power on kindergarten outcomes. Correlations were significant in the expected directions. Increases in problem behaviors as rated by both parents and teachers at the end of Head Start had significantly negative correlations with teacher ratings of cooperative classroom behavior at the end of kindergarten, ranging from -.24 (for the change score for parent reported ratings of total problem behaviors) to -.14 (for the change score for parent reported ratings of hyperactive behavior). The gain score for teacher ratings of cooperative classroom behavior was positively correlated with teacher ratings of cooperative classroom behavior at the end of kindergarten (r = .15), but the gain score for parent ratings of positive approaches to learning was not. In general, gain score for the teacher ratings showed stronger relationships with the kindergarten outcomes than those for the parent ratings.
When the rating gain scores were combined in a multiple regression, the model accounted for 11 percent of the variance in teacher ratings of cooperative classroom behavior. The best predictors in the multiple regression were the gain scores for teacher reported ratings of withdrawal behavior and parent reported ratings of aggressive and withdrawal behavior (betas = -.13), which had the stronger bivariate correlations among the set of behavior ratings (Figure 7.16).
Similarly, the fall to spring gain scores from the parent and teacher ratings of behavior were moderately correlated with teacher ratings of total problem behavior at the end of kindergarten, again indicating that the FACES measures have predictive power on kindergarten outcomes. Correlations were significant in the expected directions. Increases in problem behaviors as rated by both parents and teachers at the end of Head Start had significantly positive correlations with teacher ratings of total problem behavior at the end of kindergarten, ranging from .29 (for the change score for parent reported ratings of total problem behaviors) to .12 (for the change score for teacher reported ratings of aggressive behavior). The gain score for teacher ratings of cooperative classroom behavior was negatively correlated with teacher ratings of cooperative classroom behavior at the end of kindergarten (r = -.13), but the gain score for parent ratings of positive approaches to learning was not significantly related. In general, gain score for the teacher ratings showed stronger relationships with the kindergarten outcomes than those for the parent ratings.When the rating gain scores were combined in a multiple regression, the model accounted for 15 percent of the variance in teacher ratings of total problem behaviors. The best predictors in the multiple regression were the gain scores for parent reported ratings of withdrawal behavior, (beta = .21) (Figure 7.17).
F. Head Start Fall to Spring Gain Scores From the Behavior Ratings Predict Reading Skills and General Knowledge at the End of Kindergarten
The fall to spring gain scores for parent and teacher ratings of behavior correlated significantly with Reading scale scores at the end of kindergarten, indicating that the FACES measures have predictive power on kindergarten outcomes. Correlations were significant in the expected directions. Increases in problem behaviors were negatively correlated with kindergarten Reading scale scores, adding more evidence that behaviors that may impede learning are associated with lower reading skills in kindergarten.
[D] |
When the behavior ratings gain scores were combined in a multiple regression model, the model accounted for 4 percent of the variance in Reading scale scores. The best predictor in the multiple regression was the gain score for teacher reported ratings of withdrawal behavior (beta = -.12) followed by the gain score for parent reported ratings of withdrawal behavior (beta = -.11).
The correlations of the fall to spring gain scores of parent and teacher ratings of behavior and General Knowledge scale scores at the end of kindergarten were not as strong as those for the Reading scale scores. Four of the eight behavior rating gain scores (parent ratings of aggressive, hyperactive, and withdrawal behavior, and teacher ratings of cooperative classroom behavior) were significantly related to General Knowledge scores, indicating that these FACES measures have some predictive power on kindergarten outcomes. Significant correlations were in the expected directions. Increases in problem behaviors were negatively correlated with kindergarten General Knowledge scale scores, adding more evidence that behaviors that may impede learning are associated with lower skills in natural sciences and social studies in kindergarten. When the behavior ratings were combined in a multiple regression model, the model accounted for 2.5 percent of the variance in General Knowledge scale scores, which is an association at the trend level (p < .10).
[D] |
Head Start Fall to Spring Gain
Scores From the Behavior Ratings Contribute to the Prediction of Reading
and General Knowledge at the End of Kindergarten
In order to further examine the usefulness of the behavior ratings,
the unique contribution of the set of behavior rating gain scores
to the prediction of kindergarten outcomes was assessed. The eight
behavior ratings provided unique contributions at the trend level
(p > .10) to the prediction of the
Reading scale score, over and above the contributions of the cognitive
tests, increasing the predictive power of the assessment battery by
almost 3 percent. Similarly, the eight behavior ratings provided unique
contributions at the trend level (p > .10) to the prediction of the General Knowledge scale score, over
and above the contributions of the cognitive tests, increasing the
predictive power of the assessment battery by almost 3 percent. These
results indicate that the addition of the behavior rating gain scores
provide some unique contributions to the prediction of kindergarten
outcomes.
G. Children’s Scores on the FACES Instruments and Behavior Ratings at the End of Head Start Predict Promotion to First Grade
Another measure of the predictive validity of the FACES battery is to examine how well scores on the FACES instruments and behavior ratings at the end of Head Start are related to practical decision-making at the end of kindergarten, namely, the teacher’s decision of whether the child gets promoted to first grade or repeats the kindergarten year. In this set of analysis, teachers’ decisions at the end of the kindergarten year to have the child repeat another year of kindergarten (versus promote the child to first grade) were first correlated with the scores from each of the FACES instruments from the end of the Head Start year.Then the ability of the FACES scale scores from the end of the Head Start year to predict teachers’ decisions to have the child repeat a year of kindergarten was examined in a multiple logistic regression analysis. This approach was then repeated with the parent- and teacher-reported behavior ratings.
Children’s Scores on the
FACES Instruments at the End of Head Start Predict Repeating Kindergarten
Children’s scores on each of the component tasks in the FACES
battery at the end of Head Start correlated significantly with teacher
decisions to have the child repeat another year of kindergarten. Bivariate
correlations were in the expected directions, ranging from -.31 (for
Book Knowledge) to -.12 (for Draw-A-Design), indicating that lower
subtest scores were associated with repeating kindergarten. These
significant correlations indicate that the FACES measures have predictive
power on outcome criteria at later time points.
When the subtest scores were combined in a multiple logistic regression model, the model did quite well at predicting whether children repeated kindergarten, accounting for 24 percent of the variance in the prediction of repeating kindergarten. Information of scores from the FACES instruments led to an 82 percent accuracy rate in predicting or not a child was assigned by her teacher to repeat kindergarten. The strongest predictor of whether or not children were assigned by their teachers to repeat kindergarten was the Book Knowledge task, in which for every unit increase in Book Knowledge scores, children were 50 percent less likely to repeat kindergarten (Figure 7.18).
[D] |
Children’s Behavior Ratings
at the End of Head Start Predict Repeating Kindergarten
Parent and teacher behavior ratings at the end of Head Start correlated
significantly with teacher decisions to have the child repeat another
year of kindergarten. Correlations were in the expected directions.
Problematic behaviors were positively correlated with repeating kindergarten,
ranging from .26 (for teacher ratings of withdrawal behavior) to -.09
(for teacher ratings of aggressive behavior). Teacher ratings of cooperative
classroom behavior at the end of Head Start were negatively correlated
with teacher decisions to repeat kindergarten (r = -.14); parent ratings
of positive approaches to learning, however, were not significantly
related. The significant correlations indicate that the FACES behavior
ratings also have predictive power on outcome criteria at later time
points.
When the parent and teacher ratings were combined in a multiple logistic regression model, the model did quite well at predicting whether children repeated kindergarten, accounting for 11 percent of the variance in the prediction of repeating kindergarten. Information from the FACES behavior ratings led to a 72 percent accuracy rate in predicting whether or not a child was assigned by her teacher to repeat kindergarten. The strongest predictor of whether or not children were assigned by their teachers to repeat kindergarten was teacher ratings of withdrawal behavior, in which for every unit increase in these teacher ratings, children were 31 percent more likely to repeat kindergarten (Figure 7.19).
[D] |
The Combination of Children’s
Scores on the FACES Instruments and Behavior Ratings at the End of
Head Start Predict Repeating Kindergarten
When the parent and teacher ratings were combined with the subtest
scores from the FACES instruments in a multiple logistic regression
model, the model did quite well at predicting whether children repeated
kindergarten, accounting for 30 percent of the variance in the prediction
of repeating kindergarten. This is an increase of 6 percent of the
variance that was explained by the subtest scores alone. Information
from the combination of the FACES behavior ratings and the subtest
scores led to an 83 percent accuracy rate in predicting whether or
not a child was assigned by her teacher to repeat kindergarten. This
suggests that the additional information provided by the behavior
ratings adds to the predictive validity of the FACES instruments in
predicting kindergarten repetition.
CONCLUSIONS
The FACES battery has strong predictive validity with outcomes at the end of kindergarten. As an indicator of preliteracy skills, the cognitive measures show strong associations with reading ability at the end of the kindergarten year. As an indicator of school adjustment and social competence, the behavior ratings demonstrate ability to predict kindergarten behaviors that promote learning. These analyses show that:
-
The instruments used in FACES predict later behavior and performance in kindergarten.
-
The instruments used in FACES also predict the later practical decision of whether a child gets promoted to first grade.
-
The instruments used in FACES tap different types of abilities (“inside-out” versus “outside-in”) that are important for children’s future reading proficiency and academic achievement.
-
The multi-measure and multi-method approach to the measurement of children’s abilities provides a variety of information sources that significantly contribute to the prediction of kindergarten outcomes.
Children who had higher scores at the end of the Head Start year, and who made greater gains during the year on the Letter-Word Identification test, Applied Problems, Dictation, One-to-One Counting, and McCarthy Draw-a-Design tasks, tended to have greater early reading skills at the end of kindergarten. Children’s improved scores on the PPVT-III, Applied Problems, Book Knowledge, Dictation, and the McCarthy Draw-a-Design tasks at the end of Head Start were associated with greater General Knowledge scores at the end of kindergarten. These results suggest that efforts in improving children’s performance and behavior in preschool can result in greater school readiness and school adjustment when these children are preparing to enter first grade.
In the assessment of children’s social competencies, the use of parent and teacher ratings provides data on children’s coping skills in different situations and provides a more comprehensive picture of their behavior. Equally important, both parent and teacher ratings significantly contribute to the prediction of social skills at the end of kindergarten. The parent and teacher ratings also significantly predict reading skills and general knowledge at the end of kindergarten. Ratings of problem behaviors were negatively correlated with kindergarten Reading and General Knowledge scale scores, suggesting that behaviors that may impede learning are associated with lower reading skills in kindergarten. High ratings of behaviors that enhance learning, positive approaches to learning and cooperative classroom behavior, were positively correlated with kindergarten outcomes. These analyses suggest that curricula that strengthen children’s social skills will also have beneficial effects on their later school readiness.
The multi-measure and multi-method approach to the measurement of children’s development and school readiness provides a comprehensive assessment of children’s abilities. The addition of the criterion-referenced measures to the norm-referenced measures improves the assessment battery in many ways. First, they are short tasks that cover more specific topic areas that are typically taught in preschool curricula and are also fun for the children to do in the assessment. And second, they significantly (although moderately) increase the battery’s ability to predict kindergarten outcomes, improving its predictive validity. The addition of the parent and teacher behavior ratings adds another source of information that predict kindergarten outcomes, namely, an assessment of behaviors that can either foster or impede learning. They also provide some unique contributions to the battery’s ability to predict kindergarten outcomes, particularly in the practical decision of whether a child is promoted to first grade or repeats kindergarten.
REFERENCE
Whitehurst, G. J. & Lonigan, C. J. (1998) Child development and emergent literacy. Child Development, 69, 848-872.
24Final kindergarten data from FACES 2000 are not yet available, therefore the following analyses are conducted with FACES 1997-1998 data. All of the described analyses were conducted on children who were assessed in English at all timepoints.(back)
25In all multiple regression analyses involving the behavior ratings, teacher and parent ratings of total behavior were excluded as predictor variables from the models. Because these ratings are summative scores of the ratings for aggression, withdrawal, and hyperactivity, including them would introduce multicollinearity among these predictor variables to the model. Therefore, they were excluded from the regressions. They are included in the bivariate correlation analyses.(back)
26In analyses of gains scores, baseline scores are controlled for, effectively examining the effect of the gain scores, if all students had the same baseline score.(back)
27Control of the fall baseline score for each gain score in the multiple linear regression models was accomplished through the use of residual scores. A residual score was created for each gain score with the effects of its respective baseline score partialled out. These residual scores were then entered as independent variables in the multiple linear regression, predicting the kindergarten outcome variables.(back)
| Table of Contents | Previous | Next |