Table of Contents | Previous | Next |
C.2 CONSTRUCTS USED IN THE ANALYSIS: PSYCHOMETRIC PROPERTIES
To be included in the impact analyses, constructed variables had to meet the following criteria:
- Sufficient Data at the Item Level. If an individual was missing
25 percent or more of the items that went into a constructed variable,
we did not construct the variable for that individual and that individual
was not included in the impact analysis of that variable. If the
individual was missing fewer than 25 percent of the items needed
for a constructed variable, we imputed values based on the mean
of the nonmissing items. The proportion of scores that required
imputation was fairly low-if a parent began a measure, they generally
completed all of the items. We never imputed values for our direct
child assessments (the Bayley and the MacArthur) or our parent-child
structured play assessments.
- Adequate Distribution of Scores. For our constructed variables,
we checked the mean, standard deviation, skewness, and kurtosis
to determine whether the variables had a normal distribution and
seemed to have a similar distribution to those found in other studies
using the same measure. In general, we found that our distributions
met the criteria for normality, with skewness and kurtosis levels
within appropriate ranges. The distributions were similar to those
found in other studies of low-income families. Our sample means
and standard deviations were generally lower than the means found
in child assessment norming samples and in studies using similar
measures with a more nationally representative sample of children
and families.
- Adequate Internal Consistency Reliability. After discussion within
the consortium and consultation with outside experts, we decided
to include measures with internal consistency reliability of .65
and above in our impact analyses.
- Consistent Reliability across Major Race/Ethnicity Subgroups.
We examined internal consistency reliability across our three major
race/ethnicity groups, white non-Hispanics, black non-Hispanics,
and Hispanics, to determine whether our measures had similar levels
of reliability across these groups.
To prepare our data for analysis, we first consulted the literature and either scored questionnaires and child assessments as they had been scored by the author of the measure or we used a scoring approach consistent with the current literature. For new measures or for measures which required additional data reduction, we conducted factor analyses as needed. We also coded the parent-child structured play assessments and analyzed the ratings. The factor analysis and coding procedures are described below.
a. Factor Analysis Approach
We used exploratory factor analysis techniques with Varimax rotation to create variables from multi-item questionnaire and observational measures. All factor analyses were conducted using only nonmissing child- and parent-level data. We used the following criteria to judge the adequacy of our factor analysis results:
- Items within factors made sense conceptually
- The solution yielded internal consistency reliability (coefficient
alpha) of .65 or greater within each factor
- The solution minimized the number of items with appreciable loadings
(.35 and greater) on multiple factors
- The solution minimized the number of items that did not load
appreciably on any factor
b. The Bayley Language Score
As described in Chapter IV, we found that impacts on the cognitive and language measures at 24 months varied by program approach. To investigate whether the results were biased by the fact that parents reported about their children's language skills, we conducted a factor analysis of the Bayley MDI items to identify a set of items that might serve as a validation test of the language findings from the parent-report measure. We created a Bayley data set based on each child's individual-item Bayley scores on items 113 through 154 (the 23- to 25-month through 26- to 28-month item sets). If a child did not have an item score for a particular item because it came before their basal or after their ceiling, the appropriate score was assigned (1 if below the basal, 0 if above the ceiling). Using this data set, we conducted exploratory factor analyses to determine the underlying factor structure. We used Varimax rotations and considered factor loadings greater than .35 to be appreciable.
The 24-month two-factor solution included a first factor made up of 12 language items and a second factor made up of 15 visual-spatial items. When an item loaded appreciably on both factors, we included the item in the factor on which it had the highest loading. We created factor scores by summing the items with loadings greater than .35. The two factors account for about 22 percent of the total variance in the Bayley items, with each factor accounting for about 11 percent of the variance. The alphas are acceptable for both factors: .86 and .80 for language and visual-spatial, respectively.
Before conducting impact analyses of the factor scores, we hypothesized that if the Bayley Language score and the MacArthur scores measured the same underlying language construct, they would show a similar pattern of impacts and thereby "validate" the MacArthur data. We had no specific hypotheses about the Visual-Spatial score and did not use it in this report. As described in Chapter IV, we found that there is consistency across the three program approaches between parent reports of language and children's language ability as measured by the Bayley Language score.
c. Coding of the Parent-Child Structured Play Assessment and Variable Creation
All videotapes of the 24-month parent-child structured play assessments were coded by staff at the Center for Children and Families, Columbia University, Teachers College, according to scales adapted from the NICHD Study of Early Child Care's Three Box coding scales (NICHD Early Child Care Research Network 1997, 1999; Owen 1992; Owen et al. 1993). There are nine seven-point coding scales that address child and parent behaviors. The three child scales address engagement of parent (extent to which child initiates and/or maintains interaction with parent); sustained attention with objects (degree of child's involvement with toys in the three bags); and negativity toward parent (degree to which child shows anger or hostility toward parent).
The six parenting scales address sensitivity (the extent to which the parent takes the child's perspective, accurately perceives the child's signals, and promptly and appropriately responds to these signals); positive regard (demonstration of love, respect, admiration); stimulation of cognitive development (teaching, actively trying to expand the child's abilities); detachment (under-involvement and lack of awareness, attention, engagement); intrusiveness (over-involvement, over-control); and negative regard (discontent, anger, rejection). Box C.2A includes more information about the individual coding scales.
We conducted preliminary analyses examining correlations among these scales, possible underlying factors, and internal consistency. Based on our analyses, we created a composite parenting score, "supportiveness" (coefficient alpha = .83), by computing the mean scores for parental sensitivity, cognitive stimulation, and positive regard, which were highly and significantly correlated (correlations ranged from .52 to .67). The scales assessing parental insensitivity (detachment, intrusiveness, and negative regard) and the child scales (engagement of parent, sustained attention with objects, and negativity toward parent) were retained as individual scales. The correlations among the three child scales were moderate to high (statistically significant correlations of -.34 to .55). The correlations among the four parenting scales were small to moderate and statistically significant (correlations of .11 to .40), with the exception of supportiveness and detachment (correlation of -.56, significant) and intrusiveness and negative regard (correlation of .52, significant).
A trained coding team leader worked with a six-member coding team to establish and maintain inter-rater reliability throughout the coding period. For the coding of the 24-month parent-child structured play assessment, inter-rater reliabilities on the 9 seven-point scales between the team leader and six coders were established to a criterion of 85 percent (exact or within one point agreement). Thereafter, the team conducted intermittent inter-rater reliability checks on a randomly selected 15 percent of each coder's weekly videotape assignment. A total of 151 tapes (8.5 percent of the 1,782 codable tapes) served as reliability tapes. Percent agreement (exact or within one point) averaged 93 percent across all reliability checks for all coders, with a range of 84 to 100 percent.
d. Psychometric Information for Key Constructed Variables
Table C.2A presents key psychometric data for the main constructed variables included in this report. Table C.2B presents psychometric data for the father study constructed variables. The tables are organized by measurement domain. We include the sample size, the possible range of values for each variable, the actual range found in the Early Head Start sample, the sample mean, standard deviation, and the internal consistency reliability (coefficient alpha). In Table C.2A, these psychometric data are presented for the full sample, that is, with the program and control groups combined. In Table C.2B, these psychometric data are presented for the program group.
| BOX C.2A
24-MONTH CODING SCALES FOR THE PARENT-CHILD Child Scales Engagement of Parent Reflects the extent to which the child shows, initiates, and/or maintains interaction with the parent. This may be expressed by approaching or orienting toward parent, establishing eye contact with parent, positively responding to parent's initiations, positive affect directed to parent, and/or engaging parent in play. Sustained Attention Measures the degree to which the child is involved with the toys presented in the three bags. Indicators include the degree to which child "focuses in" when playing with an object and the extent to which child coordinates activities with several objects and/or explores different aspects of a toy. Negativity toward Parent Reflects the degree to which child shows anger, hostility, or dislike toward parent. Expressions may be overt (for example, forcefully rejecting a toy offered by parent or pushing parent away) or covert (for example, hitting or throwing an object in response to parent's behavior). Parent Scales Sensitivity Measures the degree to which the parent observes and responds to the child's cues (gestures, expressions, and signals) during times of distress as well as non-distress. Key features include being child-centered, "tuning in" to the child, manifesting an awareness of child's needs, moods, interests, and capabilities, being flexible in supporting and responding to child's emerging need for autonomy, control, independence, and mastery even while enforcing necessary rules, regulations, and constraints. Positive Regard Assesses the parent's expression of love, respect and/or admiration for the child. Key features include verbal praising of child's efforts and successes, words of encouragement or support, and nonverbal affect, the way in which parent watches child attentively and looks into the child's face. Stimulation of Cognitive Development Measures the quality and quantity of the parent's effortful teaching to enhance child's perceptual, cognitive, and linguistic development. Key features include being aware of the child's developmental level, efforts to bring the child above that level, flexibility and timing of instructions or explanations, and use of complex and varied language. Detachment Measures the parent's lack of awareness, attention, and engagement with the child. Key features include being inattentive, perfunctory, or cold when interacting with child or, at the higher levels, complete lack of attention to or interaction with child. Intrusiveness Assesses the degree to which the parent exerts control over the child rather than acting in a way that recognizes and respects the validity of the child's perspective. Intrusive interactions are clearly adult-centered rather than the child-centered and involve imposing the parent's agenda on the child despite signals that a different activity, level or pace of interaction is needed. Negative Regard Reflects the parent's expression of discontent with, anger toward, disapproval of, and/or rejection of the child. This may be expressed verbally (words of derogation or disregard toward child) or physically (parental roughness, grabbing, or hitting child). NOTE: Scales are assessed on a seven-point scale, "1" indicating a very low incidence of the behavior and "7" indicating a very high incidence of the behavior. Scales were adapted by Christy L. Brady, Claudia O'Brien, Lisa Berlin, and Anne M. Ware and are based on the "Early Head Start 14-month Child-Parent Interaction Rating Scales for the Three Bag Assessment" (Ware, Brady, O'Brien, and Berlin 1998), the NICHD Study of Early Child Care 15-, 24-, and 36-month ratings of Parent-Child Interaction, and the "Manual for Coding Freeplay - Parenting Styles from the Newark Observational Study of the Teenage Parent Demonstration" (Brooks-Gunn et al. 1992). |
| Possible Range | Range | |||||||
|---|---|---|---|---|---|---|---|---|
| Measure | Sample Size | Minimum | Maximum | Minimum | Maximum | Mean | Standard Deviation | Internal Consistency Reliabilitya |
| CHILD COGNITIVE DEVELOPMENT | ||||||||
| Bayley Scales of Infant Development - Second Edition: Mental Development Index (MDI) | 1,739 | 49 | 150 | 49 | 134 | 89.2 | 13.7 | NA |
| CHILD LANGUAGE DEVELOPMENT | ||||||||
| MacArthur Communicative Development Inventories (CDI)-Vocabulary Production | 2,026 | 0 | 100 | 0 | 100 | 54.7 | 22.9 | .98 |
| MacArthur CDI-Sentence Complexity | 1,943 | 0 | 37 | 0 | 37 | 8.2 | 8.4 | .95 |
| Parent-Child Structured Play: Engagement | 1,732 | 1 | 7 | 1 | 7 | 4.3 | 1.1 | NA |
| Parent-Child Structured Play: Negativity Toward Parent | 1,732 | 1 | 7 | 1 | 7 | 1.7 | 1.0 | NA |
| Parent-Child Structured Play: Sustained Attention with Objects | 1,732 | 1 | 7 | 1 | 7 | 5.0 | 1.0 | NA |
| Bayley Behavioral Rating Scale (BRS) - Emotional Regulation | 1,868 | 7 | 35 | 7 | 35 | 25.3 | 5.5 | .92 |
| Bayley BRS- Orientation/Engagement | 1,870 | 6 | 30 | 6 | 30 | 22.4 | 4.3 | .83 |
| Child Behavior Checklist- Aggressive Subscale | 2,052 | 0 | 60 | 0 | 60 | 21.6 | 10.6 | .91 |
| EMOTIONAL SUPPORT FOR THE CHILD | ||||||||
| HOME: Emotional Responsivity | 1,902 | 0 | 7 | 0 | 7 | 6.1 | 1.4 | .74 |
| Parent-Child Structured Play: Supportiveness | 1,732 | 1 | 7 | 1 | 7 | 4.0 | 1.0 | .83 |
| PARENT'S STIMULATION OF LANGUAGE AND LEARNING | ||||||||
| Home Observation for Measurement of the Environment (HOME): Total Score | 1,904 | 0 | 31 | 8.3 | 31 | 26.4 | 3.5 | .76 |
| HOME: Support of Cognitive, Language,and Literacy Environment | 2,096 | 0 | 12 | 0 | 12 | 10.2 | 1.7 | .68 |
| Parent-Child Activities | 2,072 | 1 | 6 | 1 | 6 | 4.5 | 0.8 | .78 |
| HOME: Maternal Verbal-Social Skills | 1,949 | 0 | 3 | 0 | 3 | 2.8 | 0.6 | .71 |
| NEGATIVE PARENTING BEHAVIOR | ||||||||
| Parent-Child Structured Play: Detachment | 1,730 | 1 | 7 | 1 | 7 | 1.4 | 0.9 | NA |
| Parent-Child Structured Play: Intrusiveness | 1,732 | 1 | 7 | 1 | 7 | 1.9 | 1.0 | NA |
| Parent-Child Structured Play: Negative Regard | 1,732 | 1 | 7 | 1 | 7 | 1.4 | 0.8 | NA |
| HOME: Absence of Punitive Interactions | 1,900 | 0 | 5 | 0 | 5 | 4.4 | 1.2 | .78 |
| KNOWLEDGE OF CHILD DEVELOPMENT | ||||||||
| Knowledge of Infant Development Inventory (KIDI) | 2,088 | 1 | 4 | 1.8 | 4.0 | 3.4 | 0.4 | .56b |
| DISCIPLINE STRATEGIES | ||||||||
| Mild Discipline Only | 2,104 | 0 | 1 | 0 | 1 | 0.4 | 0.5 | NA |
| Discipline Severity Index | 2,104 | 1 | 5 | 1 | 5 | 2.7 | 1.7 | NA |
| SELF-SUFFICIENCY | ||||||||
| Family Resource Scale | 2,223 | 39 | 195 | 68.3 | 195 | 152.9 | 19.4 | .91 |
| PARENT MENTAL HEALTH AND FAMILY FUNCTIONING | ||||||||
| Parenting Stress Index (PSI)-Parent- Child Dysfunctional Interaction | 2,077 | 12 | 60 | 12 | 56.7 | 17.2 | 5.8 | .78 |
| PSI-Parental Distress | 2,078 | 12 | 60 | 12 | 60 | 25.4 | 9.3 | .82 |
| Family Environment Scale (FES)- Conflict | 1804 | 1 | 4 | 1 | 4 | 1.71 | 0.54 | .67 |
| Composite International Diagnostic Interview (CIDI) - Short Form: Major Depression (probability)c | 2,104 | 0 | 90.8 | 0 | 90.8 | 12.6 | 30.0 | NA |
| Source: Parent interviews,
child assessments, interviewer observations, and assessments of
parent-child structured play assessments conducted when children
were approximately 24 months old, and Parent Services Interviews
conducted approximately 15 months after enrollment. aReliability was estimated using Cronbach’s coefficient alpha formula. (back) bThe KIDI items we used were a subset of the 20 used by the IHDP study. Although the resulting summary score did not meet our .65 internal consistency reliability criterion, we included the score in the impact analysis because parent knowledge was a key outcome for many of the programs and these items have been used successfully in other studies with other samples. It is likely that our reduction of the number of items resulted in the reduced reliability. (back) cA skip logic error in the version of the CIDI that we used prevented us from scoring the CIDI in the usual way. Based on the advice of the CIDI developer, we created 2 versions of the CIDI scores-a lower and upper bound (the true CIDI score is between these two scores). The lower and upper bound scores tend to be 1 to 4 percentage points apart for the full sample and most subgroups. The impact estimates and their significance using both versions are very similar. In the report, we use the lower bound version of the measure (the most conservative estimate of the probability of depression). (back) |
||||||||
| Possible Range | Range | |||||||
|---|---|---|---|---|---|---|---|---|
| Measure | Sample Size | Minimum | Maximum | Minimum | Maximum | Mean | Standard Deviation |
Internal Consistency Reliability a |
| MOTHER REPORT OF FATHER-CHILD ACTIVITIES | ||||||||
| 14-Month Father-Child Activities |
1,045 | 0 | 20 | 0 | 20 | 13.8 | 1.2 | 0.77 |
| 24-Month Father-Child Activities |
1,045 | 0 | 20 | 0 | 20 | 14.8 | 1.3 | 0.81 |
| FATHER REPORT OF FATHER-CHILD ACTIVITIES | ||||||||
| 24-Month Caregiving Scoreb | 347 | 1 | 73 | 1 | 73 | 49.4 | 11.1 | 0.84 |
| 24-Month Social Activities Scoreb |
347 | 1 | 73 | 1 | 73 | 48.4 | 11.6 | 0.71 |
| 24-Month Cognitive Play Scoreb |
347 | 1 | 73 | 1 | 73 | 49.4 | 10.6 | 0.76 |
| 24-Month Physical Play Scoreb |
347 | 1 | 73 | 1 | 73 | 49.4 | 11.1 | 0.72 |
| Source: Parent interviews
conducted when children were approximately 14 months and 24 months
old, and father interviews conducted when children were approximately
24 months old. aReliability was estimated using Cronbach’s coefficient alpha formula. (back) bStandard (T) scores based on factor analysis of frequency of father-child activities. (back) |
||||||||
| Table of Contents | Previous | Next |