ACF OPRE: National Survey of Child and Adolescent Well-Being (NSCAW)

Appendix B: Reliability of NSCAW Measures

Measures were selected for NSCAW based on a variety of criteria, including the psychometric data available about their internal consistency reliability (Cronbach, 1951). Because scores on a measure are a function of characteristics of testing conditions, respondents, and test items, the internal consistency of a measure may vary with different respondents and test circumstances. For this reason, we reassessed the internal consistency of these measures as used in NSCAW with a unique client population assessed at home. “ ‘…It is imperative to recognize that the relevant reliability estimate is the one obtained for the sample used in the study under consideration’ (Pedhazur & Schmelkin, 1991, p. 86). It is this coefficent that governs the degree of attenuation (or, alternatively, the validity of the scores), not the coefficient from the test manual” (Vacha-Haase, Henson, & Caruso, p. 565).

Using the CORR procedure in SAS with weighted data, Cronbach’s alpha was calculated for 20 of the 23 standardized measures used in NSCAW. SPSS was used to calculate the Kuder Richardson-20 coefficient for internal consistency of dichotomous data for the HOME-SF. Internal consistency was not calculated for the BDI because “the BDI does not satisfy a basic assumption. Internal consistency coefficients assume that the items of a test or test component all measure the same skill or trait…Since there are no domains or subdomains of the BDI that appear to have this characteristic, estimating reliability using internal consistency measures was deemed inappropriate” (Newborg, Stock, Wnek, Guidubaldi, & Svinicki, 1984. p. 53).

Overall, internal consistency ranges from good (.70) to high (>.90) for the majority of measures used with the population of children involved with the child welfare system. Exceptions are the HOME-SF with alphas as low as .27, the BINS for 18 months (.64), MBA Reading (.61), Peer Loneliness and Dissatisfaction for 5- to 7-year-olds (.57), and the SF-12 Physical Health (.59). Internal consistency is quite low (.27 to .62) for most of the HOME-SF subscales. The coefficient alphas for some of the CTS-PC scales are also low, including .11 for parebtal reporting of Severe Physical Assault and .39 for the Neglect subscale. Regarding the CTS-PC, the low alphas may be a result of the inclusion of items with rare occurrences (Straus, 2004). The Peer Loneliness and Social Dissatisfaction scale, which was modified for NSCAW to use with 5- to 7-year-olds, has no previous history of use with such young children. This may explain the lower coefficient. Consequently, information for these younger children should be interpreted with caution (Table B-1).

Ariana Wall, MSW
NSCAW Research Group
School of Social Work
University of North Carolina
3/24/2003

Table B-1. Internal Consistency of Standardized Measures (Core Sample)

Measure

Normative Samples

NSCAW Sample

Sample Size

Cronbach’s Alpha

Sample Size (Unweighted)

Cronbach’s Alpha (Weighted)

BDI(1)

800

Not applicable

BINS(2)

3 months

150

.73

155

.76

6 months

166

.83

306

.74

9 months

148

.84

421

.82

12 months

150

.73

417

.73

18 months

137

.83

206

.64

24 months

152

.85

.84

CBCL (2 – 3 years)(3)

Total Problem Behavior

624

.96

523

.95

Externalizing

624

.93

570

.91

Internalizing

624

.88

568

.80

CBCL (4+ years)(4)

Total Problem Behavior

4,220

.96

2,755

.96

Externalizing

4,220

.93

2,977

.92

Internalizing

4,220

.89 (♂ 4-11 yrs)/.90 (♂ 11+ & ♀ 4- 11 yrs)/.96 (♀ 11+)

3,024

.90

CDI(5) Total

1,266

.86

Males 7-12 yrs

655

.78

Females 7-12 yrs

700

.84

Males 13+ yrs

254

.86

Females 13+ yrs

378

.87

CTS1 (Domestic Violence)(6)(7)

2,143

Physical Violence Section=.82
(husband to wife)

Any Domestic Violence

4008

.90

Minor Violence

4008

.77

Severe Violence

4008

.86

CTS-PC

Child report

Data not available because normed on adults only

988

Total=.85

1,023

Nonviolent Discipline=.50

1,047

Psychological Aggression=.81

1,034

Minor Physical Assault (Corporal Punishment)=.70

1,055

Severe Physical Assault=.77

1,068

Very Severe Physical Assault=.35

1,027

Total Physical Assault=.85

CTS-PC(8)

Caregiver report

3,861

Total Scale With Supplementary Neglect Scale Included=.79

1,000

Nonviolent Discipline =.70

3,945

Neglect =.39

3,917

Nonviolent Discipline=.77

1,000

Psychological Aggression=.60

3,935

Psychological Aggression=.66

3,946

Minor Physical Assault(1) (Corporal Punishment) =.57

1,000

Severe Physical Assault=-0.2

3,961

Severe Physical Assault =.11

3,968

Very Severe Physical Assault=.22

1,000

Total Physical Assaul=.55

3,940

Total Physical Assault=.54

1,000

Neglect = .22

3,945

Neglect Subscale=.39

HOME-Short Form(9)

0-2 years

Total

1,053

.44

Cognitive Stimulation

1,663

.36

Emotional Support

1,145

.27

3-5 years

Total

472

.58

Cognitive Stimulation

1,541

.71

755

.62

Emotional Support

1,541

.59

502

.41

Physical Environment

1,541

.70

695

.71

6-10 years

Total

900

.59

Cognitive Stimulation

1,233

.48

Emotional Support

1,033

.48

Physical Environment

239

.74

KBIT(10)

Composite

1,522

.92

2,964

.84

Verbal

1,522

.91

2,964

.76

Matrices

1,522

.85

3,011

.79

MBA(11)

5-18 yrs

Reading

1,257

.94

2,491

.74

Math

1,261

.90

2,459

.61

Peer Loneliness and Social Dissatisfaction(12)(13)

5 - 7 years

684

.70

8 + years

506

.90

1,874

.88

PLS-3(14)

Total Language

1,200

.87

N ranges from 244 to 1561 responses per item

Too much missing data to calculate

Auditory Comprehension

1,200

.76

Expressive Communication

1,200

.81

RAPS Relatedness Scale(15, 16)

Total

467

.88

Emotional Security

Primary Caregiver

1024

.69

Secondary Caregiver

503

.76

Involvement

Primary Caregiver

1007

.65

Secondary Caregiver

492

.69

Autonomy Support

Primary Caregiver

1139

.36

Secondary Caregiver

499

.28

Structure

Primary Caregiver

1118

.52

Secondary Caregiver

490

.66

Satisfaction

With Caseworker

2,965

.82

With Service Adequacy

1,773

.73

School Engagement

2371

.84

SF-12(17)

Mental Health

232

.76

5,352

.79

Physical Health

232

.89

5,364

.59

SSRS - Caregiver report(18)

3 - 5 years

~200

.90

714

.90

6 - 10 years

2,400

.87

1,287

.89

11+ years

1,770

.90

944

.91

SSRS - Teacher report(18)

3 - 5 years

~200

.94

827

.95 (grades K - 6) 4 - 14yrs

6 - 10 years

2,400

.94

235

.94 (grades 7-12) 5 – 15 yrs

11+ years

1,770

.93

TRF(17)

Total Problem Behavior

5-18 yrs ♂	5-11 yrs ♀	12-18 yrs ♀
.97	.97	.98
.96	.95	.96
.90	.91	.92

896

.96

Externalizing

1,186 (♂)

1010

.89

Internalizing

1,364 (♀)

1052

.95

VABS Daily Living Skills(20)

0-2 years

600

.91-.86

1,159

.91

3-5 years

600

.88-.91

751

.77

6-10 years

1,000

.83-.92

714

.78

VEX-R(21)(22)

Total

2,738

.96

Mild Violence Witness

2,738

.86

Mild Violence Victimization

Exposure =.80 (witness+victim)

2,738

.88

Severe Violence Witness

Exposure =.86 (witness+victim)

2,738

.92

YSR(23)

Total Problems

1,108

.95

1,058

.96

Externalizing

1,108

.89

1,130

.90

Internalizing

1,108

.89 (♂) /.91 (♀)

1,112

.91

References for Appendix B

Newborg, J., Stock, J. R., Wnek, L., Guidubaldi, J., & Svinicki, J. (1984). Battelle Developmental Inventory: With recalibrated technical data and norms: Examiner's manual: Riverside Publishing.
Aylward, G. (1995). Bayley infant neurodevelopmental screener manual. San Antonio, TX: Harcourt Brace.
Achenbach, T. M. (1991). Manual for the child behavior checklist 2 - 3 and 1991 profile. Burlington, Department of Psychiatry, University of Vermont.
Achenbach, T. M. (1991). Manual for the child behavior checklist 4 - 18 and 1991 profile. Burlington, Department of Psychiatry, University of Vermont.
Kovacs, M. (1992). Children's depression inventory. North Tonawanda, NY, Multi-Health Systems, Inc.
Straus, M. A., & Gelles, R. J. (1990). Physical violence in American families: Risk factors and adaptations to violence in 8,145 families. New Brunswick, NJ: Transaction.
Straus M. (1990). Measuring physical and psychological maltreatment of children with the conflict tactics scales. Durham, NH: University of New Hampshire Family Research Laboratory.
Straus, M., Hamby, S., Finkelhor, D., Moore, D., & Runyan, D. (1998). Identification of child maltreatment with the parent-child conflict tactics scales: development and psychometric data for a national sample of American parents. Child Abuse & Neglect, 22(4), 249-270.
Sugland, B. W., Zaslow, M., Smith, J. R., Brooks-Gunn, J., Coates, D., Blumenthal, C., Moore, K. A., Griffin, T., & Bradley, B. (1995). The early childhood HOME inventory and HOME-short form in differing racial/ethnic groups Are there differences in underlying structure, internal consistency of subscales, and patterns of prediction? Journal of Family Issues, 16(5), 632-663.
Kaufman, A., & Kaufman, N. (1990). Kaufman brief intelligence test manual. Circle Pines, MN: American Guidance Service.
Woodcock, R., McGrew, K., & Werder, J. (1994). Woodcock-McGrew-Werder Mini-Battery of Achievement. Itasca, IL: Riverside Publishing.
Asher, S., & Wheeler, V. (1985). Children's loneliness: a comparison of rejected and neglected peer status. Journal of Consulting and Clinical Psychology, 53(4), 500-505.
Parkhurst, J. T., & Asher, S. R. (1992). Peer rejection in middle school- subgroup differences in behavior, loneliness, and interpersonal concerns. Developmental Psychology, 28(2), 231-241.
Zimmerman, I., Steiner, V., & Pond, R. (1992). Preschool language scale-3: examiner's manual. San Antonio, TX: The Psychological Corporation.
Connell, J. P. (1990). Context, self, and action: A motivational analysis of self-system processes across the life span. In D. Cicchetti & M. Beeghly (Eds.), The Self in Transition (pp. 61-97). Chicago: University of Chicago Press.
Lynch, M., & Cicchetti, D. (1991). Patterns of relatedness in maltreated and nonmaltreated children: Connections among multiple representational models. Development and Psychopathology, 3, 207-226.
Ware, J., Kosinski, M., & Keller, S. (1998). SF-12: How to score the SF-12 physical and mental health summary scales (3 ed.). Lincoln, RI: Quality Metric Incorporated.
Gresham, F., & Elliott, S. (1990). Social skills rating system manual. Circle Pines, MN: American Guidance Service.
Achenbach, T. M. (1991). Manual for the teacher's report form and 1991 profile. Burlington, VT: University of Vermont Department of Psychiatry.
Sparrow, S. S., Balla, D. A., & Cicchetti, D. V. (1984). Vineland Adaptive Behavior Scales: Interview Edition, Survey Form Manual. Circle Pines, MN: American Guidance Service.
Raviv, A., Erel, O., Fox, N., Leavitt, L., Raviv, A., Dar, I., Shahinfar, A., & Greenbaum, C. (2001). Individual measurement of exposure to everyday violence among elementary school children across various settings. Journal of Community Psychology, 29(2), 117-140.
Shahinfar, A., Fox, N., & Leavitt, L. (2000). Preschool children's exposure to violence: relation of behavior problems to parent and child reports. American Journal of Orthopsychiatry, 70(1), 115-125.
Achenbach, T. (1991). Manual for the youth self-report and 1991 profile. Burlington, Department of Psychiatry, University of Vermont.

Table of Contents | Previous