ACF OPRE: National Study of Child Care of Low-Income Families

Chapter Two: Overview of the In-Depth Study of Family Child Care

The study was designed to answer a broad range of questions about family child care providers and the family child care environment, as well as about the families that use family child care.

With respect to providers, the study addressed the following questions:

What is the background, education and experience of the providers?
What is their motivation for providing child care services?
What is the nature of the relationship between parents and providers?

The study investigated the following questions about the family child care environment:

What are the characteristics of the care environment?
What is the nature of young children’s experience in the child care setting?
What is the level of child functioning (in terms of language, social play and play with objects) in the child care setting?
How do children’s experiences change over time?

With respect to families, the study addressed the following questions:

What are the factors that influence parents to choose a care arrangement? Do these change over time as children grow older?
How do child care arrangements change over time and what are the reasons for the changes?
Does the presence or absence of a subsidy affect the stability and continuity of the child care arrangement?
What happens to child care arrangements when families lose their subsidy?
How do aspects of the family child care arrangement, such as the parent’s relationship with the provider, the stability, flexibility of the arrangement, etc., affect parents’ ability to work and to balance the competing demands of family and work?

The In-Depth Study was conducted in five of the 25 counties that are part of the National Study of Child Care for Low-Income Families. The five communities were purposively selected to offer geographic diversity, variation in state regulatory and subsidy policies, some variation in the rate of child poverty, and variation in ethnic mix. An important consideration was the willingness of local officials to cooperate and support the study. Finally the counties needed to contain a large enough number of subsidized families to make it possible to represent them in the study sample. The five counties selected were: Los Angeles County in California; Hamilton County (Cincinnati) in Ohio; Harris County (Houston) in Texas: King County (Seattle) in Washington; and Franklin County in Massachusetts, the one rural county chosen.

The design called for the selection of 650 low-income working parents with at least one child under age nine in family child care, and their 650 linked providers, across the five sites. Families had to be receiving or eligible for a child care subsidy. If more than one child was in family child care, one child was randomly selected to be the focus child. The sample was stratified by subsidy status and age of child. Exhibit 2-1 shows the sample design. As the exhibit shows, the sample was designed to over-represent families that were receiving child care subsidies at recruitment1. Once recruited, families were followed over the 2½ -year data collection period, regardless of changes in their child care arrangements. Parents and providers were interviewed every six to eight months, and, at the same time, the child was observed in the family child care environment (or another setting, if the care arrangement changed during the course of the study).

Exhibit 2-1: Sample Design for the In-Depth Study
		Community 1	Community 2	Community 3	Community 4	Community 5	Total
		Number of Families/Focus Children
Families receiving child care subsidies at start of study	Focus child < 5 yearsa	46	46	46	46	46	230
	Focus child 6-9 years	52	52	52	52	52	260
	All ages	98	98	98	98	98	490
Families NOT receiving child care subsidies at start of study	Focus child < 5 yearsa	14	14	14	14	14	70
	Focus child 6-9 years	18	18	18	18	18	90
	All ages	32	32	32	32	32	160
Total		130	130	130	130	130	650

a Child age at start of study

Measures for the Study

Three questionnaires and four observational measures were developed or adapted for the study. The questionnaires were designed to be administered in person to parents, providers and school-age children. All the measures are described briefly below.

Questionnaires

Parent Interview
The Parent Interview was designed to gather information on parents’ employment status and work history, barriers to employment, current child care arrangements for all children in the family, current and prior arrangements for the focus child, knowledge of the local child care market, knowledge of and experience with subsidies, out-of-pocket child care costs, considerations in choosing a provider, attitudes, values and beliefs about child care, flexibility of work and child care arrangement, work and child care as sources of stress, and the parent’s relationship with the provider. In addition, the interview gathered basic demographic information about the family.

Provider Interview
The Provider Interview obtained information about the provider’s education, training, child care experience and reasons for providing care. In addition, the interview probed the provider’s views on childrearing and the caregiver’s role, relationship with the child’s parents and the advantages and disadvantages of family child care. She was asked whether any of the care she provides was paid care and the fees charged for paid care, whether she received subsidies for any of the children and, if so, whether she experienced any problems with subsidies. The interview was also used to gather information about the care environment including: the number and ages of children cared for, the number of children who received a subsidy, the child care schedule and its flexibility, as well as items about health and safety practices that could not be directly observed.

School-Age Child Interview
An interview for school-age children was used in two circumstances: first, when the child was in an organized after-school program, where he or she was not being observed; and secondly, in cases where an observation in the care setting could not be scheduled or conducted. The interview included questions about who provided the care and about other adults and children in the care setting, activities that the child engaged in, both indoors and out and the child’s comfort with and security in the care setting.

Observation Measures

The four observation measures, supplemented with information from the interviews, were designed to measure aspects of the care setting and the child’s experience that were assumed to have implications for the child’s health, safety and overall development. The goal was to gather sufficient information to make possible a judgment about whether and to what extent the different forms of family child care included in the study are safe for children, provide a healthy environment for children’s physical development and growth, and provide adequate support for their cognitive and social development. A major challenge was to develop measures that were appropriate across a wide range of home-based care settings and that could also be used in centers (for children who changed settings). In addition, the measures had to be appropriate across a range of child ages wider than is typically found in child care or early childhood studies. The four measures are described briefly below.

The Environment Checklist rates aspects of the care environment including: space; environmental comfort; equipment and materials; indoor health and safety; outdoor health and safety; and dangerous situations in the setting. The Checklist consists of 77 items that are scored on a three-point scale: 1=usually true/or consistently evident; 2=partially or sometimes true/some evidence; and 3=not true/little or no evidence. The Checklist includes items from three sources: the National Association for Family Child Care (NAFCC) Quality Standards for Accreditation, Pilot Study Draft (Family Child Care Accreditation Project, Wheelock College, Boston, MA., 1977); the NAFCC Observation System for Accreditation, Draft (Family Child Care Accreditation Project, Wheelock College, Boston, MA., 1997); and Stepping Stones to Using Caring for Our Children: National Health and Safety Performance Standards Guidelines for Out-of-Home Child Care Programs—Protecting Children from Harm (National Research Center for Health and Safety in Child Care, University of Colorado, Denver, CO. Sponsored by the U.S. Dept. of Health & Human Services, Health Resources and Services Administration, Maternal and Child Health Bureau, 1997).

The Environment Snapshot provides a picture of the care setting at a point in time including: the adults and children in the setting; their activities and interactions (with the focus child indicated individually); and overall levels of engagement or distress in the setting. The Environment Snapshot is a synthesis of other child care snapshot measures that have been used by Abt and other researchers in previous studies. All of the prior Snapshots were developed with child care centers in mind (although the National Day Care Infant Study also looked at infants in family day care homes and in their own homes); therefore, we adapted the earlier measures to be equally applicable to center and home care.

The Provider Rating includes three parts. Part I rates the provider in terms of her relationship with children and support for learning activities. Part I consists of 55 items rated on a three-point scale: 1=usually true/or consistently evident; 2=partially or sometimes true/some evidence; and 3=not true/little or no evidence. Part II rates the provider on nine aspects of her response to the children, such as involvement, flexibility, etc. For each aspect, the provider’s responsiveness is rated on a 5-point Likert scale, from least like to most like the attribute. Parts I and II are adapted from the NAFCC Observation System for Accreditation (1997).

Part III of the Provider Rating is the Global Caregiver Rating Scale (Arnett, 1990), a 36-item scale that assesses the provider’s warmth, responsiveness, detachment, and involvement with the children. Each item is rated on a 4-point scale: 1=caregiver is “not at all like” the attribute, 2=caregiver is “somewhat like” the attribute, 3=caregiver is “quite a bit like” the attribute, 4=caregiver is “very much like” the attribute. The Arnett has been used in numerous child care studies, including settings and providers serving low-income children (e.g., the National Preschool Observation Study, the National Child Care Staffing Study).

The Child Observation describes the interactions and language of the focus child in the setting. The measure is a time-sample observation in which the child’s behavior is observed for 5 seconds and then recorded in the next 15 seconds. The focus child is observed for a total of 30 minutes during the half-day family day care visit. The observations are conducted when the child is not eating, napping, resting, or sleeping. In the observation, the focus child’s behavior is described in terms of 7 dimensions: (1) focus child with objects—whether and how the child is playing with objects, (2) focus child with peers—whether and how the child is interacting with peers, (3) focus child’s language—whether or not the child uses language and with whom, (4) focus child’s prosocial behavior—any prosocial or antisocial behavior displayed by the child and to whom, (5) peer responses to the focus child—any prosocial or antisocial behavior displayed by other children to focus child, (6) adult contact with focus child—the level of interaction between any adult and focus child, and (7) adult language with focus child—the content of any language directed one-on-one by an adult to the focus child.

For each five seconds of observation, one and only one code is selected from each of the seven categories. That is, the child’s behavior during the five seconds of observation is described in terms of the child’s level of play with objects, his/her level of play with peers, whether the child speaks to adults or peers, whether the child displays either prosocial or antisocial behavior, whether a peer displays these behaviors to the child, the response of any adult to the child, the level of the child/adult interaction, and the adult’s language to the child.

The Child Observation is modeled on the work of Carollee Howes at UCLA. Howes has developed a number of variants of a time-sampled observation measure that focuses on an individual child. The advantages of Howes’ coding system are (1) the child behaviors that are recorded can be linked to developmental outcomes; (2) the behaviors are linked to constructs that are relevant for children across a wide age range; (3) the behaviors are relevant and can be observed in all types of care settings; and (4) the observation system has been used with populations similar to the families and children in this study. After discussions with Dr. Howes, two of her observation measures were adapted to create a measure with the widest possible age application and the broadest range of constructs.

Observation Schedule

Over a half-day observation of a family child care home, all four observation measures were completed according to a schedule. Two of the measures--the Provider Rating and the Environmental Checklist-- were completed once, but based on the observations over the entire half-day. The Environment Snapshot and the Child Observation were completed multiple times across the half day, on the schedule shown in Exhibit 2.2. (Note that the schedule shown in Exhibit 2.2 represents an ideal schedule. The frequent off-hour care hours and relatively fluid daily schedules of child care homes meant that the schedule of observations often had to be adapted.) The important issues for the observations, regardless of the exact schedule, involve the two measures that were completed multiple times. For the Snapshot, the critical issue was to obtain as many Snapshots as possible, spaced relatively evenly apart, over the full observation period. For the Child Observation, the critical concern was to obtain the full 30 minutes of observation during the time that the child was awake and involved in activities, with the five-minute sessions also spread evenly apart.

Exhibit 2-2: Schedule of Administration of Observation Measures in a Half-Day Observation
	8-8:30 am	9-9:30	10-10:30	11-11:30	12-12:30	1pm
Environment Checklist [1 each half day]
Environment Checklist [1 each half day]
Provider Rating [1 per provider each half day]
Provider Rating [1 per provider each half day]
Environment Snapshot [1 every 20 minutes over half-day]
Environment Snapshot [1 every 20 minutes over half-day]
Child Observation [Two 15-minute coding periods, 1 record/20 sec]		45 20-sec records		45 20-sec records
Provider Interview [1 per visit]	at end of observation

Reliability of the Observation Measures

The reliability of the observation measures developed for the family child care study was established as part of the pre-testing of the measures prior to the actual data collection. Reliability was established in different ways for different measures. For the Environment Snapshot, two methods of establishing reliability were employed. First, a set of written vignettes (descriptions of children in a family child care home) were developed and pre-coded by the instrument developers. Another set of independent coders were trained on the measure and then asked to complete the vignettes. On the vignettes, reliability was calculated as the percent of agreement between the criterion coding and the coding by the observers being trained. The average level of agreement on the coding of the Environment Snapshot against the criteria was 95 percent. The reliability varied only slightly across the different components of the Snapshot (activities, roster counts, summary classroom descriptors). In the field, double-coding of Snapshots by two live coders had lower but still acceptable inter-rater agreement (86%) as shown by their correlation; this somewhat lower reliability was caused most often by the difficulty in synchronizing two coders to the exact moment in time reflected in the Snapshot coding.

For the Child Observation, the same two methods were employed to establish reliability. Against pre-coded descriptions of a child’s behavior in a five-second period, coders agreed with the criterion coding 87 percent of the time. The reliability for each of the seven categories ranged from 81 to 94 percent. In the field, inter-rater reliability averaged 79 percent, with the lower reliability again related often to the difficulty of coordinating two coders to observe exactly the same five seconds of the child’s behavior.

For the Environment Checklist, inter-rater reliability averaged 82 percent, with the disagreements always representing a difference of one point on the rating (a code of “always” versus a code of “sometimes” or “sometimes” versus “rarely”) rather than representing a large discrepancy in how coders perceived the environment (e.g., one coder choosing “always” and the second coder choosing “rarely”).

For the Provider Rating, one of the components—the Arnett Caregiver Rating—has been used in many other studies and has been reported to have high reliability. The remainder of the Provider Rating was assessed in terms of inter-rater agreement on the coding. Across the items, the average inter-rater agreement was 79 percent.

Recruiting the Sample

Our original strategy for recruiting the sample of parents and linked providers was to obtain linked lists of subsidized parents and family child care providers in each of the five sites from state child care subsidy agencies. Lists of subsidized providers are theoretically in the public domain, and child care agency staff in the five selected states assured us cooperation in getting lists of subsidized parents. Because these lists would contain many more names than we needed, our plan was to do the initial recruiting by telephone from Abt. Telephone interviewers would call randomly selected parents, screen for continued eligibility, attempt to recruit the parent and ask the parent to encourage her provider to participate. They would then call the linked provider, screen to ensure that she was indeed a family child care provider (rather than a center) and was still providing care for a child in the family just recruited, and attempt to recruit her to the study. We anticipated that the process might entail several callbacks until the parent and provider jointly agreed to participate. Once both members of the pair agreed to participate, the names would be sent to field staff, so that interviews and observations could be scheduled. Parents and providers were both offered a financial incentive to participate in the study.

Our plan was to recruit approximately 500 subsidized parents and their linked providers in this way and then to use these parents and providers to help us find approximately 150 unsubsidized families and their providers, to complete the sample of 650 paired parents and providers.

Recruiting Through Lists

In reality, this strategy could be implemented in only one site – Hamilton County (Cincinnati) Ohio, where the state was willing to merge parent and provider lists and where there was no legal impediment to our obtaining the merged list. A computer tape containing a list of 2200 parents and their providers was sent from the state to Abt Associates at the end of June 1999 and telephone recruitment began in July, after interviewers had been trained. Recruiters encountered some problems, because the list was somewhat outdated; the two lists had been downloaded sometime in early spring but the actual merging of the lists was delayed for at least two months because agency programmers were not able to do the necessary removal of center providers and merging of the remaining provider list with the parent list in a timely fashion. The result was that telephone interviewers encountered many parents who were no longer receiving subsidies or using non-parental care.

Almost 200 pairs were recruited in this way and sent to the field. In the field, however, interviewers encountered a variety of situations that reduced the number of pairs to about 100. In some cases, the parent had stopped working when the interviewer contacted her, or had moved the child to a different care setting (e.g. a child care center). In other cases, the provider had changed her mind about participating in the study or was no longer caring for the child. At this point, interviewers effectively became field recruiters, struggling to find another eligible child or recruit a new provider. As other studies have found, parents were easier to recruit than providers, even though we had raised the incentive payment for providers from $25 per visit to $50.

No other state was able to provide us with a merged list in the way that Ohio did. In three of the other four states, new regulations required active consent on the part of parents and providers before their names could be released to us. In King County, Washington, the state was willing to accept passive consent, but the process was delayed by a complicated and lengthy human subjects review, required of all studies that use agency data in Washington State. Senior staff at Abt worked with agency staff over a period of nine months, preparing the original submission, responding to the IRB committee’s questions, submitting all letters, flyers and other study material for their review and revising materials to make them suitable for very low-literacy families. IRB clearance was received in February 2000.

Using materials prepared by Abt, the state mailed out letters to parents and providers on their subsidy list, supporting the study and asking recipients to respond only if they were not willing to be contacted (passive consent). The first mailing went only to parents who were using licensed providers and their linked providers. At the time, the state was embroiled in a court case in which a major newspaper in the state was suing the state to obtain its list of informal providers who were receiving public subsidies. Agency staff felt that our chances of a positive response would be increased if a mailing to informal providers was delayed for two to three months. The letters to parents using informal providers, and their linked providers went out in May 2000.

Once the lists had been purged of parents and providers who refused permission to release their names the state provided names and contact information to Abt and recruiting began by telephone.

In Harris County (Houston) Texas, the subsidy lists were maintained by the local subsidy agency, whose staff were very willing to cooperate. However, they were constrained by a ruling from the state’s legal staff that active consent was required. As in Washington State, the subsidy agency sent out letters in English and Spanish to thousands of parents and providers asking them to give active consent to be contacted and recruited, and supporting the study. Only ten pairs of names were obtained through this effort.

A similar procedure was followed in Franklin County, Massachusetts, the single rural site in the study. Although we selected this site because it was estimated to have more subsidized families than our other rural sites, the number of families potentially eligible to participate was less than twice the desired sample. The initial mailing was, therefore, to a very small number of parents and providers and, since active consent was required, yielded only a handful of pairs to be recruited.

In Los Angeles County, California, a similar procedure was followed, but only after prolonged negotiations with the county agency that administers subsidies. The agency staff member at first assured us that only passive consent would be required, only to inform us a week before letters were sent out that he had received a legal ruling stipulating that active consent would be required. This mailing also produced a handful of parents and providers who agreed to be contacted.

The probable failure of this strategy became apparent to us in the fall of 1999, at the beginning of Year 3. We therefore decided to recruit and train on-site recruiters, who would work closely with the on-site interviewers but whose only job would be to find and recruit parents and providers.

Recruiting with On-Site Recruiters

In the fall of 1999, we decided to hire and train on-site recruiters in four of the five sites. In Ohio, interviewers were still working through parents and providers recruited through lists, and one of the interviewers was already actively pursuing other possible recruiting strategies.

Through our Field Managers, we placed ads in local papers, recruited, interviewed and hired four recruiters (we hired an African-American recruiter initially, in Los Angeles and then hired an additional Hispanic recruiter for that site), and brought them to Cambridge for training.

We instructed the recruiters to pursue a variety of strategies to recruit parents and providers including: searching the Web, local phone books and other media for provider lists and organizations; posting notices about the study in community colleges, stores, local churches, housing projects, WIC clinics and community agencies; talking to local child care resource and referral agencies (CCR&Rs) and other referral agencies. We trained recruiters to give a brief presentation about the study, if invited by provider organizations or church groups.

This strategy took some time to yield results, but the pace of recruiting speeded up after a year of intensive site work as the recruiters established connections with organizations in the five sites. In Houston, Neighborhood Centers, a local agency that works with both providers and parents, was willing to host monthly meetings at which we recruited from both groups. In California, a CCR&R agency, which was initially unwilling to help at all, sent out a letter to providers encouraging them to participate, and our Hispanic recruiter was very successful in recruiting hard-to-find relative providers. In Ohio, we recruited unsubsidized families from snowball scripts and went back to providers who were originally dropped because their paired parent had refused or was no longer using the arrangement. Recruiting in King County and Franklin County proceeded steadily through the use of similar strategies.

Franklin County, the rural county in Massachusetts, proved to be the smallest site, despite intensive efforts by the on-site recruiter. Therefore, we over-enrolled in other sites, to make up for the small sample there.

Exhibit 2.3 shows the final study sample.

Exhibit 2-3: Actual Sample for the In-Depth Study
		Community 1: CA	Community 2: MA	Community 3: OH	Community 4: TX	Community 5: WA	Total
		Number of Families/Focus Children
Families receiving child care subsidies at start of study	Focus child < 5 yearsa	26	43	72	25	29	195
	Focus child 6-9 years	27	9	79	65	32	212
	All ages	53	52	151	90	61	407
Families NOT receiving child care subsidies at start of study	Focus child < 5 yearsa	24	16	17	34	13	104
	Focus child 6-9 years	25	15	30	27	34	131
	All ages	49	31	47	61	47	235
Total		102	83	198	151	108	642

a Child age at start of study

As this description should make clear, the sample of families and providers recruited for this study is not a representative sample. By design, we recruited only providers who served low-income families and, even within that group, oversampled providers who received a child care subsidy for one or more of the children in their care. The variety of strategies used to find families and providers provided us with a very large sample, but it is still a sample of convenience, as in all the studies that preceded this one. The concern that providers who are willing to participate in such a study may offer higher-quality care is one that plagues any study in which participation is voluntary. We acknowledge that we may not have captured the worst care, although slightly more than one percent of the homes in the sample appeared to provide unsafe or inadequate care. Nevertheless, we believe that this account provides useful insights into the factors that shape the child care decisions of low-income families, the kind of care provided in the homes they select, and the implications for their children’s experience in that care.

1 Later in this chapter we describe the recruitment process and show the actual distribution across sites of families recruited into the study. (back)

Table of Contents | Previous | Next