Home > Programs & Funding > Developing a System of Program Accountability under the John H. Chafee Foster Care Independence Program

REPORT TO THE CONGRESS

DEVELOPING A SYSTEM OF PROGRAM
ACCOUNTABILITY UNDER THE
JOHN H. CHAFEE FOSTER CARE INDEPENDENCE
PROGRAM

The Department of Health and Human Services' Plan for Developing
and Implementing the National Youth in Transition
Information System

SEPTEMBER 2001

U.S. Department of Health and Human Services
Administration for Children and Families
Administration on Children, Youth and Families
Children's Bureau

This report was prepared under contract with Westat, with James Bell Associates as subcontractor, under Contract #282-98-0015, Task Order 24.

Additional copies of the report may be obtained from the National Resource Center For Youth Development at the following address:

National Resource Center for Youth Development
University of Oklahoma
College of Continuing Education
4502 E. 41st St., Bldg 4 West
Tulsa, Oklahoma 74135-2553
Phone: 918-660-3700
FAX: 918-660-3737
Nrcyd.htm
hlock@ou.edu

The report will also be available through the Children's Bureau web site at the following address:

http://www.acf.hhs.gov/programs/cb

Introduction

Each year approximately 16,000 young people are discharged from the Nation's foster care system, having reached the age at which they are no longer eligible for services. The age of discharge is generally 18, although most States continue to provide foster care services to young people until they are 21 or 23 years old (usually with the requirement that the youth are in school).

Over the past few decades, research and anecdotal evidence have indicated that many young people discharged from foster care to independent living experience numerous difficulties in their attempts to achieve self-sufficiency. Rather than making a successful transition to independent living, a significant percentage of these youth experience homelessness, unemployment, victimization, and dependency on various types of public assistance. Other research has shown, however, that involvement in independent living programs has had a positive impact on achievements. (See Appendix A to this report for a brief review of the research.) In response to this problem, the U.S. Congress enacted legislation designed to ensure that youth who "age out" of the foster care system have the necessary skills and experience to achieve self-sufficiency. The first piece of legislation, enacted in 1986, was the Title IV-E Independent Living Initiative (P.L. 99-272). As a result of this legislation, all States began to provide independent living services to foster care youth between 16 and 18 years of age in an effort to prepare them for independent living. These services include life skills classes, supervised living arrangements, mentoring, and/or employment readiness and training.

Although P.L. 99-272 increased the availability of independent living services for foster care youth, many service providers, youth advocates, and researchers expressed the opinion that a broader effort was necessary if these youth were to make a successful transition from foster care to independent living. To meet this need, the Chafee Foster Care Independence Act of 1999 (P.L. 106-169), enacted on December 14, 1999, established the Chafee Foster Care Independence Program (CFCIP). The Chafee legislation increases Federal funding for independent living services (Section 477(h)) and directs that States provide services to youth who leave foster care because they attain 18 years of age (Section 477(b)(3)(A)). All former foster care recipients between 18 and 21 years of age may be provided services depending upon an individual State's eligibility criteria. Also, under the CFCIP, the States' use of Federal funds for independent living services is not restricted to youth age 16 and older. Instead, States may provide these services to all children ". . . who are likely to remain in foster care until 18 years of age and to help these children make the transition to self-sufficiency. . ." (Section 477(a)(1)). States are to use Federal funds for a variety of financial, housing, counseling, employment, education, and other appropriate supports and services.

Although States have a great deal of flexibility in deciding how to use their CFCIP funds, the legislation suggests services that include assistance in obtaining a high school diploma; career exploration; vocational training; job placement and retention; training in daily living skills; training in budgeting and financial management skills; substance abuse prevention; and preventive health activities such as smoking avoidance, nutrition education, and pregnancy prevention (Section 477(a)(1)). The Chafee legislation also specifies that funding may be used to provide personal and emotional support to children aging out of foster care, through mentors and the promotion of interactions with dedicated adults (Section 477(a)(4)).

In addition to these features, the Chafee legislation focuses on the issue of State accountability for the outcomes experienced by youth when they leave foster care. Section 477(f)(1) of the legislation requires the Secretary of Health and Human Services (the Department), in consultation with various stakeholders, to:

"(A) develop outcome measures (including measures of educational attainment, high school diploma, employment, avoidance of dependency, homelessness, non-marital childbirth, incarceration, and high-risk behaviors) that can be used to assess the performance of States in operating independent living programs;

(B) identify data elements needed to track-
(i) the number and characteristics of children receiving services under this section;
(ii) the type and quantity of services being provided; and
(iii) State performance on the outcome measures. . . "

In accordance with the requirements to identify outcome measures that can be used to "assess the performance of States in operating independent living programs" and to identify data elements for purposes of tracking performance, the Department plans to develop a National Youth in Transition Information System (NYTIS). NYTIS is envisioned as a comprehensive system that integrates information pertaining to the characteristics of the youth served, services delivered, and outcomes achieved into a unique data collection and data reporting process. By specifying what is to be counted and measured by NYTIS, the Department will be establishing a framework for what the States are expected to accomplish for the youth they serve.

The basic premise guiding these initial efforts is that the focus of the Nation's child welfare systems must be on ensuring children's safety, permanency, and well-being. The national goals of child safety, permanency, and well-being were emphasized in the Adoption and Safe Families Act of 1997 and apply to all children served by child welfare systems regardless of age. Therefore, the assessment of State performance in operating independent living programs must focus on determining the extent to which a State's child welfare system provides adolescents in its care with the skills, services, and supports to allow them to transition into adulthood, where they can provide for their own permanency, safety, and well-being. The outcomes incorporated into NYTIS will reflect this focus.

The purpose of this Report to the Congress (as required by Section 477(f)(2) of the Chafee Foster Care Independence Act of 1999) is to present the Department's plan and timetable for developing and implementing NYTIS and establishing a penalty structure for States that do not comply with NYTIS data collection and reporting requirements. The plan specifies a number of activities designed to ensure the development of an effective information system, including an extensive consultation process for identifying outcomes and measures and a pilot test of the system on a limited basis prior to full implementation. Nationwide implementation of NYTIS is planned for October 2003.

With the enactment of the Chafee Foster Care Independence Act on December 14, 1999, the Department immediately took steps to implement the legislation. As a first step, an Implementation Strategy was developed to guide the Department's actions. In early 2000, a Program Instruction (PI) was issued, allowing States to access the newly available funds in an expeditious manner. In response to this PI, States submitted an abbreviated application describing their plans and progress in developing programs that complied with the Chafee legislative requirements, addressing the application requirements in Section 477(b) of the Chafee Foster Care Independence Act, and requesting funding for Fiscal Year 2000. A second PI, issued in February 2001, provided instructions for incorporating the CFCIP State Plan for Fiscal Years 2001-2004 into the existing five-year title IV-B Child and Family Services (CFS) Plan. The CFS Plan is the vehicle by which States request program funding from the Department for title IV-B, Child Abuse Prevention and Treatment Act (CAPTA), and CFCIP. Information on prior years' CFCIP activities will be included in the title IV-B Annual Progress and Services Report (APSR).

NYTIS Development and Implementation Components

The Department's plan for meeting the performance assessment requirements of the Chafee Foster Care Independence Act under Section 477(f) includes the following key components:

A. A conceptual framework for NYTIS that incorporates criteria and guidelines;
B. Outcomes and measures for assessing whether youth are making a successful transition from foster care to independent living;
C. Data elements pertaining to recipients of independent living services, the types of services provided, State performance on the outcome measures, and a data collection methodology;
D. A pilot test of NYTIS, including the data elements and data collection and reporting procedures;
E. A penalty process for noncompliance with Federal reporting requirements;
F. A period for public comment; and
G. National implementation of NYTIS.

The following sections provide a discussion of the activities to be undertaken under each component.

A. Develop a conceptual framework for NYTIS that incorporates criteria and guidelines.

During the fall of 2000, after reviewing the literature on effective performance assessment systems, conducting discussion groups, and examining performance assessment systems developed by both the Children's Bureau and the Family and Youth Services Bureau, the Department developed criteria and guidelines for NYTIS.

NYTIS will:

• Be guided by a statement of purpose and goals. The Department will establish a statement of purpose and a set of goals for NYTIS that will address: (1) why the system is necessary; (2) how performance information will be used; and (3) the ways that NYTIS can support State and local program staff and activities. The statement of purpose will be consistent with the purposes set forth in the Chafee Foster Care Independence Act and will reflect input from the field obtained through an extensive consultation process (described later in this chapter).
  
  
• Establish a set of outcomes that are both meaningful and measurable. In accordance with information obtained during the literature review and from discussion groups, the outcomes selected will be those that: (1) are consistent with the mission of the CFCIP; (2) reflect the statutory provisions of the CFCIP; (3) are meaningful to stakeholders, particularly the youth served by the program; (4) pertain to desired results for CFCIP program participants (e.g. focus on positive events); (5) reflect youth's strengths rather than deficits; and (6) are measurable.
  
  
• Incorporate measures that are reliable, valid, relevant to the program context, feasible to implement, and culturally relevant. The Department will adopt the framework established by Friedman (1997) for selecting outcome measures. In that framework, a useful outcome measure is one that has "communication power" (it is compelling and clear), "proxy power" (there is a strong relationship, established either through research or common sense, between the measure and the outcome), and "data power" (the measure is assessed by high-quality, regular, and frequently reported data). The outcomes and measures mandated by the Chafee legislation will have primary consideration in the review and selection process.
  
  
• Comprise a system for annual or semi-annual reporting of demographic and performance information. The Department will establish a NYTIS reporting system for States to submit data on youth characteristics, services provided, and outcome measures on an annual or semi-annual basis and will prepare reports for dissemination.
  
  
• Be supported by the Department through training and technical assistance provided to States on data collection and reporting. Once NYTIS is established, the Department will make available training and technical assistance (T/TA) to assist States in meeting the reporting requirements. T/TA will focus on clarifying the definitions of terms used in the outcome measures and other data elements and addressing data collection requirements for each data element. T/TA may also be provided to assist States in incorporating collection of NYTIS data into their Statewide Automated Child Welfare Information Systems (SACWIS).
  
  
• Be designed to reflect the goals and methodology of performance measurement rather than program evaluation. NYTIS will track the extent to which State child welfare agencies achieve specified outcomes for youth and meet basic requirements regarding service provision (such as educational support, training in daily living skills, and housing support for young people ages 18 to 21 formerly in foster care). Information from this process will be used to enhance Federal and State management decision-making regarding program operations and to identify areas where T/TA may be needed. Although this type of tracking system will provide information about the "what" of program performance, it will not answer the "why?" or "how?" questions. For example, information from NYTIS could be used to identify the percentage of youth served that had attained a high school diploma or GED at the time of discharge from foster care. However, it cannot be used to provide an explanation of why some youth graduated from high school and others did not, or how this came about (e.g., what resources and services were necessary).
  
  The Department plans to conduct an evaluation of innovative or promising State programs in order to obtain information regarding causal relationships between services and outcomes. In addition, evaluation will also be used to capture long-term outcome measures for youth who received services from Chafee Independent Living Programs.
  
  
• Be designed to foster continuous improvement of individual State CFCIP programs. The development of a universal performance standard against which each State's performance is compared is not planned. The primary assessment question addressed will be: From one measurement time to the next, did a State's performance on a particular measure improve, show no change, or decline?
  
  
• Incorporate a limited number of key outcomes that are significant to both the Federal government and the States. One of the lessons consistently reported in performance assessment literature is that too many outcomes can confuse and overwhelm users and make the system unmanageable. This issue also was raised by discussion group members during the consultation process which is described later. NYTIS will include key outcomes and measures that are considered vital by the Congress, key stakeholders, and the Department.
  
  
• Be designed to ensure that confidentiality of information on youth is maintained at all levels. NYTIS will incorporate explicit procedures for (1) obtaining informed consent from youth for data collection efforts, if relevant, and (2) encrypting individual youth identifiers so that linkages cannot be made from the data to a specific young person.

B. Identify outcomes and measures for assessing whether youth are making a successful transition from foster care to independent living.

This component of the Department's plan involves two primary activities:

• Reviewing relevant literature to identify key outcomes and measures used to assess youth's progress toward successful adulthood; and
• Consulting with stakeholders to identify appropriate outcomes and measures and critical issues to consider in developing NYTIS.

Review of the Literature on Youth Outcomes

The review of the relevant literature was completed in the fall of 2000. (See Appendix A for a summary of the literature.) Through this activity, the outcomes and measures used by researchers to assess youth discharged from foster care to independent living were identified, as well as those established by youth development programs to assess positive developmental pathways.

The numerous outcomes assessed in the literature were grouped into general domains or areas of youth functioning. A domain may be understood as a broad area of functioning that is related to successful independent living. Each domain encompasses multiple outcomes appropriate for assessing functioning in that general area. The organization of outcomes into domains provides a structure that permits differentiation of outcomes into meaningful and somewhat distinct areas of functioning and allows a consideration of the relationships among these areas. This structure will serve as a conceptual framework for the Department's selection of outcomes for NYTIS.

Six key domains were identified from the research literature on youth outcomes: (1) self-sufficiency; (2) knowledge and skills/readiness for self-sufficiency; (3) social/interpersonal supports; (4) high-risk behaviors; (5) physical and mental health; and (6) self-perceptions/personal characteristics. Outcomes that might be included in the self-sufficiency domain pertain directly to youth functioning with respect to living independently (e.g., Does the young person have a job that pays a living wage and a safe place to live? Does the young person avoid reliance on public assistance?).

Outcomes relevant to the knowledge and skills domain concern whether a youth has the necessary "tools" to attain self-sufficiency. That is, does the youth have the necessary education or training to be able to find a job that pays a living wage? This domain also may be termed the "readiness" domain. Outcomes within the domain can be appropriately assessed as youth are leaving foster care to examine whether they are leaving with the tools to make their transition to independent living successful.

The domain of social/interpersonal supports pertains to how youth function in their relationships with others and in their relationships to the community. Outcomes in this domain might focus on the quality and quantity of the youth's interpersonal relationships as well as on the youth's social networks and community involvement. The inclusion of the social/interpersonal area of functioning reflects the assumption in the field that having social supports and positive interpersonal relationships is a necessary condition for youth to succeed in living independently. These outcomes may be assessed when a young person leaves foster care to examine whether social or interpersonal supports are in place at that time.

The domain of high-risk behaviors could incorporate outcomes that are believed to be or have been found to be negatively correlated with making a successful transition to independent living, such as substance abuse, criminal activities, or becoming a parent prior to marriage. These outcomes may be assessed at the point in which youth leave foster care to determine whether they have been successful in avoiding high-risk behaviors up to that time. However, by themselves, outcomes in this domain are not useful for assessments of youth's success in making the transition to independent living. Outcomes within this domain are more useful as potential explanations for failures to attain self-sufficiency than they are as assessments of youth's success in making a successful transition to independent living.

The physical and mental health domain encompasses outcomes relevant to the physical and mental health status of youth and their self-care skills in terms of maintaining physical and mental health and accessing services to meet physical and mental health needs. Outcomes in this domain are assumed to pertain to a youth's ability to achieve self-sufficiency because holding a job and managing a household become difficult when an individual is not physically or mentally healthy. Again, these outcomes may be assessed as a youth leaves foster care to determine whether physical or mental health problems are likely to be barriers to making a successful transition to independent living.

Outcomes pertaining to a young person's attitudes regarding him- or herself and his or her role in the world can be incorporated in the self-perception domain. This includes perceptions of self as competent and empowered to set and accomplish goals, to view one's actions as making a difference in life, and able to take initiative and feel confident that it will make a difference. This is particularly relevant to an assessment of the CFCIP, provided appropriate and feasible measures can be developed, since anecdotal information from many independent living professionals suggests that one of the major impacts of independent living services is that it has empowered youth to know what they need, ask for what they need, and access what they need.

The Consultation Process

The Department's consultation plan involves a series of activities designed to gather information from the field regarding: (1) appropriate outcomes for assessing CFCIP performance at the State level; (2) measures that are effective indicators of outcome attainment; and (3) critical issues for the Department to consider in developing an outcome-based information system for the CFCIP. The Department's consultation plan on the outcomes and measures includes the following activities:

• Convene discussion groups with various stakeholders to assist in the process of identifying outcomes and measures (completed in May 2001).
• Review State plans and contact State independent living coordinators and data systems staff for information on what individual States are doing with respect to specifying outcomes and measures and collecting and reporting data to assess their independent living program efforts (completed in August 2001).
• Establish and convene a Standing Work Group (SWG) comprised of representatives from State child welfare directors, State child welfare agency independent living coordinators, group home administrators, youth services training and technical assistance providers, youth advocates, researchers, national resource centers, and State child welfare agency data systems experts. The purpose of the SWG is to generate the initial outcomes and measures for NYTIS, and provide further deliberation and revisions after the completion of the discussion groups and the pilot test. The first two meetings were held March 1 and June 14, 2001. The third meeting is planned for November 15, 2001.
• Implement a public review and comment process for the proposed outcomes and measures by early 2002.

The first round of discussion groups was completed in December 2000. Participants included child welfare agency administrators and independent living coordinators at the State and local levels, public agency service providers of independent living services, private youth services agency administrators and providers of transitional living and other youth services, providers of T/TA to youth services programs, child welfare advocates, current and former foster care youth, foster parents, and group home staff and/or administrators. Four groups were convened in conjunction with Department staff in Regions I (Boston), II (Newark), III (Philadelphia), and VI (Dallas). One group was convened at the regional Child Welfare League of America conference in New Orleans, and another group was conducted via telephone with key research and child welfare personnel in mid-western States.

The following discussion topics for the first round of discussion groups were developed to capture outcomes, measures, and key issues:

• The kinds of skills, attributes, knowledge, and experience that youth need in order to make a successful transition from foster care to independent living (outcomes);
• The evidence that individuals working with the youth use to determine whether these skills, attributes, knowledge, and experiences have been attained (measures); and
• The issues that need to be considered in developing NYTIS.

With respect to the first topic (the kinds of skills, attributes, knowledge, and experience necessary to make a successful transition to independent living), discussion group participants generated numerous outcomes. The one outcome that was provided by all groups and by almost all members of the groups was that youth needed to feel connected to a caring adult. Although this outcome was expressed in a variety of ways, the main theme was that if young people had a caring adult in the community that they could go to for advice or talk with about problems, then they had a greater chance for success in living independently than if they did not have this type of connection. As an example, during one of the discussion groups, a foster youth, in discussing her need to feel connected in order to be successful in living independently, said to her foster mother: "I just needed to hear your voice."

Some participants indicated that their States were already collecting data on youth who leave foster care on factors such as parenting status, educational status, housing status/living arrangements, employment status, reliance on public assistance, use of community services, incarceration, income status, and contact with a significant adult. During early 2001, the Department contacted the State independent living coordinators to learn more about what individual States are doing with respect to specifying outcomes and measures and collecting and reporting data to assess their independent living program efforts.

The measures generated in response to the second discussion group topic ranged from very concrete variables to more abstract personal characteristics. For example, the primary measures generated to assess educational attainment were a high school diploma or GED, although some participants noted that measures dealing with improvements in school performance with respect to grades might be appropriate for some adolescents. Concrete measures also were offered for outcomes in the domain of self-sufficiency, such as having a job, earning a living wage, having a residence, having a credit card, paying bills on time, having money for food, and having a driver's license.

Less concrete measures were generated for other types of outcomes. For example, measures such as "having positive relationships with peers and adults" or "having good kids as friends," were offered to assess the outcome of connection to adults or significant others. Other less concrete measures concerned outcomes in the knowledge and skills domain and included measures such as "ability to set and meet goals," "getting to school or work on time," "following rules," and "being oriented toward a career rather than just a job."

The third discussion group topic - issues for the Department to consider in developing the CFCIP performance assessment system - elicited numerous ideas and suggestions from participants. These ideas and suggestions have been incorporated into various sections of this report, including discussions of NYTIS criteria and data elements.

The domains and outcomes generated through the literature review and discussion groups were reviewed by the SWG at their March 1 meeting. During this meeting, SWG members identified the outcomes shown in Exhibit 1 below as critical for inclusion in NYTIS. Using the information in the exhibit as a guide, the Department obtained feedback on the most appropriate and effective measures for assessing those outcomes through eight discussion groups held during March through May. These groups were convened in conjunction with Department staff in Regions IV (Atlanta, GA), V (Chicago, IL), VII (Kansas City, MO), VIII (Denver, CO), IX (San Francisco, CA), and X (Seattle, WA). In addition, groups were held at the Fourth National Child Welfare Data Conference in Alexandria, VA, and at the Transitional Living/Independent Living National Conference in Austin, TX.

Exhibit 1: Outcomes Relevant to Assessing the CFCIP*
Domain	Expected Outcomes
	Increase the number of former or current foster care youth who:
Self-sufficiency	1.Have attained economic self-sufficiency, i.e., are employed and have avoided public assistance
	2.Have a stable and safe place to live
Knowledge and skills/ Readiness for self-sufficiency	3. Have attained academic or vocational educational goals, i.e., have received a high school diploma
Social/interpersonal supports	4. Have connections to caring adults
	5. Exhibit citizenship and community involvement
High-risk behaviors	6.Postpone parenthood, i.e., avoid non-marital birth
	7.Avoid unsafe behavior, i.e., high-risk and illegal
Physical and mental health	8. Have access to physical and mental health care
Self-perceptions/ Personal characteristics	9. Have a sense of well-being and aspirations for the future
	10. Have a strong personal and cultural identity

*Shaded cells indicate outcomes relevant to the outcomes/measures specified in the Chafee Foster Care Independence Act of 1999.

At its meeting in June 2001, the SWG reviewed feedback from the discussion groups, recommended revisions to the draft outcomes, and developed outcome measures. The product of this meeting was a revised list of outcomes and measures, shown in Exhibit 2 below, that will be pilot tested. At the final meeting in November 2001, the SWG will review the results of the pilot test of NYTIS' data elements, definitions and data collection instruments and make final recommendations regarding the outcomes and measures.

Exhibit 2: NYTIS Revised Draft Outcomes, Measures, and Data Elements*
OUTCOME 1: INCREASE THE NUMBER OF YOUTH WHO HAVE RESOURCES TO MEET THEIR LIVING EXPENSES
Measure 1.1: Of all youth discharged from foster care or receiving independent living services during the reporting period, what percent were employed full-time (%) and part-time (%) during the reporting period?	Data elements: 1. Full-time employment 2. Part-time employment
Measure 1.2: Of all youth discharged from foster care or receiving independent living services during the reporting period who were not employed full time, what percent were enrolled in school?	Data element: 3. Current school enrollment
Measure 1.3: Of all youth discharged from foster care or receiving independent living services during the reporting period, what percent held a job, apprenticeship, internship, etc., for at least 3 consecutive months during the reporting period?	Data elements: 4. Job experience 5. Training experience
Measure 1.4: Of all youth discharged from foster care or receiving independent living services during the reporting period, what percent had financial resources other than employment, such as SSI (%), scholarships (%), stipends (%), TANF (%), support from family or spouse (%), Chafee room and board (%), or other (%) during the reporting period?	Data elements: 6. SSI 7. Scholarship 8. Stipends 9. TANF 10. Support from family or spouse 11. Other
Measure 1.5: Of all youth discharged from foster care or receiving independent living services during the reporting period, what percent had a bank or credit union account?	Data element: 12. Bank account
OUTCOME 2: INCREASE THE NUMBER OF YOUTH WHO HAVE A SAFE AND STABLE PLACE TO LIVE
Measure 2.1: Of all youth receiving independent living services during the reporting period, what percent was homeless at some point during the reporting period?	Data element: 13. Homelessness
Measure 2.2: Of all youth who were homeless, what was the duration of homelessness (3 or fewer nights, more than 3 nights but less than 2 weeks, 2 weeks to a month, or more than a month)?	Data element: 14. Duration of homelessness
OUTCOME 3: INCREASE THE NUMBER OF YOUTH WHO ATTAIN EDUCATIONAL (ACADEMIC OR VOCATIONAL) GOALS
Measure 3.1: Of all youth discharged from care or receiving independent living services during the reporting period, what percent had a high school diploma (%), GED (%), or AA/BA (%)?	Data element: 15. Degrees received
Measure 3.2: Of all youth discharged from care or receiving independent living services during the reporting period, what percent had a vocational certificate or license?	Data element: 16. Certifications received
Measure 3.3: Of all youth discharged from care or receiving independent living services during the reporting period, what percent were enrolled in high school (%), GED classes (%), or post-high school vocational training or college (%) during the reporting period?	Data element: 17. School enrollment during reporting period
OUTCOME 4: INCREASE THE NUMBER OF YOUTH WHO HAVE POSITIVE PERSONAL RELATIONSHIPS WITH ADULTS IN THE COMMUNITY
Measure 4.1: Of all youth discharged from care or receiving independent living services during the reporting period, what percent reported that there is at least one adult in the community that they could go to for emotional support (%)?	Data element: 18. Connection to an adult for emotional support
Measure 4.2: Of all youth discharged from care or receiving independent living services during the reporting period, what percent reported that there is at least one adult in the community that they could go to for job/school advice or guidance (%)?	Data element: 19. Connection to an adult for job/school advice or guidance
OUTCOME 5: INCREASE THE NUMBER OF YOUTH WHO AVOID INVOLVEMENT WITH HIGH RISK BEHAVIORS
Measure 5.1: Of all youth discharged from care or receiving independent living services during the reporting period, what percent were was referred for substance abuse assessment or counseling during the reporting period?	Data element: 20. Referral for substance abuse assessment or counseling
Measure 5.2: Of all youth discharged from care or receiving independent living services during the reporting period, what percent had been incarcerated at some time during the reporting period?	Data element: 21. Incarceration
Measure 5.3: Of all youth discharged from care or receiving independent living services during the reporting period, what percent gave birth or fathered a child born during the reporting period?**	Data element: 22. Children born
OUTCOME 6: INCREASE THE NUMBER OF YOUTH WHO ARE ABLE TO ACCESS NEEDED PHYSICAL AND MENTAL HEALTH SERVICES
Measure 6.1: Of all youth discharged from care or receiving independent living services during the reporting period, what percentage of youth had (or would have after discharge) medical insurance?	Data element: 23. Medical insurance
Measure 6.2: Of all youth discharged from care or receiving independent living services during the reporting period, what percentage of youth had (or would have after discharge) health insurance with mental health benefits?	Data element: 24. Mental health benefits
Measure 6.3: Of all youth discharged from care or receiving independent living services during the reporting period, what percentage of youth had (or would have after discharge) dental insurance?	Data element: 25. Dental insurance
Measure 6.4: Of all youth discharged from care or receiving independent living services during the reporting period, what percentage of youth required ongoing medication for maintenance of physical or medical health?	Data element: 26. Medication needs
Measure 6.5: Of all youth discharged from care or receiving independent living services during the reporting period, what percentage of youth who required ongoing medication would be able to access resources to continue receiving their medications?	Data element: 27. Knowledge of accessing resources
OUTCOME 7: INCREASE THE PERCENTAGE OF YOUTH WHO HAVE OR CAN OBTAIN ESSENTIAL DOCUMENTS
Measure 7.1: Of all youth discharged from care during the reporting period, what percent received their birth certificates, social security cards, medical records, and education records at the time of discharge?	Data element: 28. Received essential documents
Measure 7.2: Of all youth discharged from care during the reporting period who did not receive all their essential documents at discharge, what percentage was given information on how to obtain the documents?	Data element: 29. Given information on how to obtain essential documents

*Shaded cells indicate outcomes relevant to the outcomes/measures specified in the Chafee Foster Care Independence Act of 1999.
**This measure will be analyzed in conjunction with the youth characteristic regarding "marital status" to determine out-of-wedlock births.

C. Develop data elements pertaining to recipients of independent living services, the types of services provided, and State performance on the outcome measures; and develop a data collection methodology.

Development of the initial set of data elements, definitions, and instruments for youth characteristics and services provided was completed in June 2001, with further refinement in July 2001 based on feedback from the Chafee Data Work Group which was convened for this purpose. Experience with other national data collection systems, such as the Adoption and Foster Care Analysis System (AFCARS), has shown that the most crucial and often problematic components are the data element definitions, which frequently vary from State to State.

Key activities follow:

• Develop data elements to track the number and characteristics of independent living service recipients. This activity identified the population to be assessed in NYTIS and the characteristics of that population that are important to capture. Tentatively, the population to be identified is all youth age 14 and older in foster care and youth up to age 21 who were previously in foster care and are receiving CFCIP services. Discussion group participants indicated that some of the key characteristics to capture in NYTIS include: how the youth entered foster care (e.g., through the child welfare, juvenile justice, or mental health system), the youth's physical and mental health status, and the youth's current placement type (e.g., group home, institution, family foster home, kinship care, or in-home care under State custody).
  
  
• Develop data elements to track types and quantities of independent living services and activities. The Department has identified CFCIP-related services and activities through reviews of State CFCIP plans and discussions with a sample of State data managers and independent living coordinators. Standard definitions have been constructed of the various services so that data elements pertaining to service delivery efforts will be understood consistently across States and reporting entities within States.
  
  
• Develop data elements to track State performance on the outcome measures and standardized definitions of terms used in the measures. Once the draft outcomes and measures were identified, the Department determined the data elements needed to calculate the measures. This activity involved developing definitions of terms used in the outcome measures so that the meaning of each element is clearly understood. Information from discussion group participants indicated that standardized definitions are particularly critical given the wide variations across States and localities in the ways that services, such as employment readiness or educational support, are defined. Reporting entities must conform to established definitions so that all data can be interpreted within the parameters of the outcome measures. The draft version of the definitions was completed in July 2001.
  
  
• Determine the data collection time-frame. The Department will require States to report the data collected on youth receiving services on an annual or semi-annual, Federal fiscal year basis.
  
  
• Develop a data reporting format. Once all the data elements and definitions were drafted, the Department developed a format (including all necessary instruments) for data reporting. The reporting format includes unique case identifiers, allowing longitudinal analysis of the information. The final reporting format will allow States to provide the required data electronically. The draft reporting format was completed in July 2001.

D. Conduct a pilot test of NYTIS and make necessary revisions.

The purpose of the pilot test is to implement the data collection plan specified under Section C on a small scale to assess and remedy any potential problems prior to full-scale implementation. The Department plans to pilot test NYTIS in several sites representing a range of sizes, locations, and degree of urbanism. The perceptions and experiences of all relevant staff at each of these sites and the challenges they encounter will provide valuable insight on the system's reliability, relevance, and usefulness. The Department currently is planning to pilot in the following States: California, Colorado, Indiana, New Hampshire, North Carolina, North Dakota, Oklahoma, and Texas. Once pilot sites within these states are selected, staff will be trained and quality control procedures established. The pilot test will occur, and the SWG will be convened to provide consultation on pilot test findings in November 2001. Requisite changes to instruments, protocols, and the overall system will be made, and a revised set of instruments and protocols is expected to be completed by early 2002.

E. Establishing a penalty process for State non-compliance with Federal reporting requirements.

The statute at section 477(e) specifically requires the Secretary to assess a penalty against a State if it does not comply with the Department's CFCIP information collection plan. As specified in the legislation, the penalties range from 1 to 5 percent of a State's CFCIP fiscal year allotment and are based on the degree of noncompliance with the Department's CFCIP information collection plan.

In developing a penalty structure, the Department reviewed recent experiences in implementing existing child and family services programs under titles IV-B and IV-E of the Social Security Act. The Department is encouraging continuous improvement in those programs by allowing non-compliant States a period of corrective action and technical assistance prior to taking penalties. States have responded well to this strategy by refocusing their efforts on addressing the problems that affect noncompliance and ultimately child and family outcomes. The Department believes that this strategy of continuous improvement also is essential to promoting strong State-Federal partnerships while ensuring accountability in meeting Federal requirements.

The Department, therefore, is proposing that the penalty structure for Chafee include an opportunity for a State to correct and resubmit data that is non-compliant prior to the withholding of program funds. Our expectation is that a period of corrective action coupled with the provision of technical assistance will result in the submission of State data that are more accurate and reliable. The Department believes that this strategy is consistent with the need for quality data that lends itself to evaluation. Furthermore, the penalty structure will be in line with the Department's overall approach to continuous improvement in child and family services programs.

The Department is proposing the following details of the penalty structure for the CFCIP:

1. Penalties will be applied in the first year of reporting if the State is found not to be in compliance. A reasonable time-frame between the publication of Federal requirements for the collection system and the time for States to collect and report the information required will be provided.
   
   
2. Corrective action will consist of an opportunity for the State to correct and re-submit data that fails to meet data standards established by the Department. The Department believes that an actual plan is not necessary, as it is expected that the notification of non-compliance will identify the minimum action steps a State must take to come into compliance with the standards. Technical assistance will be available to States, to the extent possible, during the period of corrective action. Currently, the Department is considering a six-month period of corrective action, however this time-frame may be re-examined pending the results of pilot testing.
   
   
3. For a State that comes into compliance following corrective action, no penalty will be assessed. For a State that continues to be out of compliance following the period of time allowed for corrective action, CFCIP funds will be withheld from the allotment to the State. The penalty amount will be based on the degree of non-compliance with the information collection plan contained in the State's corrected submission.
   
   
4. The Department will consider three standards of data reporting in developing a penalty structure: the timeliness of the reporting, the accuracy of the data, and the completeness of the data. The penalties will be based on the extent to which the State meets these standards. This means that the penalty will take into consideration the percent of data that is accurate, complete, and whether the State meets the reporting deadline.
   
   
5. The penalty assessed for each data standard (timeliness, accuracy and completeness) will be added cumulatively; however, the maximum percentage assessed will be 5 percent across all standards per reporting period. For a State that receives the minimum allotment of $500,000, the maximum 5 percent penalty would be $25,000; for a State that receives the current highest allotment of $21 million, the maximum penalty would amount to $1,050,000.

The Department has provided a framework for the penalty structure and expects to provide additional details once the service descriptions, client characteristics and outcome measures have been further defined. The consultation with the field and Congress and the results of the pilot testing will inform and further shape the data standards and the penalty structure.

F. Provide for a period for public comment.

The Department will publish the final outcomes and measures in the Federal Register, following the revisions based on the pilot test, and seek public comment by May 2002.

G. Implement NYTIS nationally.

Through the consultation process, pilot test, and period for public comment, the Department will have developed an effective and workable system and achieved extensive stakeholder support. Nationwide implementation is planned for October 2003.

The Department plans to allow States to claim administrative costs for the Chafee Program under Section 474(a)(3) of the Social Security Act. Section 474(a)(3) permits States to claim administrative costs at the 50 percent matching rate for each State that has a plan approved under title IV-E. Administrative costs are those that are necessary for the proper and efficient administration of the Chafee State Plan. States are required to serve children likely to remain in foster care until age 18 and former foster care youth, ages 18 through 20, who have "aged out" of foster care, using Chafee dollars. Therefore, they may claim title IV-E funds for the administrative costs associated with providing Chafee services to these children.

The Department believes that this approach to funding the administrative costs of the Chafee program will assist States with the major expenses of development and operation of the data collection system, as well as other administrative functions, while preserving most CFCIP funds for services to youth.

Conclusion

The Department's plan for developing and implementing NYTIS meets all statutory requirements as set forth in the Chafee Foster Care Independence Act. In addition, the plan reflects the Department's knowledge and experience derived from implementing other outcome-based performance assessment systems in both child welfare and public welfare (i.e., Temporary Assistance for Needy Families, or TANF). The Department believes that the activities presented in this report will ensure that NYTIS is implemented in an effective and efficient manner and reflects the views and goals of a wide variety of stakeholders.

APPENDIX A

OUTCOMES FOR FOSTER CARE YOUTH

The Chafee Foster Care Independence Program (CFCIP) was established in response to research findings indicating that a considerable number of youth discharged from foster care do not make a successful transition to independence and that foster care youth are not adequately prepared for independence while in care. This Appendix summarizes the findings of research that show that youth leaving foster care experience homelessness, unemployment, victimization, and mental health problems at a greater rate than adolescents not in care. It is within this context that the Department developed its plan for implementation of the Chafee legislation.

Research Findings on Outcomes for Foster Care Youth

Studies conducted on youth leaving foster care in the 1980s discovered that rather than attaining self-sufficient adulthood, a significant percentage of these youth experienced homelessness, unemployment, and dependency on various types of public assistance (Barth, 1990; Blome, 1997; Mech, 1994; Westat, 1991). Even youth who were employed rarely had earnings sufficient to support independent living. In fact, a national study, conducted for the Department (Westat, 1991) found that two and one-half to four years after leaving foster care, youth discharged to independent living in 1986 experienced economic conditions that resembled individuals living below the poverty line.

The difficulties these youth experienced in achieving self-sufficiency were attributed in large part to their deficiencies in both educational attainment and independent living skills at the time of discharge from care (Barth, 1990; Blome, 1997; Mech, 1994; Westat, 1991). For example, the Westat study (1991) found that only 48 percent of the 18- and 19-year-olds leaving care in 1986 had completed high school at the time of discharge (the national high school completion rate for the same age group was 64 percent at that time). Another national study, using data from the High School and Beyond Annual Survey of Youth 1980-1986, reported that foster care youth, in comparison to non-foster care youth, were more likely to have dropped out of high school, less likely to have received a graduate equivalency diploma (GED) after they dropped out of high school, and less likely to be enrolled in college preparatory classes while in school (Blome, 1997).

Youth leaving care in the 1990s continued to experience difficulties in living independently, despite the fact that by that time almost all States had developed independent living services for this population in response to P.L. 99-272 (Title IV-E Independent Living Initiative, enacted in 1986). For example, a study conducted in Wisconsin (Courtney and Piliavin, 1998) found that at 12 to 18 months after leaving care, as many as 33 percent of the youth in their sample reported that they were not prepared to obtain a job, manage money, secure housing, or live on their own. Only 50 percent reported being employed. Employed males earned on average $210 per week while females earned $157 on average. In addition, 40 percent of females and 23 percent of males received some kind of public assistance such as AFDC, Food Stamps, Supplemental Security Income, general assistance, and/or housing assistance. Several other State evaluations found that at one year following exit, only slightly more than 50 percent of the youth surveyed were employed, and between 30 to 48 percent of those youth were receiving some form of public assistance after discharge from care (HHS, 1999; McMillen and Tucker, 1999).

Homelessness also continued to be a problem for youth discharged from foster care to independent living in the 1990s. Across studies, youth reported having trouble paying for housing expenses and/or experiencing episodes of having no place to stay (Courtney and Piliavin, 1998; HHS, 1999; McMillen and Tucker, 1999). In the Wisconsin study, at 12 to 18 months after discharge from care, only 37 percent of youth in the sample reported that they had been living in the same place since discharge (Courtney and Piliavin, 1998). Also, 14 percent of males and 10 percent of females reported being homeless (i.e., living on the street or in a shelter) at least once since discharge. A study conducted in Baltimore County found that former foster children were approximately five times more likely than members of a matched comparison group to have experienced at least one period of homelessness (Zuravin, Benedict, and Stallings, 1999).

Again, problems in attaining self-sufficiency were attributed to the fact that many youth continued to leave foster care without the education, employment experience, and life skills necessary for independent living. A review of several studies indicated that about 50 percent of youth left foster care with a high school diploma or GED (Courtney and Piliavin, 1998; HHS, 1999; General Accounting Office [GAO], 1999; Scannapieco, 1996). In a study conducted in Missouri (McMillen and Tucker, 1999), only 33 percent of youth discharged from care at age 17 or older had a high school diploma at the time of discharge, and only another 6 percent had a GED.

In addition to educational deficiencies, many youth surveyed in the evaluations indicated that they were not adequately prepared for independent living at the time of discharge (Courtney and Piliavin, 1998; McMillen et al., 1997). For example, in the Wisconsin study (Courtney and Piliavin, 1998), 25 to 33 percent of youth reported a lack of preparedness in several skill areas; only 18 percent reported receiving assistance in job training, 11 percent in conducting a job interview, 14 percent in finding a job, 12 percent in obtaining housing, 15 percent in obtaining personal health records, and 11 percent in obtaining health insurance. Only 44 percent of youth surveyed had their driver's licenses before they were discharged from care.

Researchers also examined whether difficulties in achieving self-sufficiency may be explained by youth engaging in certain types of behaviors, specifically substance use, early parenting, or involvement in criminal acts. With respect to substance use, studies found that youth discharged from foster care to independent living were not more likely to use alcohol or illegal substances than other youth (McMillen and Tucker, 1999; Westat, 1991, Zuravin et al., 1999). Another study comparing adults who had been in foster care with adults in a matched comparison group found no differences between the groups with respect to drug or alcohol use (Zuravin, Benedict, and Stallings, 1999).

Several studies reported that a substantial percentage of foster care youth discharged to independent living had children within a short time. The Wisconsin study (Courtney and Piliavin, 1998) found that 12 to 18 months after discharge, eight percent of the males and 19 percent of the females had children. The Westat study (1991) reported that at follow-up (two and one-half to four years after discharge), 24 percent of the men reported having fathered a child and 60 percent of young women had given birth to at least one child. The Westat study also noted that these women were half as likely to have completed further schooling than were former foster care women who had not given birth. The Missouri study reported that one in five females discharged from foster care at age 17 or older were pregnant or parenting at the time that they left care (McMillen and Tucker, 1999).

Research also has found high rates of involvement with the criminal justice system among youth discharged to independent living. The Wisconsin study (Courtney and Piliavin, 1998) reported that at the 12- to-18-month follow-up, 27 percent of males and 10 percent of females in the sample had been incarcerated at least once. The Westat study (1991) found that by the time of the follow-up interview, 25 percent of the youth in their sample had been in trouble with the law and four percent had been incarcerated.

A few studies reported that specific aspects of a youth's personal experience were associated with difficulty attaining self-sufficiency. The findings pertaining to personal experiences included the following:

• Youth who entered foster care at older ages were more likely than others to be receiving public assistance at the follow-up point (Westat, 1991).
• Youth who experienced multiple placements while in care were more likely than youth who did not experience multiple placements to be receiving public assistance or to be institutionalized (Blome, 1997; Westat, 1991).
• Youth who entered foster care because of severe abuse performed less well in school than other foster care youth. This pattern of low achievement tended to follow them into their adult lives (Fanshel, Finch, and Grundy, 1990).

An alarming finding, and one that was consistent across several studies, concerned the high level of mental health problems experienced by older youth discharged from foster care. The Wisconsin study (Courtney and Piliavin, 1998) reported that the youth in their sample scored significantly lower on a mental health inventory than youth of the same age group not in foster care, yet 12 to 18 months after leaving care, only 21 percent of these youth reported receiving some kind of mental health services in the previous year. The Missouri study (McMillen and Tucker, 1999) reported that 44 percent of their sample had been in a psychiatric hospital by the time of discharge. This finding was particularly surprising because the study population was comprised of a randomly selected sample of case records of youth who were discharged from foster care during a one-year period and who were age 17 or older at the time of discharge. In other studies, the incidence of mental health problems found among youth in foster care ranged from 22 to 75 percent (Iglehart, 1994; Mech, 1994).

One study reported that many youth discharged to independent living experienced serious victimization after leaving care (Courtney and Piliavin, 1998). This study found that 12 to 18 months after leaving care, 25 percent of males and 15 percent of females reported experiencing some kind of serious physical victimization and 13 percent of the females had been sexually assaulted and/or raped since discharge. The researchers noted that a preliminary analysis of the violence committed against the youth since they left care suggests that at least some of it occurred at the hands of kin. Thus, the issue of victimization may be relevant for the population of youth discharged from foster care to independent living since 31 percent of the youth in this study reported living in the home of a relative at some time after discharge from care.

Some research findings, however, support the contention that providing appropriate services to prepare youth for independent living will improve their outcomes. For example, the Missouri study (McMillen and Tucker, 1999) reported that among youth who had completed high school, participation in an independent living program while in care was associated with higher odds of being employed at exit from care. In a Baltimore County study, youth who received independent living services were more likely to complete high school, have an employment history at the time of discharge, and be employed when they left foster care than youth who did not receive independent living services (Scannapieco, 1996). In a study conducted in North Carolina comparing 44 independent living program (ILP) participants with 32 non-participants, 30 percent of the ILP participants were paying all their housing expenses within one to three years after leaving foster care as compared to 19 percent of non-participants (Lindsey and Ahmed, 1999). A GAO report (May 1999) noted that a study conducted in Harris County, Texas, found that graduates of the county's independent living program achieved full-time employment earlier and were more likely to complete high school or a GED at a younger age than youth who did not receive independent living services.

Finally, a study (Mallon, 1998) conducted of an independent living program in New York City operated by a private nonprofit agency (Green Chimney's Children's Services) found several positive outcomes for youth discharged to independent living from that program, although no comparison group was included in the study. Specifically, the study found that at discharge, 75 percent of the youths had completed high school or obtained a GED and more than 25 percent of these had some college. In addition, 72 percent were employed full-time at discharge, and 78 percent at follow-up.

REFERENCES

Barth, R.P. 1990. "On Their Own: The Experiences of Youth After Foster Care." Child and Adolescent Social Work, 7, 419-440.

Blome, W.W. 1997. "What Happens to Foster Kids: Educational Experiences of a Random Sample of Foster Care Youth and a Matched Group of Non-Foster Care Youth." Child and Adolescent Social Work, 14, 41-53.

Courtney, M.C. and Piliavin, I. 1998. Foster Youth Transitions to Adulthood: Outcomes 12 to 18 Months After Leaving Out-of-Home Care. Madison, WI: Institute for Research on Poverty, University of Wisconsin-Madison.

Fanshel, D., Finch, S.J., and Grundy, J.F. 1990. Foster Children in a Life Course Perspective. New York: Columbia University Press.

Friedman, M. 1997. A Guide to Developing and Using Performance Measures in Results-Based Budgeting. Washington, DC: The Finance Project.

Government Accounting Office (GAO), May 1999. Foster Care: Challenges in Helping Youth Live Independently. GAO Document No. GAO/T-HEHS-99-121. Washington, DC: GAO.

Iglehart, A.P. 1994. "Adolescents in Foster Care: Predicting Readiness for Independent Living." Children and Youth Services Review, 16, 159-169.

Lindsey, E.W. and Ahmed, F.U. (1999). "The North Carolina Independent Living Program: A Comparison of Outcomes for Participants and Nonparticipants." Children and Youth Services Review, 21, 389-412.

Mallon, G. P., 1998. "After Care, Then Where? Outcomes of An Independent Living Program." Child Welfare, 77, 61-78.

McMillen, J.C., and Tucker, J. 1999. "The Status of Older Adolescents at Exit from Out-of-Home Care." Child Welfare, 78, 339-360.

McMillen, J.C., Rideout, G.B., Fisher, R. H., and Tucker, J. 1997. "Independent-Living Services: The Views of Former Foster Youth." Families in Society: The Journal of Contemporary Human Services, September-October, 471-479.

Mech, E.V. 1994. "Foster Youths in Transition: Research Perspectives on Preparation for Independent Living." Child Welfare, 73, 603-623.

Scannapieco, M. 1996. "An Independent Living Program: Characteristics, Outcomes, and Indicators for the Future of Foster Care Services." Community Alternatives, 2, 19-25.

U.S. Department of Health and Human Services. November 1999. Title IV-E Independent Living Programs: A Decade in Review. Washington, DC: U.S. Government Printing Office.

Westat, 1991. A National Evaluation of Title IV-E Foster Care Independent Living Programs for Youth: Phase 2 Final Report, Volume 2. Rockville, MD: Westat.

Zuravin, S. J.; Benedict, M.; and Stallings, R. 1999. "The Adult Functioning of Former Kinship and Nonrelative Foster Care Children." In Hegar, R.L. and Scannapieco, M., eds. Kinship Foster Care: Policy, Practice, and Research. New York: Oxford University Press, 208-222.