ACYF-CB-IM-03-05

ACF Administration for Children and Families	U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Administration for Children, Youth and Families
	1. Log No.: ACYF-CB-IM-03-05	2. Issuance Date: 11/21/2003
	3. Originating Office: Children's Bureau
	4. Key Words: Child and Family Services Reviews, Item Ratings

INFORMATION MEMORANDUM

To:	State and Territorial Agencies Administering or Supervising the Administration of Titles IV-B and IV-E of the Social Security Act, ACF Regional Administrators
Subject:	Ratings for the Child and Family Service Reviews
Legal and Related References	Titles IV-B and IV-E of the Social Security Act, the Social Security Act Amendments of 1994 (Public Law 103-432), 45 CFR 1355.34 (a) and (b).
Purpose:	The purpose of this Information Memorandum is to provide clarity on rating the items under each of the seven outcomes evaluated during the Child and Family Service Reviews.

Information:

In accordance with 45 CFR 1355.34(a) and (b), a State must, in part, meet national standards for certain Statewide data indicators and achieve certain outcomes for children and families in order to be determined in substantial conformity in a child and family service review (CFSR). Specifically, States must substantially achieve the outcome in 90 percent (95 percent in reviews subsequent to the first round of reviews) of the cases reviewed on-site to be considered in substantial conformity.

The outcomes included in the reviews fall into three domains: safety; permanency; and child and family well-being. Within each domain, the CFSR assesses specific outcomes, through the on-site review and statewide assessment, using a set of performance indicators to evaluate each outcome. In rating individual cases on both the outcomes and the performance indicators, the on-site review instrument provides standard criteria for determining if an outcome has been substantially achieved or if a performance indicator is a Strength or an Area Needing Improvement for the case. Beyond the individual case ratings, the regulations at 45 CFR 1355.34(b)(3)(ii) set forth the percentage of cases in which an outcome must be rated as substantially achieved in order for the outcome to be in substantial conformity for the State as a whole. The regulations also authorize the Department of Health and Human Services to establish national standards as the threshold for certain performance indicators, i.e., those six data indicators with associated national standards, to be determined as Strengths or Areas Needing Improvement.

In the final reports of the CFSRs to the States, in addition to identifying the outcomes which meet the criteria for substantial conformity, we have found it helpful to identify those individual performance indicators reviewed in the cases during the on-site review that, in the aggregate, are primarily Strengths or Areas Needing Improvement. This aggregate characterization of individual indicators pertaining to the outcomes, beyond those data indicators with associated national standards, does not affect the determination of substantial conformity for the outcomes. We believe, however, that identifying them as Strengths or Areas Needing Improvement helps to guide States in understanding which performance indicators have contributed to the States' performance on the outcomes and, consequently, the areas to be addressed in the program improvement plan (PIP) in order to improve future performance on the outcomes.

Since the aggregate determination that a performance indicator is either a Strength or an Area Needing Improvement is not directly related to substantial conformity for the outcome with which it is associated, the regulations do not offer guidance regarding the threshold for determining that such indicators are Strengths or Areas Needing Improvement in the aggregate for the State. Thus, this Information Memorandum lays out the criteria that we have adopted in order to make such a determination and our basis for adopting them.

An individual performance indicator may be rated as a Strength or an Area Needing Improvement in the aggregate data in a State's final report, depending on whether the performance indicator was rated as a Strength in 85 percent of the applicable cases reviewed on-site. If the performance indicator was rated as a Strength in 85 percent or more of the applicable cases, we will identify the performance indicator as a Strength in the final report. If the performance indicator was rated as a Strength in less than 85 percent of the applicable cases in the on-site review, we will identify the performance indicator as an Area Needing Improvement in the final report.

Our rationale for adopting the 85 percent threshold is as follows. The combination of ratings for performance indicators in individual cases determines whether or not the outcome in an individual case is substantially achieved or not. That is, for some outcomes, an individual performance indicator may be rated as an Area Needing Improvement and the outcome can still be substantially achieved. Therefore, adopting the same 90 percent threshold for determining if individual indicators are Strengths or Areas Needing Improvement would be too stringent, and possibly misleading to States, in developing program improvement plans designed to improve performance on the outcomes. For example, we feasibly could run the risk of identifying indicators as Areas Needing Improvement that, in fact, were not major contributors to a State's failure to achieve substantial conformity on an outcome. Moreover, if we adopted a threshold that was too far below the 90 percent mark, we might fail to identify performance indicators that were, in fact, contributors to the State's performance on the outcome. Consequently, we deemed the 85 percent threshold to be a high enough standard that was consistent with the concept of substantial conformity, allowed for the reduction of sample size (since fewer cases are evaluated for each item than for an outcome), and provided the State with clear guidance on what it needs to address in the PIP in order to realize improvements in the outcome over time.

Using the 85 percent threshold will assure consistency in final reports across the Regional Offices and help States identify their specific strengths and weaknesses with greater precision than would be the case if we did not have a standard approach to this issue.

Inquiries To: Regional Administrators, Regions I-X.

Joan E. Ohl
Commissioner