Home > Statistics & Research > Child Welfare Outcomes 1999: Appendix E
Appendix E: Information Memorandum
ACF
Administration on Children, Youth and Families
U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES
Administration on Children, Youth and
Families
1. Log No: ACYF-CB-IM-01-07
2. Issuance Date: August 16, 2001
3. Originating Office: Children’s Bureau
4. Key Words: National Standards, Program Improvement
Plans, Child and Family Services (CFS)
Reviews
INFORMATION MEMORANDUM
TO: State and Territorial Agencies Administering or
Supervising the
Administration of Titles IV-B and IV-E of the Social Security
Act,
ACF Regional Administrators and Hub Directors
SUBJECT:National Standards for the Child
and Family Service
Reviews and Guidance on Program Improvement Plans
LEGAL AND RELATED REFERENCES: Titles IV-B and IV-E
of the Social Security Act, the Social Security Act Amendments of
1994 (Public Law 103-432), 45 CFR 1355.34(a) and (b), 45 CFR
1355.35(a) and (e), 45 CFR 1355.36(d), ACYF-CB-IM-00-11, and
ACYF-CB-IM-01-01
PURPOSE: The purposes of this information memorandum
(IM) are: (1) to provide
updated information on the national standards that are used to
determine, in part, whether or not States are operating in
substantial conformity with State plan requirements, and; (2) to
provide guidance for use by States and Regional Offices in
negotiating the amount of improvement necessary toward meeting the
national standards through an approved program improvement plan
(PIP).
INFORMATION:
Background
In accordance with 45 CFR 1355.34(a) and (b), a State must, in
part, meet national standards for certain statewide data indicators
and achieve certain outcomes for children and families to be
determined in substantial conformity in a child and family services
review. In ACYF-CB-IM-00-11 we transmitted national standards
for the six statewide data indicators used, in part, to determine
if States are operating in substantial conformity with titles IV-B
and IV-E of the Social Security Act (the Act). We have updated the
national standards because of corrected State data that form the
basis for the standards and a number of improved procedural
measures that we describe below.
In addition, if a State is determined not to be in substantial conformity because the State’s data did not meet the national standards, the State is required to develop a PIP (45 CFR 1355.35(a)) that sets forth the amount of progress the State will make toward meeting the national standards for the statewide data indicator(s). In addition to updating the national standards, we are providing guidance for use by States and Regional Offices in determining the amount of improvement that States must make toward meeting the standards through the implementation of a PIP. We note that a PIP is required of States determined not to be in substantial conformity on any of the outcomes and systemic factors examined as a part of the child and family services reviews. A State may be determined not to be in substantial conformity for reasons other than failure to meet the national standards, such as the failure of a sufficient number of cases reviewed onsite to be rated as substantially achieved. This Information Memorandum addresses the extent of improvement that will be required of States through a PIP due to determinations that States are not operating in substantial conformity based in whole or in part on the failure of the State to meet the national standards, and does not address States’ failure to achieve substantial conformity due to other reasons.
Updated National Standards
The updated national standards reflect changes in four
areas:
• The use of corrected State data submitted to AFCARS and
NCANDS for the time periods used to calculate the standards;
• The application of new rules that disregard skewed data
points in calculating the national standards;
• The inclusion of a sampling error associated with each
national standard; and
• The rounding of the national standards to one decimal
point.
We discuss each of these changes below.
Corrected State data. In preparing for the CFS reviews, a number of States examined their data submissions to AFCARS and NCANDS in previous years and re-submitted their data for purposes of the CFS reviews. We encouraged States to analyze their data critically so that the reviews would reflect the most accurate picture available of outcomes for children and families. Therefore, we have incorporated the corrected data into the national standards.
New rules for disregarding extreme data points. When recalculating the national standards, we applied new rules that disregarded extreme data points in individual States’ data that were used to calculate the national standards. We excluded State data indicators that were 0% and 100% from the data points used to determine the standards, believing that such extreme values were unlikely to be accurate and would skew the standards in one direction or another.
Inclusion of sampling error in the standards. In calculating the new national standards, we included the sampling error associated with each data indicator as part of the actual standard. Even with the use of more accurate data to calculate the national standards, we realize that there is a certain range within which a State’s data can be considered to meet the standard statistically. We chose to incorporate the sampling error into the standards because we do not believe it is appropriate to subject a State to a potential penalty if the State is within a statistically acceptable range of the national standard.
Rounding of the national standards. When we updated the national standards to incorporate sampling errors, we believed that the full benefits to the States of including the sampling error in the standard would be best applied by rounding the standards using decimal points. We chose to round the standards to only one decimal point because we do not believe it is appropriate to attempt to measure State performance to the degree of precision implied by two decimal places. The exception to this rounding procedure is the incidence of child abuse and/or neglect in foster care and has been rounded to two decimal points due to the small number of such incidents reported by States.
Determining a State’s Achievement of
the National Standard
In determining whether or not a State meets the national standard
for a particular statewide data indicator, we will compare the
State’s data used in its CFS review to the updated national
standards below. We will round the State’s data for
individual indicators using the same rationale described above for
rounding the national standards, i.e., rounding to one decimal
point for each of the six indicators except the incidence of child
abuse and/or neglect in foster care, which will be rounded to two
decimal points. As shown in the chart below, each of the updated
national standards works to the advantage of the States in meeting
the standards. For States that have already undergone CFS reviews
in FY 2001, we will apply the updated national standards
retroactively to decisions regarding substantial
conformity.
Our methodology for calculating the updated national standards and for determining whether a State's data meet the national standards are indicated in the chart below:
| Data Indicator | 75th Percentile | Sampling Error | Old National Standard (For comparison only) | Method of Determining if State Meets National Standard | ||
| Recurrence of maltreatment | 5.15% | 0.90% | 6.1% | 5% | A State will be determined to meet the national standard if its data on this indicator are equal to, or less than, the updated national standard. | |
| Incidence of child abuse and/or neglect in foster care | 0.4247% | 0.1439% | 0.57% | 0.4% | A State will be determined to meet the national standard if its data on this indicator are equal to, or less than, the updated national standard. | |
| Foster care re-entries | 7.23% | 1.35% | 8.6% | 6% | A State will be determined to meet the national standard if its data on this indicator are equal to, or less than, the updated national standard. | |
| Stability of foster care placements | 88.58% | 1.90% | 86.7% | 89% | A State will be determined to meet the national standard if its data on this indicator are equal to, or less than, the updated national standard. | |
| Length of time to achieve reunification | 78.62% | 2.42% | 76.2% | 78% | A State will be determined to meet the national standard if its data on this indicator are equal to, or less than, the updated national standard. | |
| Length of time to achieve adoption | 34.89% | 2.90% | 32.0% | 36% | A State will be determined to meet the national standard if its data on this indicator are equal to, or less than, the updated national standard. | |
*Rounded as noted above
Determining a State’s Improvement
with regard to the National Standards
In situations where a State does not meet a national standard, the
regulations require that the PIP, which is jointly developed by the
Regional Office and the State, set forth the amount of progress the
statewide data will make toward meeting the national standards. (45
CFR 1355.35(a)(1)(i),(iv)). The PIP must also specify a period in
which ACF can measure a State's progress in achieving the
negotiated improvement (45 CFR 1355.5(e)(1)). We will consider the
sampling error for each indicator as a minimum percentage of
improvement for a State to make over the course of a PIP. For
example, if a State’s recurrence of child maltreatment is
10% at the time of the review, at a minimum we would expect the
State to improve by at least .90% on this indicator as part of an
approved PIP, thus reducing the incidence of recurrent maltreatment
to 9.1%, in order to consider that improvement has been made. We
believe that using the sampling error as a framework is appropriate
because where a State's improvement exceeds the sampling error, we
can conclude that a statistically significant improvement has been
made.2
In addition to the sampling error, the Regional Office may also
consider other programmatic factors in negotiating the amount of
improvement States are required to make in a PIP. Such programmatic
factors may include the extent to which a State’s data fall
below or exceed the national standard, the factors affecting the
State’s lack of substantial conformity, the difficulty and
time involved in program improvement efforts in achieving overall
improvement, whether the area needing improvement is an egregious
area affecting child safety, and the amount of performance
improvement the State may have already achieved subsequent to
generating the original data profiles used in the
reviews.
Evaluating a State’s Improvement in
Meeting the National Standards
To determine the amount of progress a State has made toward
meeting the national standard, we will compute the difference
between the State’s data as indicated in the original data
profile and the subsequent period specified in the PIP. For
example, if a State fails to meet a national standard based on data
from Federal fiscal year 1999, and the PIP includes the negotiated
provision that we use data from Federal fiscal year 2003 to gauge
the State’s progress toward meeting the national standard,
we will compute the difference between the data from those two time
periods.
If the difference indicates a level of improvement that meets or
exceeds the approved amount of improvement specified in the PIP, we
will determine that the State has successfully completed that
portion of the PIP and the associated penalties will be rescinded
(section 1355.36(d)). If the State does not meet or exceed the
approved amount of improvement specified in the PIP, the associated
penalties will be withheld as specified in 45 CFR 1355.36. States
that do not achieve the updated national standard in subsequent
reviews will again be determined out of substantial conformity and
be required to establish new benchmarks of improvement to be made
toward attainment of the national standard, using the same
procedures described in this Information Memorandum3
and subject to the penalty provisions in 45 CFR
1355.36.
INQUIRIES TO: ACF Regional Offices
James A. Harrell
Acting Commissioner
Administration on Children, Youth and
Families
1) ACYF-CB-IM-00-11 was later amended by
ACYF-CB-IM-01-01 due to a typographical error in describing our
methodology for calculating the national standard for the statewide
data indicator, “foster care re-entries. ” We are
rescinding both ACYF-CB-IM-00-11 and ACYF-CB-IM-01-01 due to the
revisions in the national standards and are placing a background
paper on the Children’s Bureau website that includes
relevant information from the two Information Memoranda noted
above. Back
2) In situations where the amount
of a State’s improvement on an indicator exceeds the
sampling error, we can determine that improvement has occurred in a
statistical sense, since exceeding the sampling error will indicate
a statistically significant difference (improvement) between the
State’s performance on the period under review and the
subsequent evaluation period specified in the PIP. Back
3) More information on developing
program improvement plans can be found in chapter 7 of the CFS
review procedures manual. Back