Appendix H: Developing the Child Welfare Outcomes and Measures
During the fall of 1998, the Department of Health and Human Services engaged in a consultation process with representatives from State, Tribal, county and municipal child welfare agencies; private nonprofit child and family services agencies; State legislatures; Governors’ offices; juvenile and family courts; local child advocacy organizations; and a national public employees union. The purpose of the consultation process was to obtain assistance from the field in developing a national set of child welfare outcomes and measures.
Discussions held during consultation meetings resulted in establishing four principles that served to guide the development of the child welfare outcomes and measures. These principles are the following:
• The outcome measures should reflect performance that is to a large extent within the control of State child welfare systems. It is difficult to identify outcomes that can be linked solely to child welfare system operations because of the dependency of child welfare on the functioning of many other systems. The outcomes related to safety, permanency, and well-being are significantly influenced by factors beyond the control of a child welfare agency. For example, the capacity and functioning of juvenile or family court systems, juvenile justice systems, law enforcement systems, mental health services, and substance abuse treatment systems in a State or community significantly affect a child welfare agency ’s ability to impact outcomes in the areas of safety, permanency, and well-being for the children it serves. In addition, economic conditions or the extent of substance abuse problems in a community can increase the number of children who are unsafe in their own homes or reduce the number of children who can be safely returned to their families once they have been removed from their homes.
• The outcome measures should be assessed in ways that
limit the potential for misinterpretation. Determining the
meaning of changes in State performance is complex. Such changes
can be the result of other influencing factors and/or imbalances in
the child welfare system. Positive or negative changes must be
viewed in relation to other conditions in the State. For example,
an increase in the number of child abuse and/or neglect reports
could be the result of an economic downturn or an increase in the
extent of substance abuse problems in a community. Equally
important is for States to focus attention on strengthening all
parts of the system to ensure a balanced approach to systems
performance. This will result in (1) better decisions for the
children and families who come into contact with the child welfare
agency and (2) improved performance on the outcome measures
overall. Conversely, if a State focuses on improving one part of
its system such as reducing the time children spend in foster care,
the State may achieve the desired goal but negatively impact some
other part of the system, e.g., increase the rate of re-entry of
children into foster care.
• The outcome measures should be used to assess the
continuous improvement of each State over time, rather than compare
the performance of States with one another. Comparison across
States on their performance on the outcome measures is difficult
due to variations in State population demographics, programs, and
policies and must be undertaken with great care. State performance
is better assessed by tracking a State’s own continuous
improvement over time. Some useful factors in considering State
variations may include the proportion of children in poverty in
each State; the State’s definition of abuse and/or neglect;
the State’s standards that prompt the removal of children
from their homes and the decision to return them home; the
resources available in the community to meet the needs of children
and families; the resources allocated by the State to the child
welfare system to develop and test innovative strategies to attain
desired outcomes; and the internal capacity of the child welfare
agency in terms of staffing, caseload size, and other
factors.
• The outcome measures should be based on data that are
available through existing data collection systems in order to
limit the reporting burden on the States. There have been great
advances at the State and Federal Government levels in collecting,
analyzing, and reporting child welfare data, using the National
Child Abuse and Neglect Data System (NCANDS) and the Adoption and
Foster Care Analysis and Reporting System (AFCARS). Despite this
progress, a significant number of States are in the process of
improving their data collection capabilities through the design and
implementation of new and improved Statewide Automated Child
Welfare Information Systems (SACWIS). As a result, many State
SACWIS systems are still developing and being refined, which means
that the national collection and reporting of data to support
measuring some or all of the outcomes are not yet available for all
States. The Department is confident that State data will continue
to improve each year and in subsequent annual reports and is
providing assistance to States on developing and implementing their
SACWIS systems, in addition to assistance on improving the quality
of their data submissions on the outcome measures.
