Findings

Two major areas are assessed during an AFCARS assessment review. These are the AFCARS general requirements and the data elements. The general requirements include the population that is to be reported to AFCARS and the technical requirements for constructing a data file. The data elements are assessed to determine whether the State is meeting the AFCARS definitions for the information required, if the correct data is being entered and extracted, and the quality of the data submitted.

This section provides the major findings resulting from the review of the State's AFCARS data collection. Tab A provides detailed information on the findings for each of the foster care and adoption elements, the general AFCARS requirements, and the case file review. The AFCARS reporting period under review was April 1, 2000 through September 30, 2000 (2000B). Frequencies for the report period October 1, 2000 through March 31, 2001 (2001A) are included in Tab D, but the data were not used during the post-site visit phase analysis.

Strengths

Strengths identified by the Federal review team include that:

The State project team is aware and concerned about problems with the system and data quality;
The State converted 1993-1996 historical data into LINK;
The review process was enhanced by the participation of State strategic planning and regional staff, in addition to program and technical staff; and
The State AFCARS reporting population includes the youth in juvenile justice that have title IV-E reimbursable placements and children that are in dual custody.

The State agency staff were aware of many areas that need to be modified. The staff specifically requested a review of the State's AFCARS data collection due to concerns they had regarding the accuracy of the data and they way it is collected. The staff shared that their intention is to write a new program code to map and extract the AFCARS data.

Based on the analysis of 53 foster care paper case files, a significant amount of the data in the AFCARS report matched the information the reviewers found in the paper file. Most of the errors were found in the areas of child removal and placement information.

General Requirement Errors

Reporting population: Foster care population (Factor: 2)

The State is not submitting the complete foster care population as defined in the AFCARS regulation and policy issuances. For AFCARS purposes, a removal episode is defined as the time from the child's removal from home (for more then 24 hours) until the child is no longer under the care and placement, or supervision of the State agency. This includes the time that the child is returned home, and the agency still has custody of the child, with or without post-reunification services.

In the selection logic for the AFCARS foster care population, the State incorrectly reports as discharged children that have a current placement setting that meets the AFCARS definition of "trial home visit". The State is to continue reporting these children to AFCARS for the time specified of the return home. If no duration of time has been specified the State is to continue reporting the child for six months after the return, then the State can enter the case as "discharged" for AFCARS purposes. (See Child Welfare Policy Manual, Section 1, AFCARS)

Technical Requirements (Factor: 1)

Encryption/Sequential Numbering of Case Records

The State does not properly report the case record number. The State has the option of using sequential numbers for the case record number or encrypting the case record number. While the State has the option to use either sequential numbering or an encryption routine, the Children's Bureau would prefer the State use an encryption routine. This allows a more reliable method to create an annual file from the two six month submissions.

Conversion

The State did have a conversion plan and made a conscientious effort to convert open and closed cases from the legacy system (CMS), as far back as 1993. LINK was implemented in 1996. However, data on removal episodes are not in LINK if the removal occurred prior to 1993. Additionally, if a case re-opens, and for those cases open at the time of conversion, "historical" data cannot be entered into the system by the worker. Historical data includes information on the first-ever removal from home, date of discharge from the last removal episode, and the number of removals from home. Also, depending on the accuracy of the date of the current removal, this information, the circumstances associated with removal, the number of placement settings, and the date of the current placement setting may all be affected by the conversion process, as well as whether the mother was married at the time of the child's birth.

The State must develop a method for the system to collect information on the "date of first removal from home," the "date of discharge from the last removal episode," and the "number of removals," such as a "historical" data screen. Also, if a case was open at the time of conversion and all placement information was not entered, then the State must develop a method to allow entry of all placements in the current removal episode.

The majority of the errors found in the case file review were related to "historical" data. The sample of cases was selected based on dates of latest removal that occurred prior to the date LINK became operational Statewide. Several cases do not accurately reflect the number of removals, and the date of first removal from home. Some of these cases had dates of latest entry into foster care after October 1, 1995. All of the data, including historical information, are required for AFCARS reporting purposes.

These changes will have an impact on the data that are used by the State for program and evaluation and for the Federal National Standards. The information that the State has provided may be under-reporting the number of removals from home that a child has experienced.

Statewide Automated Child Welfare Information System (SACWIS)

The State's information system is primarily payment-focused and, as a consequence, the overall accuracy of the data is jeopardized. SACWIS should be a case management information system that supports the business practices of child welfare. Several of the recommendations and some of the required changes to the system could result in the State needing to submit an update to its Advanced Planning Document, especially if the changes result in a significant commitment of resources, change in scope, or a change in schedule. The State should coordinate the findings and changes required from the AFCARS review with its response to the SACWIS Assessment Review Findings.

Data Element Errors

Eighty-three percent (55) of the foster care and eighty-nine percent (33) of the adoption data elements require system modifications. Once the program logic changes are made, the State will need to monitor caseworkers' data entry in order to ensure that the quality of the data continues to improve.

The State's program code that extracts and maps the AFCARS data defaults missing data to valid AFCARS code for nearly every element. Not only does this approach not meet the requirements for reporting the foster care and adoption data, and provides a misleading and inaccurate account of the children in foster care and those that have been adopted, but it also has allowed the State to avoid penalties that may otherwise apply. The State must map all missing data to blanks and encourage the workers to keep the electronic case file up-to-date.

Also, please note that as you implement corrective action the changes you make to your system, with regard to data entry, will inevitably result in improving the quality of data. This may correspondingly result in the State's semi-annual data submission not meeting the missing data standard and, consequently, the assessment of a penalty. In order to ensure that the data are complete, the agency will need to require workers to enter the data, and assess the validity of the data prior to submitting it to ACF. This can be done with the management reports created by the agency, using the Data Quality Utility, and the Frequency Utility.

Most Recent Periodic Review Date (Factor: 1, Number of elements affected: 1)

The State does not correctly report this information to AFCARS. There are several issues related to the accuracy of this information. First, the information system does not have full capacity to collect this information. The system program logic does attempt to derive this date from the court disposition date, however, it is not always the correct date. Second, if there is no date for a review, then the program code subtracts one month from the last day of the report period and enters this date as the date of a periodic review.

It is apparent from the case file review that the State does do periodic reviews and this information is available in the reports that the workers must complete for the reviews. Lastly, without a case plan module in the system, this date is not properly recorded electronically.

The system must not arbitrarily create dates for the AFCARS data file, and the State must cease submitting these dates to ACF. If the information is missing, the State must leave this information blank in the AFCARS file. The State staff shared after the site visit that the form "DCF-553" will be used to collect this information, but that its implementation is on hold. The State must implement the case plan module as soon as possible, have workers enter the information, and clean up the existing data files.

Case Plan Goals (Factor: 1, Number of elements affected: 1)

The State does not collect this information. The response rate in the AFCARS data frequencies is 100% "not yet determined". It is apparent from the case file review that workers are developing case plan goals, and are recording them in non-system generated documents.

The State must implement its case plan module as soon as possible. The State will also need to implement methods and train workers to enter this information and to keep it up-to-date. This element is used in the Child and Family Services Statewide Data Profile, and presently the State is failing to meet the standard.

Information on Children Diagnosed with Disabilities (Factor: 2, Number of elements affected: 6)

In the foster care data set, element #10 asks if the child has been clinically diagnosed with a disability. If the response is "yes", then some or all of the applicable disability categories (medical condition, mental, physical, emotional disability, or other medically diagnosed condition) are to be selected. The primary problem is that this information is being collected on the same screen where "conditions associated with a child's removal" is also collected. Consequently, this information is not getting entered, nor can it be updated at any time. The number of responses to the question in foster care element #10 is only 3% of the cases submitted. Also, workers are not able to select the options of "yes", "no", or "not yet determined." The AFCARS definitions for these selections are:

Yes - Indicates that a qualified professional has clinically diagnosed the child as having at least one of the disabilities listed.

No - Indicates that a qualified professional has conducted a clinical assessment of the child and has determined that the child has no disabilities.

Not Yet Determined - Indicates that a clinical assessment of the child by a qualified professional has not been conducted.

The information on disabilities needs to be removed from the screen containing the "conditions associated with removal" and a placed on another more appropriate screen (i.e., medical information screens) or on a new screen. The State staff suggested the medical profile as a source of this information. This form needs to be a part of the LINK database.

Conditions Associated with Removal (Factor: 2, Number of elements affected: 15)

The most significant finding for these elements is that the condition for removal of a child's disability is linked with the information in foster care element #10, "has child been diagnosed with a disability." These two questions are not to be dealt with at the same time. While it is important to have an edit that checks that the child was removed due to a disability, that information should then be reported in foster care element #10. The responses for the information related to removal reasons may be under-reported. The case file review found more information related to the removal then what was reported in AFCARS. Workers may need to be reminded to select all categories that apply.

Information Regarding Placements (Factor: 2, Number of elements affected: 3)

The State is not collecting placement information for the AFCARS values "runaway" and "trial home visit". The State must add these as a status for placement setting on the input screens. Also, note that these "placement" settings should not be counted in the "number of placement settings" (foster care element #24). If a child returns from a "trial home visit" or "runaway" to the same foster care setting, then the number of placement settings does not change.

The agency has a problem with workers entering placement information in a timely manner, as evidenced by the use of "retro-payments". This is affecting the information that is reported to AFCARS, both in terms of accuracy and reliability. There are also "gaps" in the placement information. This is most likely due to workers not entering information on unpaid placements.

The State is also not counting placements in accordance with AFCARS guidance. Once the State makes the program code changes, this may reduce the number of placements. However, during the case file review, reviewers identified some cases that should have had more placements then what was reported to AFCARS. These cases were those that are affected also by the discussion above on "historical" data due to conversion. Some cases have been opened much longer then what is being reported in AFCARS, therefore, there is historical information missing on the number of placements.

Before system modifications are made, the State should analyze how information is collected and recorded with regard to the beginning and ending of removal episodes and placements. In light of the case file review results, it is suspected that placement changes are being recorded, or at least reported, as a beginning or end of a removal episode, even when the child remains continuously in care.

Data Quality

In addition to the changes needed in the program code, there are significant issues related to data quality. Primarily there is a data entry issue and a lack of use by workers of the system. There is a need for additional training of caseworkers on the use of the system and of specific screens. This will also require ongoing monitoring by supervisors.

Return to Table of Contents