Findings

Two major areas are assessed during an AFCARS assessment review: the AFCARS general requirements and the data elements. The general requirements include the population that is to be reported to AFCARS and the technical requirements for constructing a data file. The data elements are assessed to determine whether the State is meeting the AFCARS definitions for the information required, if the correct data is being entered and extracted, and the quality of the data submitted.

This section provides the major findings resulting from the review of the State's AFCARS data collection. Tab A provides detailed information on the findings for each of the foster care and adoption data elements, the general AFCARS requirements, and the case file review. The AFCARS reporting period under review was April 1, 2001 through September 30, 2001 (2001B).

This report and the matrices reflect changes made by the agency staff to the foster care program code since the on-site review and the issuance of this report. Also, as part of the post-site visit analysis there were further re-examination of the program code, notes made by the review team during the on-site review, and the re-submitted AFCARS data files for the report period under review. As a result, some of the original rating factors were modified from those given at the end of the on-site review. The findings matrix in Tab A reports the previous rating with a "strike-through" mark on it, and then the new rating. The AFCARS improvement plan in Tab B contains the final rating factor. The final rating factor also takes into account the post on-site changes made by the agency staff.

Strengths

Strengths identified by the Federal review team during the review include:

A very dedicated staff and good working relationship between technical and program staff.
There were no areas found to be in full non-compliance.
A significant percentage (49%) of the foster care and adoption (51%) elements were in full compliance.
There were many areas needing improvement that the staff had already identified.
There is good use of the information system by caseworkers to record case information.

We reviewed 37 foster care and nine adoption paper case files. A significant amount of the data in the AFCARS report matched the information the reviewers found in the paper file. Three areas had the greatest number of records where the data did not match. These were in the foster care file: disability information; information pertaining to removals; and, foster parent demographics. A detailed summary of the case file findings can be found at Tab A. Findings from the case record review are consistent with the findings from the review of the AFCARS program code.

Back to Table of Contents

General Requirement Errors

Technical Requirements (Factor: 2)

Extraction

The State is not extracting the foster care records based on the transaction date of discharge (ACYF-PI-CB-95-09, Re-issued May 23, 1995).

The State has corrected the program code and submitted the revised code to ACF. The revised code is now correct.

Conversion

The State did have a conversion plan and committed a considerable amount of time and resources to converting data from the legacy system to the new Statewide Automated Child Welfare System (SACWIS). However, the accuracy and quality of several elements were effected by the conversion. The dates of first removal and latest removal (if the removal occurred prior to conversion) were added to the legal screen, which is not where the program code goes to get the information for the removal episode. The AFCARS data incorrectly reflects the date of conversion for all cases open at the time of conversion, and that are still open, for the date of first and latest removal. Other elements effected are #19, the number of removals from home; #20, date of discharge from the last removal episode; and #24, the number of placements in the current episode.

The conversion issue presented in 11 of the 17 case records reviewed (65%). The effected cases showed the conversion date in AFCARS (July 28, 1997), while the actual date of removal ranged from 1988 to 1997. The results were nearly the same for the date of latest removal.

The State must develop and implement a process that will allow this information to be entered on all appropriate open cases. The process the State develops must also be able to accommodate cases that re-open where there was a removal prior to 1997.

The State has proposed adding a "historical" screen to the system to collect and report historical data. The review team agreed with the States proposed approach.

Statewide Automated Child Welfare Information System (SACWIS)

Some of the recommended and required changes to the system may require the State to submit an update to its Advanced Planning Document, especially if the changes result in a significant commitment of resources, change in scope, or a change in schedule. The State should coordinate the findings and changes required from the AFCARS review with its response to the SACWIS Assessment Review Findings.

Back to Table of Contents

Data Element Errors

Based on the on-site findings and the post-site visit analysis, thirty-three percent (22) of the foster care and thirty-eight percent (14) of the adoption data elements require system modifications. Once the program logic changes are made, the State will need to monitor caseworkers' data entry to ensure that the quality of the data continues to improve.

Changes made to the system with regard to data entry will inevitably result in improved data quality. The State's semi-annual data submission may as a result, fail to meet the missing data standard. In order to ensure that the data are complete, the agency must require workers to enter the data, and assess its validity prior to submitting it to ACF. To do so, the State may utilize the management reports created by the agency, as well as the Data Quality Utility and the Frequency Utility.

Information on Children Diagnosed with Disabilities (Foster care elements number 10 through 15)

The State has a policy requiring that a child receive a medical and/or psychological evaluation within a specified period of time of their removal from home. However, it appears that the workers are not keeping the information up-to-date in the electronic case file. One reason may be that there are two places in the system where a worker can enter this information. One is on the "Removal From Home Reasons" screen. There is a section where the worker checks the question "Does child have a clinically diagnosed disability:" If the answer is "yes," the worker checks the box then selects all the medical categories that apply. On the "Certification of Special Needs" screen workers again have to record the disability information. The screen contains two radio buttons, one for adoption and one for foster care. The information on this screen actually pertains to adoption elements nine through fifteen and not to the disability information in foster care. Also, the current program code extracts data from the information on the removal screen. Having two screens for workers to record disability information may be contributing to the under-reporting of information pertaining to a children with disabilities in foster care.

The State already has a plan that should address the issue of reporting the disability information. This September, the State will implement release 3.1, which will have new disabilities screens. Information will be entered only on medical information screens using more diagnostic codes and verification.

Placement and Foster Parent Information (Foster care elements number 23 - 24, 41, and 49 through 55)

The review identified two issues with regard to the placement information that the State is extracting for AFCARS. One issue is that "trial home visit" and "runaway" are not options for workers to select for the "current placement setting," element #41. Instead, workers select these as "placement end reasons" and not as a current location of the child.

A more significant issue is that the State contracts with placing agencies for therapeutic foster care. The State is not reporting the actual location of the children placed in these homes/group homes. If the child moves from one setting to another within the contracted agency, the number of placement moves are also not being reported. The contract agencies do not have access to FACTS and they do not provide the information to the workers in order for them to enter it into the system. There are several elements for which quality and accuracy of the data is effected. These are: foster care #23, date of placement in current setting; #24, number of previous placement settings; #41, current placement setting; and elements #49 - #55 that pertain to the demographic information of foster parents.

Title IV-D, Child Support Information (Foster care elements number 62)

The SACWIS plan, based on the SACWIS assessment review, addresses the lack of a title IV-D interface. The frequency report shows all records as "does not apply."

The worker can enter this information on the child's assets screen. It would require receiving information from the Child Support Office that a payment had been received. We recommend that the State develop a process to receive this information until the interface is completed. The AFCARS information is required regardless of the status of the interface.

Issue of false "no" data (Foster care elements number 10 and 16)

The review identified two areas that provide a false response in the AFCARS data due to a flaw in system design. The two primary areas affected include the information regarding whether a child has been diagnosed with a disability and whether the child has been previously adopted. Both questions appear on the screen with a checkbox for the worker to select if the answer is "yes." However, one cannot interpret what a blank response means.

In the case of whether a child has been diagnosed with a disability, if the box is not checked it is not known whether the answer is a true "no," if the child has not been seen by a physician, or the worker has skipped over the question. The AFCARS definitions for "no" and "not yet determined" are:

No - Indicates that a qualified professional has conducted a clinical assessment of the child and has determined that the child has no disabilities.
Not Yet Determined - Indicates that a clinical assessment of the child by a qualified professional has not been conducted.

Additionally, the State incorrectly maps blanks as "no;" missing data must be mapped to blank. When making the modifications for the new release, the State must include the options for workers to select "Yes," "No," and "Not yet determined" and map missing data to blanks. In regard to whether a child has been previously adopted, the situation is the same. If the box is not selected one cannot know what this truly means. In the event the child had been abandoned, the worker does not have the option to select "unable to determine." Again, the program code sets a blank response to "no." The State needs to modify the screen to allow the worker to select "yes," "no," and "unable to determine."

Back to Table of Contents

Data Quality

Several elements (18% for foster care, 11% for adoption) are being correctly extracted by the program logic, but data quality remains an issue. Addressing this issue will require additional training of caseworkers regarding the use of the system and of specific screens, and monitoring by supervisors to ensure accurate data entry. The State may want to consider system ticklers/edits that will remind workers to update the information at appropriate times, and review the data in the file at the time of a periodic review (see AFCARS Federal regulation at 45 CFR 1355 Appendix A, I. I. E.

Back to Table of Contents