Findings

This section provides the major findings resulting from the review of the State's AFCARS data collection. Tab B provides detailed information on the findings for each of the foster care and adoption data elements, the general AFCARS requirements, and the case file review. The AFCARS reporting period under review was April 1, 2002 through September 30, 2002 (2002B).

As part of the post-site visit analysis of the State's documents, the data, the case file review findings, and team member notes are assessed to make the final determination of findings. As a result, some of the original rating factors were modified from those given at the end of the on-site review. The findings matrix in Tab B reports the previous rating with a "strike-through" mark on it, and the new rating. The AFCARS Improvement Plan in Tab C contains the final rating factor. Several changes to the rating factors were made due to the final analysis of the paper case file review.

Strengths

The State has implemented several features into their SACWIS system - FACTS (Families and Children's Tracking System) - since its Statewide Automated Child Welfare Information System (SACWIS) Assessment Review. Many of these features can be used to encourage data entry and data accuracy. One of these features is a bulletin board that allows messages to be sent between supervisors and their workers and/or unit. Supervisors could use the bulletin board to remind a worker(s) to update case records and to send reminders regarding the proper use of certain values on the input screens. Another feature is the title IV-E eligibility determination module that was implemented to allow AFDC (Aid to Families and Dependent Children) determination for title IV-E in FACTS. The State intends to use the framework for this module for the AFCARS summary section in FACTS. Another good feature of FACTS is that caseworkers can update information from summary screens. If information is missing from the summary report, there are hyperlinks that will link to the screen where the information needs to be entered. This feature is going to be extended to the AFCARS elements. A significant improvement to the system links the court and placement screens together. The State has also implemented several edit checks into the system (i.e., date edit checks) and is adding ad hoc management reports. The reports can be run statewide, by county and by worker.

Based on the on-site findings and the post-site visit analysis, 17% (11) of the foster care and 43% (16) of the adoption elements are reported correctly. The State is also correctly reporting the adoption population.

General Requirements - Population Errors

The standards for the AFCARS foster care population require that children who have been placed in out-of-home care for more than 24 hours be included in the reporting population. Currently, the State is including children that have been in care for less than 24 hours. The State needs to make a correction to the way it extracts the foster care population to exclude children that have not been in care for 24 hours or more. The State may need to use the date and time that the child was placed to indicate whether the child had been in care for less than 24 hours.

Also, children who have been in foster care and are returned to their home while under the placement, care, or supervision of the State agency must be included in the AFCARS population. If the child is returned home for a specified period of time, the requirement is that the State report the child in AFCARS for the entire specified period of time. If the child is returned home for a non-specified period of time, and the timeframe exceeds six months, the State may consider the child discharged from care, placement or supervision for AFCARS purposes. The State has been reporting all children that are returned home under the responsibility of the agency with discharge dates after six months, regardless of a specified period of time that may exceed six months. The State must include these children in the AFCARS report as appropriate, and according to the criteria listed above.

Data Element Errors

Based on the on-site findings and the post-site visit analysis, 42% (28) of the foster care and 38% (14) of the adoption data elements require system modifications. Changes made to the system with regard to data entry will inevitably result in improved data accuracy and quality. The State's semi-annual data submission may, as a result, fail to meet the missing data standard. In order to ensure that the data are complete, the agency must require workers to enter the data, and assess its validity prior to submitting it to ACF. To do so, the State may utilize the management reports created by the agency, as well as the Data Quality Utility and the Frequency Utility issued by ACF.

Information on Children Diagnosed with Disabilities (foster care elements #10 - 15)

This is an area that needs significant attention with regard to both system design and caseworkers entering the information. One of the problems associated with the quality of the data was due to an error in the program code that requires all the disabilities to be selected before element #10 can be answered as "yes." The State indicates this has been corrected and the frequencies for the report period ending March 31, 2003 indicate a higher response rate.

A more significant finding is that the State's information system does not ask the question "Has the child been clinically diagnosed as having a disability?" If none of the conditions are checked, then the program code maps blank information to "not yet determined." This results in inaccurate data. For AFCARS purposes, "not yet determined" indicates that "a clinical assessment of the child by a qualified professional has not been conducted." The State must modify the system to include this question and the values "yes," "no," and "not yet determined." This will provide more accurate information regarding whether a medical or psychological professional has seen a child.

The State's system also has more than one location where medical and disability information can be entered by the caseworkers. The State needs to review the screens and the system design and decide which screen is most suitable for the entry of this data, or modify the program code to check each of the screens.

There is an extensive list of State medical and psychological codes in the program code. There are some conditions that could be reported to AFCARS that the State is not mapping. At the end of the Tab C, the AFCARS Improvement Plan, there is a table listing the State's codes and the appropriate AFCARS value.

Additionally, the design of FACTS requires caseworkers to re-enter medical information on a child if the child re-enters foster care. The State should consider "moving" prior medical information into the new removal episode. A tickler could be added to remind workers to review the information and update appropriately.

Hierarchy in the program code that determines primary basis for special needs (adoption element #10) There is not a field on the screen for the adoption specialist to enter the primary basis for determining special needs. Instead, the system determines the primary basis by following a hierarchy written into the program code: medical, emotional, intellectual handicap; age; sibling; race and over age of three; and other.

The State must add a field on the screen to identify a primary basis for special needs and remove the hierarchy from the program code.

Information on sources of Federal support/assistance for the child (foster care elements #59 - 65)

The requirement for these elements is to report if a payment has been made on behalf of the child during the six-month report period. Instead, the State is incorrectly reporting if the child is eligible for one of these sources of income. This has specific implications for the title IV-E foster care reviews, and must be modified accordingly.

Missing data due to defaults

Of the elements that require a system change, many of them (seven foster care and six adoption elements) only require a change of mapping missing data to blanks and not to a valid AFCARS value. Additionally, eight foster care elements and one adoption element need defaults corrected along with additional system modifications. While this will be an easy correction to the system requirements, it will require workers to enter any missing data and clean up data fields. The State should note that by removing the defaults the data elements may fail the AFCARS 90% standard for incorrect data.

Data Quality

Forty-one percent (27) of the foster care and 19 percent (7) of the adoption elements are correctly extracted to the AFCARS file, but the quality of the data needs to improve. The majority of the data quality issues are to due to timely entry of data by caseworkers, and keeping the information up-to-date. As noted in the previous section, there are many elements that will need to be further assessed for accuracy once the default values are removed from the program code. Workers will need to clean up and add missing data.

Data accuracy regarding removal dates and the number of removals (foster care elements #18 through #21)

While the State had a very well-developed plan for data conversion to the new SACWIS, there were some inaccuracies noted during the case file review that need to be addressed, and the data needs to be cleaned up.

Also, training needs to be provided to workers on how to enter placement changes for youth that are placed in juvenile justice facilities while still in the State's responsibility for care and placement. Workers must be instructed not to enter these placement changes as discharges from care. The incorrect entry of this information also affects the accuracy of the information for current placements, date of placement and the total number of placements.

Missing data on foster care placements provided by the contract providers

We have determined that not all placements made by private providers are entered into the system. The staff assisting with the case file review indicated that there was a lot of missing information regarding the private agency providers. The lack of this information being shared with the agency in a timely manner affects several of the foster care elements. These include: the date of the current placement, the current placement, the family structure of the foster family, and the demographics of the foster family. The State must address this area as soon as possible and clean up the affected data files.

Increased supervisory oversight of data entry by workers

Once changes are made to the program code and/or to the data entry screens, the quality of the data will need to be monitored for accuracy. It may be necessary to implement additional training for caseworkers and monitoring by supervisors to ensure accurate data entry. The State may want to consider system ticklers/edits that will remind workers to update the information at appropriate times, and review the data in the file at the time of a periodic review (see AFCARS Federal regulation at 45 CFR 1355 Appendix A, I. I. E).

Return to Table of Contents