Findings

Two major areas are assessed during an AFCARS assessment review. These are the AFCARS general requirements and the data elements. The general requirements include the population that is to be reported to AFCARS and the technical requirements for constructing a data file. The data elements are assessed to determine whether the State is meeting the AFCARS definition for the information required, if the correct data is being entered and extracted, and the quality of the data submitted.

This section provides the major findings resulting from the review of the State's AFCARS data collection. Tab A provides detailed information on the findings for each of the foster care and adoption elements, the general AFCARS requirements, and the case file review. The AFCARS reporting period under review was April 1, 2000 through September 30, 2000 (2000B). Data from the report period October 1, 2000 through March 31, 2001 (2001A), and the re-submitted 2000B AFCARS file were also used for comparison purposes during the post-site visit review phase.

Strengths

The State was found to fully satisfy the general requirements for the reporting population. Based on the analysis of 34 foster care paper case files, a significant amount of the data in the AFCARS report matched the information the reviewers found in the paper file. Several of the errors were found in the cases opened prior to the conversion of data from the State's legacy system to the Children Reporting Information System (CHRIS).

Other strengths identified by the Federal review team in regard to the way the State has developed its AFCARS collection and reporting are:

The State staff identified and made appropriate required system modifications prior to the site visit.
Management reports are sent to the local offices informing staff of corrections that need to be made to data.
There were no standards that received a finding of non-compliance.
The State's information system has several mandatory AFCARS fields.
The State maintains high-quality technical documentation.
The information system has a good screen design that displays removal and placement history.
The State has implemented a good technical approach for the collection of the date of discharge from a removal episode.

Also, the information system allows for collection of education information on foster care children and other well-being indicators. This should help the State know more about the children in an independent living (ILP) placement before they are emancipated.

The program code was re-written from COBOL programming language to structure query language (SQL). At the time the program code was re-written, the agency staff identified areas that needed changes and implemented several of those for the report period that ended March 31, 2001. Additionally, the staff made many other changes during the week of the review. Because of this, the Federal team was able to assess many of the changes either on-site or during the post- site segment and incorporate the findings into this report. Another strong point of the State's program logic is that it did not contain many defaults of missing data to valid AFCARS values. When the State made corrections to the program code, missing data were mapped to blanks and not to valid AFCARS values for most of the data elements. The data extracted for the 2001A report period were based on the new program code.

General Requirement Errors

Transaction Dates. The State must extract all foster care records based on the transaction date of discharge (see ACYF-PI-CB-95-09, Reissued May 23, 1995). The State had not been extracting the cases based on the transaction date. During the site visit, the agency staff stated that as of the 2001A transmission, the foster care extraction was based on the transaction date. During the Federal team's post-site visit analysis, it appears the correct extraction method was used for the 2001A file, but not for the resubmitted 2000B file.

In order for the AFCARS submissions to accurately reflect the correct data as of the end of the report period, the transaction date and extraction routine must be correct. By not extracting the cases based on the transaction date, fluctuations occur between regular and subsequent submissions. This brings into question the reliance of the data for use in reports, data profiles, etc.

Case Record Numbers. The State has been using both sequential and encryption routines for the case record number. The State must choose a method and consistently use it for each report period. The program code has been modified to use an encryption routine as of the 2001A report period.

While the State has the option to use either sequential numbering or an encryption routine, the Children's Bureau would prefer the State use an encryption routine. This allows a more reliable method to create and annual file from the two six month submissions.

Conversion. The agency did not add closed cases to CHRIS at the time of conversion from the legacy system. Also, it would appear there was not a policy to require workers to enter the entire removal history on open cases. The majority of the errors found in the case file review were related to "historical¹" data. The sample of cases was selected based on dates of latest removal that occurred prior to the date CHRIS became operational Statewide.

A number of the cases do not accurately reflect the number of removals, and the date of first removal from home. Some of these cases had dates of latest entry into foster care after October 1, 1995². Therefore, all the data, including historical information, are required for AFCARS reporting purposes. If a case re-opens, "historical" data cannot be entered into the system by the worker. The agency needs to add a "historical" data screen, or develop some other method to collect "date of first removal from home", "number of removals", and whether the mother was married at time of the child's birth.

These changes will have an impact on the data that are used by the State for program and evaluation and for the Federal National Standards. The information that the State has provided may significantly be under-reporting the number of removals from home that a child has experienced.

Data Element Errors

Sixty-five percent of the foster care and thirty-five percent of the adoption data elements require system modifications and subsequent training for caseworkers. Once the program logic changes are made, the State will need to ensure that the quality of the data continues to improve by monitoring case workers' data entry.

Also, please note that as you implement corrective action the changes you make to your system and with regard to data entry will inevitably result in improvement in the quality of data. This may correspondingly result in the failure of the State's semi-annual submission of the AFCARS data to meet the missing data standard and, consequently, the assessment of a penalty. In order to ensure that the data are complete, the agency will need to require workers to enter the data, and the validity of the data will need to be assessed prior to submitting it to ACF. This can be done with the management reports created by the agency, using the Data Quality Utility, and the Frequency Utility.

Information on Children Diagnosed with Disabilities (Factor: 2 Number of elements affected: 6)

In the foster care data set, element #10 asks if the child has been clinically diagnosed with a disability. If the response is "yes", then some or all of the applicable disability categories (medical condition, mental, physical, emotional disability, or other medically diagnosed condition) are to be selected. The State's system logic derives the response to the question by checking if the caseworker has selected any of the categories. If none of the categories are selected then "not yet determined" is selected as the response to whether a child has been diagnosed with a disability. This method may be resulting in an under-reporting of the number of children determined to not have a disability. In the 2000B file on 560 records of 4,479 records were marked as "applies." The AFCARS definitions for "not yet determined" and "no" are:

No - Indicates that a qualified professional has conducted a clinical assessment of the child and has determined that the child has no disabilities.
Not Yet Determined - Indicates that a clinical assessment of the child by a qualified professional has not been conducted.

The State's use of a default prevents elements that are blank from failing the AFCARS penalties. However, by defaulting missing data to "not yet determined" the State's data is misleading, and it may be masking the possibility that case workers are not properly filling out the screen that list the types of disabilities. The State must default missing data to blanks. The best approach to collecting this data is to have a question on the screen and the worker to select: "yes," "no," or "not yet determined." Additionally, the selections should not be pre-set, instead each selection should be blank and the worker selects the answer. The screen may need to be made mandatory.

It is also probable that the under-reporting of disability information is due to not all disability information being mapped to AFCARS values. Tab D contains a list of medical conditions and the appropriate mapping to AFCARS.

The data in the 2001A report period indicates that 1,125 records of 5,015 were marked as "applies," 3,143 were "no," and 747 were marked as "unable to determine." This indicates an improvement in the collection and reporting of this information.

Information on Special Needs Adoptions (Factor: 2 Number of elements affected: 7)

AFCARS asks for the primary basis for determining special needs, this should be based on what the worker and his/her supervisor believe is the main barrier to adoption. In CHRIS, the caseworker selects all of the applicable reasons, which are then alphabetized in the selected box on the screen. The program logic selects the first one on the list for reporting to AFCARS. The agency needs to allow the worker to select a primary basis in addition to all other reasons that may apply. If this information is accurately entered, then the response to "has the agency determined special needs" could be derived from whether a selection was made for the primary basis. Otherwise, the system should be modified to include the question "has the agency determined special needs" with "yes/no" as options for the workers to select.

For those children for whom the primary basis for determining special needs was a "medical condition, mental, physical or emotional disability," the same issue described above related to children diagnosed with disabilities apply to the special needs information. Additionally, the State currently does not carry forward the information pertaining to a child's disability information from the foster care record to the adoption record, therefore, the caseworker has to re-enter the information. It may be helpful for this information to populate the relevant adoption record information and may result in more accurate information.

Information Regarding Placements (Factor: 2 Number of elements affected: 3)

The State is not collecting placement information for the AFCARS values "runaway" and "trial home visit". The State must add these as a status for placement setting on the input screens. Also, note that these "placement" settings should not be counted in the "number of placement settings" (foster care element #24). If a child returns from a "trial home visit" or "runaway" to the same foster care setting, then the number of placement settings does not change.

There are several State placement codes that need to be removed from the program code and the caseworker selection list. Additionally, the number of previous placement settings in a removal episode is significantly high. In reviewing the program code and through discussions with the agency staff, it appears the cause is due to the design of the program logic, which is counting every placement setting. For instance, moves that occur for reasons of respite or short term hospitalizations were included in the count of previous placement settings. AFCARS policy is not to count these types of placements. Tab B includes additional questions regarding changes made in the June 22, 2001 version of the program logic. The agency has corrected many of the errors in the program logic. However, the frequency numbers for foster care element #24 remain high in the 2001A file. The agency will need to implement data cleanup procedures.

Also, the State is currently not mapping any placement settings to "group homes." The State is mandated not to have group homes due to a lawsuit. However, it appears that there are placement settings that meet the AFCARS definition of group home. The AFCARS definition is:

"A licensed or approved home providing 24-hour care for children in a small group setting that generally has from seven to twelve children."

In order for the Children's Bureau to be able to compare data on placement settings across all States, the information submitted must meet the AFCARS definition. The agency must map settings that meet the above definition to group home, and not to family foster homes or institution. The State should footnote its AFCARS foster care data file for this element by stating that the State does not license group homes. This information will then be added to reports published by ACF.

Information Regarding Circumstances Associated with Removal (Factor: 2 Number of elements affected: 16)

This is information that is known at the time of the child's removal from home, and should not be updated based on information learned after the removal. The system allows workers to update this information. The agency needs to modify the system to "freeze" circumstances associated with removal. Also, based on the frequencies and the case file review, it appears that workers are not selecting all of the circumstances that apply.

Information Regarding Discharge Dates and Reason (Factor: 2 Number of elements affected: 2)

The agency has designed a good method to ensure dates of discharge are entered into the system. The date of discharge (foster care element #58) is automatically populated when the worker enters an end date for a foster care placement with a permanency reason. Workers do not enter the date of discharge. However, the frequencies for the date of discharge and the reason for discharge (foster care element #56) should be consistent. There should be an equal number of discharge reasons as there are discharge dates. The State staff reported that there are certain circumstances when a discharge reason does not get entered when a case closes. For instance, if a placement setting is closed without a permanency reason, and the exit from placement was due to the child/youth running away, at the point in time that DHS is relieved of custody, the worker enters the date of discharge from foster care on the legal status screen. However, the worker is not currently required to enter a discharge reason. The State is implementing a change in the next release that will have a pop-up window to enter a permanency goal. The new release should not allow a discharge date without a discharge reason.

Data Quality

There were several elements (18% for foster care, 16% for adoption) that are being correctly extracted by the program logic, but there could be an improvement in the quality of the data. This is primarily a data entry issue and will require additional training of caseworkers on the use of the system and of specific screens, and monitoring by supervisors. The State staff suggested modifications that can be done from a system perspective to enhance the quality of the data, for instance, making some additional screens/fields mandatory. The agency is currently using several good management reports to provide information to line supervisors. It was suggested that this approach could be built upon by adding a couple more reports in several areas.

The State staff needs to check that the number of foster care children in independent living is not undercounted. Current undercounts may be due in part to faulty conversions of case records, where foster children in independent living placements were coded as being placed in institutions.

¹ Date of first removal, total number of removals, and date of discharge from the last foster care episode. back
² For cases that were open prior to October 1, 1995, and continue to be open, only "core" data elements are subject to the AFCARS penalties in 45 CFR 1355.40(e). back

Return to Table of Contents