Chapter IV: Procedures for the Pre-Site Visit Phase: State Guide to an AFCARS Assessment Review

Chapter IV: The Pre-Site Visit Phase

A. Introduction

This chapter outlines the activities State agency staff need to complete in order to prepare for the on-site phase of the AAR. During this phase the State should identify those areas it wants to specifically address during the site visit. The State agency should view the AFCARS Assessment Review as an opportunity to examine its foster care and adoption data concerns completely and take advantage of the intensive technical assistance available from ACF. The State may also want to review its CFSR data profile to identify any areas in which it may want to request assistance from the Federal AFCARS review team.

The pre-site visit phase involves the following activities:

documentation exchange;
case scenario entry;
case file review preparation; and
finalize the on-site agenda.

Back to Table of Contents

B. Overview Of The Pre-Site Visit Activities

The pre-site visit phase officially starts when a State is notified of its AFCARS Assessment Review. Ideally, a State will know by July that it will have its AFCARS Assessment Review the next Federal fiscal year. A State will not have less then three months notice of an AFCARS Assessment Review.

Once the dates are selected for the AFCARS Assessment Review, a confirmation letter will be sent to the State. This letter contains the following information:

The date of the on-site review.
The date the State's system documents are due to ACF.
The date the case file sample and test cases will be sent to the State.
The date the test case extraction is due to ACF.
The contact information for the Federal review coordinator and members of the Federal team.

Back to Table of Contents

C. Documentation Exchange

This section describes the documents that are exchanged between the State and ACF. It is not ACF's expectation that the State create new system documentation for the review team to analyze. The State should submit its technical documentation in its current format. Exhibit 4 contains examples of the types of documents to submit. The documentation should reflect the options available to the case workers for each AFCARS data element, and the State value for that option.

The Federal review team needs a "fixed" set of documentation in order to conduct a quality review for the State. The State should submit its most current version of the extraction program code and reference tables. Any modifications made to the screens or the program code as it relates to AFCARS, after the time ACF receives the State's documentation, will be included in the post-site visit review.

The State's system documentation may be submitted earlier then the due dates. States should begin consolidating the system documentation as soon as possible and maintain an up-to-date library of system documentation. This is helpful for those States that are developing a SACWIS and to those preparing for the SACWIS Assessment Review.

Procedure

By the due dates, submit copies of the documentation to ACF. The documentation can be submitted in hard-copy form or electronically.
- If it is e-mailed, send to the ACF Regional Office and the Children's Bureau staff.
- If sent as a hard-copy in lieu of electronic copies of the documentation, then two copies of the documentation should be sent to the ACF Regional Office and two copies to the Federal review team coordinator.
- If electronic copies of the documents are sent on a compact disk, this must be sent via overnight mail to the Children's Bureau . Do not send zip disks.
Contact the ACF Technical Lead for questions concerning the technical documentation.

Exhibit 4: State Documentation List

Programming Logic (Source Code): The programming logic that derives and extracts each AFCARS element from the State's System.
Data Element Dictionary: A data dictionary or similar documentation will provide the Federal team with definitions and representations of the State's values for AFCARS related fields. This document should be complete and reflective of options available to the case worker to select. This documentation may be supplemented by code/lookup tables that will enable the team to understand elements referenced in the source code or mapping forms. For example, placement code 257 = relative foster care, licensed.
Relevant Reference Tables and Fields: Specific fields and tables from which the data are pulled.
Database Structures: Tables referenced in mapping documentation.
Extraction Routine Notes: A logical description, or the steps in pseudo code, of how records are selected for inclusion in the foster care and adoption submission.
Data Entry Screens: Printouts of screens relevant to AFCARS data elements.
Change Request Orders: Changes that the State has planned that impact the AFCARS data elements that are not included in the current version of the extraction program code.
Interface Structure: Information on interfaces to non-SACWIS systems that contribute data to the AFCARS submission.
Organizational Charts: Charts of the program and information technology divisions.

Note: It is crucial that the documentation be submitted to ACF on time. Delays in the submission of the documents affect the Federal team's preparedness for the on-site review and may result in an incomplete on-site evaluation.

Back to Table of Contents

D. Test Cases

The use of test cases is a way to assess the State's collection and extraction of the AFCARS data. This process is another way for the Federal team to gain a better view of the State's policies, practices, training, and extraction routine. The records that are created by the test cases are extracted from the State's data system and compared with an established answer key. This information in conjunction with the analysis of the extraction program code provides, and the other components of the AAR, the Federal team with a comprehensive view of the State's system.

The intent of the test cases is to validate the State's information system for accuracy. As such, the Federal review team coordinator may answer some questions the State has on the cases, but most will not be answered in the interest of determining objectively the State's AFCARS capability. Appendix G contains the instructions for entering the case scenarios.

Procedure

The Federal review coordinator e-mails the test cases to the State review coordinator by the established due date. The State will receive the test cases no later than six weeks prior to the on-site phase.
The State review coordinator will identify a staff member who is familiar with the system and/or case practice to enter the cases. Approximately 40 working hours should be allotted for this task.
A training or test environment may be used to enter the test cases. This should mirror the production system. The State needs to ensure that the method used to enter the data will also allow for the entry/extraction of the financial data elements.
Once the test cases are entered into the system, the State review team coordinator will contact the ACF Technical Lead for instructions on transmitting the test file via Connect:Direct. The file must be submitted by the established due date.

Back to Table of Contents

E. Case File Review Preparation

Another tool used in the AFCARS Assessment Review process is the case file review. A sample of 80 foster care records and 30 adoption records is pulled for the AFCARS reporting period under review. The AFCARS data submitted to ACF on each record is then compared to information found in the paper case file during the onsite review. The process involves all members of the State and Federal teams, technical and program. The State is encouraged to incorporate field staff, including supervisors, staff from training units, etc., as part of the State team for the purpose of reviewing cases. See Chapter III for additional information on who the State may want to include on the case file review team. A preliminary analysis of the findings is conducted during the onsite review and shared with the State. A further discussion of the onsite process can be found in Chapter V.

ACF has found that while there may be challenges to identifying the information in the paper file, the process provides very valuable information to the review teams. The case file review is also the only means for the Federal team to assess the accuracy and the level of completeness of the State's conversion process from a paper or legacy system to its new information system. The findings often provide additional supplemental information that increases the Federal teams understanding of the data reported to AFCARS. Also, this process allows the review team to assess how well records are being kept up-to-date, how accurate the data are in AFCARS, and usage of the State's information system. Oftentimes, this process does not identify new problems, but usually confirms findings from the other components of the AAR.

While ACF recognizes that current data may no longer be in the paper files but will, instead, be in the automated information system, there are some documents that may not be part of the State's information system, such as medical reports, court reports, home studies, etc. These documents oftentimes provide a significant amount of the information for the case file reviewers. It is important that the State have available the complete case file; oftentimes relevant information is found in a sibling's folder instead of the folder on the child under review. Intake documentation can also be a valuable source of information.

Since the case file review is the only means to assess conversion, the cases selected for the review will primarily be those in which the most recent removal date, or the first removal date, precedes the date the State's system went operational. If the State phased in its operational status, then the sample may reflect these dates.

Depending on the length of time since the State converted to a new information system, this portion of the AAR may be modified to accommodate the State's situation, such as how to handle placement information. During the initial conference call, ACF and the State will discuss and plan for the case file review and make adjustments as necessary.

Procedure

Once the State review team coordinator receives the list of encrypted or sequential case record numbers from the Federal review team coordinator, he/she should:

Create a list to cross-reference the encrypted/sequential record number with the actual case record number.
Arrange for the case files to be delivered to the State office by the first day of the on-site review.
Ensure that case records are as complete and up-to-date as possible, and include any files maintained separately, e.g., child protective services files, provider files, and eligibility files. All case records should include:
- court orders;
- placement history;
- caretaker (person child was removed from) information;
- medical history;
- biological parent file for the adoption records; and
- foster care records on children adopted (if the relevant AFCARS information cannot be found in the child's adoptive record).
  
  Note: Sealed adoption records. If a sufficient number of cases cannot be identified for review due to their having been sealed, then the State will have to petition the court to open the records prior to the on-site review.
Identify and schedule staff to participate in the case file review. It is recommended that the State have field workers, supervisors, and/or regional administrators participate in this part of the review.

Back to Table of Contents

F. The On-Site Agenda

Appendix H contains a sample agenda for the on-site review. Any changes to the overall agenda must be discussed with, and given to, the Federal review team coordinator. The State review team coordinator should provide the Federal review team coordinator with a list of the names and titles of the State staff participating in each segment of the AAR.

Back to Table of Contents

G. Site Logistics

The on-site review activities are conducted in one location. The State capitol is generally the designated location since it is the location of the State agency central office/administrative staff, but another site may be mutually agreed upon. It is preferable to conduct the meetings in one large room to allow enough space for the size of the team and materials needed for the AAR. A separate room can be added on the day of the case file review, as this activity will include additional staff and the case files. The room used for the case file review should be secure to protect the confidentiality of the case records.

Back to Table of Contents

Continue to Chapter V: Procedures for the On-Site Review Phase

Attachments:

ACYF-CB-IM-02-05

State Guide to an AFCARS Assessment Review HTML or PDF(332 KB)