ACYF-CB-PI-01-08

ACF Administration for Children and Families	U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Administration for Children, Youth and Families
	1. Log No.: ACYF-CB-PI-01-08	2. Issuance Date: August 22, 2001
	3. Originating Office: Children's Bureau
	4. Key Words: Child Welfare Demonstration Projects, Reporting Formats, Title IV-E Waivers

PROGRAM INSTRUCTION

TO: State Agencies Implementing Child Welfare Demonstration Projects under Section 1130 of the Social Security Act, and All Other State Agencies Administering or Supervising the Administration of Titles IV-B and E of the Social Security Act.

SUBJECT: Child Welfare Demonstration Project Reporting Formats.

LEGAL AND RELATED REFERENCES:
Titles IV-B and IV-E of the Social Security Act; Section 1130 of the Social Security Act, as amended by Public Law 105-89 (Adoption and Safe Families Act of 1997); ACYF-CB-IM-98-01; and ACYF-CB-IM-00-01.

PURPOSE:
The purpose of this Program Instruction is to provide States operating Child Welfare Demonstration Projects with standardized formats for reporting information required in the Terms and Conditions that govern each State's Child Welfare Demonstration Projects. These Terms and Conditions have been jointly agreed to by the Department of Health and Human Services ("the Department") and each State agency selected to operate a demonstration, and are based upon the operation and evaluation planned for each demonstration project.

BACKGROUND:
Under Section 1130 of the Social Security Act ("the Act"), the Department is given authority to approve up to ten States for Child Welfare Demonstrations in each of the five fiscal years 1998-2002. These demonstration projects involve the waiver of certain requirements of titles IV-B and IV-E, the sections of the Act that govern foster care, adoption assistance, independent living, child welfare services, promoting safe and stable families, family preservation and support, and related expenses for program administration, training and automated systems.

Child Welfare Demonstration Projects operating under this waiver authority (according to the original announcement that appeared in the Federal Register of June 15, 1995) are required to:

Be consistent with the purposes of titles IV-B and IV-E of the Act in providing child welfare services, including foster care and adoption, that is:

Assure the safety of children and protect the rights of children and their families; and

Ensure permanency for children through intensive family preservation and support or through reunification or adoption efforts.

Be cost-neutral to the Federal government; and
Be independently evaluated.

Such demonstrations will provide valuable knowledge that will lead to improvements in the delivery, effectiveness, and efficiency of services. A primary goal of the demonstration projects is to produce information on how title IV-B and IV-E funds can be used to improve outcomes for children and families. The Terms and Conditions agreed to by the State and the Department represent a written commitment to conduct the project and the evaluation, and to report to the Department on project implementation, operation, and evaluation. The reporting formats addressed in this Program Instruction provide the infrastructure to facilitate the compilation and dissemination of this information. The standardization of report formats allows States to provide information more efficiently and in a uniform manner, enabling greater comparability for evaluation purposes. The report formats can be found on the ACF Web site at http://www.acf.dhhs.gov/programs/cb.

INSTRUCTION:
Under the authority of Section 1130 of the Act, the Department requires information from States applying for a waiver demonstration project, as well as those States that have been approved as a demonstration. These instructions provide guidance on the specific reporting that is required by the Department; report formats are included as attachments. Each State with a Child Welfare Demonstration Project must submit all reports noted in their Terms and Conditions according to the established time-frames. States operating more than one demonstration project are required to submit separate reports for each demonstration.

1. Initial Design and Implementation Plan (See Form A on Page 7)

The Initial Design and Implementation Plan (which will serve as the first Quarterly Progress Report) will summarize the detailed plans and procedures for implementing the demonstration and for ensuring that evaluation activities have been incorporated into the start-up period. This report will be useful to the Children's Bureau in documenting how the State intends to implement the major activities and key tasks under the demonstration, including timelines for accomplishing activities. Because of the variations across States in the specific provisions of their waivers and related Terms and Conditions, the Initial Design and Implementation Plan will also assist the Children's Bureau in understanding whether States are implementing their demonstrations in a way that is consistent with their Terms and Conditions. By establishing benchmarks for implementation objectives early on, the Children's Bureau will be better able to assess both the various barriers and challenges that demonstration projects are facing during the start-up period and how they may impact their implementation of planned activities. The report should include a final Work Plan that outlines key tasks, reporting requirements, and timelines throughout the demonstration. The report also should include plans for selecting participants in the demonstration, including how they will be selected and how the suitability of services will be determined. Other information concerning demonstration and evaluation activities, as stated in the Terms and Conditions, should also be included in the Initial Design and Implementation Plan.

For demonstrations with a managed care component, the Initial Design and Implementation Plan should specify the financial payment schedules and amounts for providers, the risks to the State and the providers, provider incentives, and any performance monitoring standards. Copies of negotiated contracts with providers should be submitted with the report. These must be provided prior to implementation. Any agreements with public and private providers who will deliver support and services must include standards of quality and safety, and practice requirements identified by the State.

2. Evaluation Plan (See Form B on Page 9)

Demonstration projects are expected to test policy-related hypotheses and assess whether proposed interventions support improved child welfare outcomes. The preparation of a detailed and comprehensive evaluation plan will assist the Children's Bureau in understanding: the specific research questions to be explored; the major variables to be measured; the data sources for these variables; the final sampling plan; the data collection procedures; and the major data analyses to be performed. Generally, each Evaluation Plan will contain information on how the Process Evaluation, the Outcomes Evaluation, and the Cost-Benefit Analysis will be conducted. Since the ability to attribute changes to the demonstration (rather than to other factors) is increased when the method of random assignment is followed, most demonstration projects are required to use random assignment to assign cases to experimental and control groups (unless otherwise noted in their Terms and Conditions). The Evaluation Plan should include detailed procedures explaining how random assignment will be used. If applicable, the Evaluation Plan should include procedures for using Comparison Sites or a Matched Sample, as well as any other information stated in the Terms and Conditions.

3. Quarterly Progress Reports (See Form C on Page 11)

Prior to demonstration project implementation, Progress Reports shall be submitted on a quarterly basis. These reports should provide a basic update on the status of each activity or task that was identified in the Initial Design and Implementation Plan report. By providing information organized according to activities, the Children's Bureau will be able to assess whether the demonstration is proceeding according to the original plan, if there are any technical assistance needs, and whether States are still likely to meet their anticipated implementation schedule. The Quarterly Progress Reports should also identify any problems encountered that may have an impact on the design or anticipated implementation schedule, along with suggestions for resolving these problems, particularly those which will require review and approval of the Children's Bureau. Any other information stated in the Terms and Conditions regarding demonstration and evaluation activities should be included in the Quarterly Progress Reports.

4. Semi-Annual Progress Reports (See Form C on Page 11)

After demonstration project implementation, Progress Reports shall be submitted on a semi-annual basis. The Semi-Annual Progress Reports are intended to provide the Children's Bureau with a summary of the project's accomplishments and evaluation activities during the reporting period, so that the Children's Bureau can remain informed of States' progress regarding the operation of their demonstration and evaluation findings. Every other Semi-Annual Progress Report will serve as an annual overview, summarizing the progress over the previous four quarters in implementing provisions of the demonstration and carrying out the evaluation. The reports shall also be used to present any interim findings from the evaluation. Information that documents "what works" can be especially helpful to other waiver States, particularly those that are implementing similar or related activities in their demonstrations. The Semi-Annual Reports should indicate any issues or problems encountered in implementing the demonstration or evaluation, as well as plans for resolving the problems. These reports shall include updates on the resolution of any significant problems identified in the Initial Design and Implementation Plan report, as well as on any other specific issues stated in the Terms and Conditions that pertain to demonstration and evaluation activities.

5. Interim Evaluation Report (See Form D on Page 13)

The Interim Evaluation Report will be formatted similarly to the Final Evaluation Report, and shall be submitted approximately two and a half years following the date of implementation. This report will include key findings from the Process Analysis, such as whether the demonstration was implemented as designed and planned, characteristics of the population served, characteristics of the service delivery system, and whether implementation objectives were achieved. Organizational aspects, such as the planning process, staffing structure, and methods of project implementation at various organizational levels, should be addressed. The Process Analysis will focus on the frequency, nature, and intensity of services provided to each case in the experimental group by each type of provider. For cases in the control group, receipt of comparable services and the provider of services will be documented. Differences in timeliness, access, and cost of services between the experimental and control groups will be provided in the report. Any contextual factors, such as the social, economic, and political forces that may have a bearing on the replicability of the intervention, or influence the implementation or effectiveness of the demonstration, should be included. Outcomes based on a comparison between experimental and control groups should be reported, as well as any cohort- or sub-population-analyses, where applicable (e.g., differences in outcomes when age, race, gender, and/or special needs are considered). Differences between the experimental and control groups with regard to comparable resources, services, activities, staffing, number and quality of contact hours should also be reported. Other information concerning demonstration and evaluation activities that were stated in the Terms and Conditions shall be included as well. These findings should be submitted to the Children's Bureau as an interim report to provide preliminary information about the waivers.

6. Final Evaluation Report (See Form D on Page 13)

The Final Evaluation Report is to be submitted six months after the demonstration project ends. This period is intended to provide States with sufficient time to analyze and report findings from the entire project period. The content of this report follows the same outline as the Interim Evaluation Report.

SUBMITTAL DATES FOR REPORTS:
The following table provides dates for submitting the various reports discussed in this notice. Any agreements on report submittal made in the Terms and Conditions will supersede the dates set forth below. Once approved as a demonstration, States are required to submit all of the following reports to the appropriate ACF Regional Office and the Children's Bureau Title IV-E Waiver Evaluation and Technical Assistance contractor, James Bell Associates; Suite 1120; 2111 Wilson Blvd.; Arlington, VA 22201.

Report	Required Timeframe for Submission
Initial Design and Implementation Plan	Within 90 days following acceptance of the Terms and Conditions
Evaluation Plan	Within 60 days following award of the evaluation contract
Progress Reports (Pre-Implementation)-quarterly until implementation begins	No later than 30 days after the conclusion of each quarterly period
Progress Reports (Post-Implementation)-semi-annually once implementation begins	No later than 30 days after the conclusion of each semi-annual reporting period
Interim Evaluation Report	Prior to conclusion of the 10^th quarter (i.e., two and a half years) following the implementation date
Final Evaluation Report	Six months after the project ends

PAPERWORK REDUCTION ACT:
Under the Paperwork Reduction Act of 1995 (Public Law 104-13), an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB Control Number. The OMB Control Number for the demonstration reporting formats is 0980-0272, and the Expiration Date is 6/30/2003.

INQUIRIES:
Please direct all inquiries to the appropriate ACF Regional Office.

_/SIGNED/_________
James A. Harrell
Acting Commissioner
Administration on Children, Youth, and Families

Attachments

Form A - Initial Design and Implementation Plan Outline
Form B - Evaluation Plan Outline
Form C - Progress Report Outline
Form D - Interim and Final Report Outline