|
|---|
|
|
||||
|---|---|---|---|---|
|
|
|
|||
| ACF Home | Services | Working with ACF | Policy/Planning | About ACF | ACF News | HHS Home | ||||
Questions?
|
Privacy
|
Site Index
|
Contact Us
|
Download Reader™
|
Print
|
|---|
| Demonstration Type: | Enhanced Child Welfare Training1 |
| Approved: | August 2, 2001 |
| Implemented: | January 1, 2003 |
| Expected Completion: | Terminated early on June 30, 2005 |
| Interim Evaluation Report Expected: | N/A2 |
| Final Evaluation Report Expected: | January 31, 2006 |
Enhanced Training was delivered to all new child welfare case managers in the Illinois Department of Children and Family Services (IDCFS). Enhanced training was also offered to a random sample of newly hired child welfare workers from 48 private child welfare agencies in the Chicago area. Due to lower than expected enrollments, the offer of enhanced training was extended to caseworkers in all private child welfare agencies throughout the State in April 2003.
All IDCFS offices and selected private agencies in Cook County (Chicago area) and surrounding counties (Lake, McHenry, DuPage, Kendall, Grundy, Will, and Kankakee) participated in the project. The demonstration expanded statewide beginning in April 2003.
The Enhanced Training demonstration was designed to improve the efficiency and efficacy of child welfare services and to help new caseworkers improve outcomes for children and families. The State implemented an outcome-focused training and development program to equip new caseworkers with the knowledge and skills necessary to perform in an outcomes-focused child welfare environment. The primary topics covered in the training curriculum included the following: assessing safety and risk within families; Family Group Decision Making; Family Team Meetings; conducting risk and safety assessments; service, permanency, and concurrent planning; attending juvenile court; cultural competency; child development and well-being; working with adolescents; and working with foster parents.
The Enhanced Training curriculum built upon competencies taught as part of the State’s standard Foundation Training, which is provided to all new child welfare workers in the State. The Enhanced Training program included both classroom instruction and on-the-job training. The classroom component involved four weeks of classroom-based instruction. New child welfare workers in teams assigned to the control group received two weeks of Foundation Training before returning to their agency to begin carrying a caseload. New child welfare workers in teams assigned to the experimental group received two weeks of Foundation Training followed immediately by four weeks of Enhanced Training.
Originally, new hires from the private sector also received structured field support for one year following completion of the classroom training. Field support included coaching, shadowing, and post-training “booster sessions.”
The evaluation included process and outcome components, as well as a cost analysis. The State’s evaluator, the Child and Family Research Center (CFRC), used a two-phase random assignment design to evaluate the enhanced training demonstration. Originally, 48 private child welfare agencies participated in the project evaluation. Random assignment occurred at the level of the agency “team,” with each team consisting of approximately seven caseworkers and one supervisor. Of the 150 teams identified in the participating agencies, half were assigned to the control group while the other half were assigned to the experimental group. New child welfare cases were then randomly assigned to teams in either the experimental or control group.
Sampling Plan
The sampling plan called for a minimum of 14 additional new workers to be assigned to the control and experimental groups at a 1:1 ratio each month, for a total of 84 new workers per year in each group. The State had originally estimated that 420 workers would be assigned both to the control and experimental groups, for a total sample of 840 workers. By the end of the demonstration, only 130 caseworkers were assigned to the experimental group and 148 to the control group.
Data Collection
CFRC worked with Northern Illinois University (NIU) to develop two instruments for use in telephone surveys of caseworkers and their supervisors; these surveys - the Caseworker Survey and the Supervisor Assessment of the Caseworker - were designed to measure caseworkers’ and supervisors’ perceptions of changes in knowledge and skills as a result of the Enhanced Training. CFRC originally planned to administer the surveys at 6, 12, and 18 months following a caseworker's completion of training.
Data collection began for the caseworker and supervisor surveys in November 2003. Of the 101 caseworkers identified as enrolled in the control and experimental group, 59 six-month interviews were completed, 29 twelve-month interviews were completed, and 9 eighteen-month interviews were completed, for a total of 97 interviews. Due to contractual problems, collection of further interview data was discontinued in January 2005. Therefore, the analysis of supervisors’ and caseworkers’ perceptions of knowledge and skills is limited to interviews completed between November 2003 and December 2004.
In addition, CFRC had originally planned to track the satisfaction of demonstration participants with the Enhanced Training. At the conclusion of each week of training, participants were asked to complete paper feedback forms to gauge their reaction to the content and presentation of the trainings. However, technical problems with maintaining the feedback form database prevented subsequent analyses of these data.
Process Evaluation
Project Enrollment
During the project’s pilot phase from August 2002 through January 2003, only six private agencies enrolled new caseworkers in the training program, or roughly one worker from each agency. IDCFS staff largely outnumbered private agency staff in the training sessions. An analysis of training registration data revealed that the operational needs of the private agencies prevented the release of new employees to participate in trainings; for many agencies, the six-week commitment was too burdensome. In addition, those agencies experiencing high employee turnover failed to register eligible staff for the training program.
Based on these findings, the State’s original sampling plan was abandoned in April 2003 and the training program was made available to staff in all private child welfare agencies throughout the State. As a result, participation in trainings by private agencies increased during the remainder of the project. By this time, however, the unsystematic withdrawal or withholding of private agency caseworkers from part or all of the training sessions had weakened the original random assignment design and created irremediable bias in the research sample. This made it difficult to attribute any observed outcomes to the effects of the waiver demonstration.
Revisions to the Training Curriculum
Illinois engaged in a continual review of all aspects of the training program. An in-depth analysis of the enhanced curriculum revealed several needed improvements, and IDCFS made several subsequent changes to the curriculum to incorporate additional practice improvements, performance expectations, and statutory mandates. Constant revisions to the enhanced training curriculum became a confounding variable that affected both the implementation of the waiver demonstration and the evaluator’s ability to measure meaningful changes in key project outcomes.
Suspension of Field Support
In January 2004, the field support component of the Enhanced Training program was suspended indefinitely after one of the three trainers left the project. The Enhanced Training program was originally conceived of as a rotational “co-trainer model” in which two trainers provided classroom instruction while a third trainer provided field support to caseworkers. Once a training session ended, one trainer rotated out of the classroom to provide field support while the original field trainer returned to the classroom. The departure of one trainer rendered the continuation of this co-trainer model unfeasible. The termination of the field support component further diluted the fidelity of the State’s original Enhanced Training model and affected CFRC’s subsequent ability to measure key project outcomes.
Post-Training Surveys of Caseworkers and Supervisors
Based on available results from the Caseworker Survey, the Enhanced Training curriculum did not appear to change workers’ perceptions of their preparedness in core case management activities, including (1) facilitating progress toward permanency, (2) engaging in concurrent planning, (3) testifying in court, and (4) participating in family meetings. In addition, many experimental group workers perceived the content of the Enhanced Training in these core areas to be repetitive of what they were exposed to in the standard Foundations Training.
Supervisors of experimental group workers were asked to assess workers’ level of preparedness in core casework activities six months following completion of the Enhanced Training program. Overall, 42 percent of supervisors rated the performance of experimental group workers as “very good.” When asked to compare experimental group workers to other new workers in the agency, 48 percent of supervisors rated experimental group workers as having the same level of preparation as other new workers, while 38 percent rated experimental group workers as better prepared than other new workers.
Outcome Evaluation
The State’s evaluation plan called for the identification of statistically significant differences between the control and experimental groups on the following outcome measures:
Overall, no major differences were apparent between the experimental and control groups on most child welfare outcomes of interest. However, children served by caseworkers in the experimental group did appear to spend somewhat less time in foster care prior to permanency, although sample sizes were too small to determine statistical significance:
Evaluation reports associated with all three Illinois demonstrations are available on the following Web page:
http://cfrcwww.social.uiuc.edu/cfrcdescrip/AcitivitiesMainProjectsEVAL.htm
The Illinois Training Demonstration March 2004 report is available at: http://cfrcwww.social.uiuc.edu/pubs/Pdf.files/IVETrainingWaiver.pdf
1This profile is based on information submitted by the State as of January 31, 2006. This was one of three Illinois Child Welfare Demonstration Projects. Back
2The State did not submit an interim evaluation report due to early termination of this waiver demonstration. Back
Back to Table of Contents
