Home > Child Welfare Monitoring > Instruments, Tools, and Guides > Procedures Manual > Contents > Chapter 2: The Review Teams
Chapter 2
The Review Teams
Each Child and Family Services Review (CFSR) is a two-phase process that comprises a Statewide Assessment and an onsite review of child and family services outcomes and program systems. States determined not to be in substantial conformity with these outcomes and systemic factors then develop and implement a Program Improvement Plan (PIP).
The reviews are conducted jointly by Federal and State representatives, with Federal staff providing overall guidance during the planning and implementation of the review. During the review process, Federal and State teams conduct the activities associated with the two phases and the PIP process: these include a Statewide Assessment Team (State); Onsite Review Team (Federal and State); and PIP Development Team (State). States may appoint different personnel to each team; however, they are encouraged to provide some overlapping membership between teams to ensure the transfer of experience and knowledge. The Statewide Assessment Team, for example, is encouraged to begin thinking about the program improvement process during the Statewide Assessment phase.
The State child welfare agency administrator appoints a senior State staff person (State Lead) to provide oversight to the State teams and to all State review activities. This point person serves as the liaison to the Children's Bureau Central and Regional Office staff, and Federal contractors; participates in all review-planning activities; and oversees, as appropriate, the staff designated to plan and conduct the review. This person also may serve as the State Team Leader during the onsite review (as described below), or the agency administrator may appoint another senior staff person to fill that role.
The State leadership then appoints members to the review teams. The Statewide Assessment Team and PIP Development Team are made up of State agency staff and representatives of the principal agencies, organizations, or groups working on child welfare issues in the State. The Onsite Review Team comprises both Federal staff and trained consultants and State and local child welfare agency staff and their external partners. (The consultants are part of a national pool of experienced child welfare professionals managed by the Child Welfare Review Projects. 1)
The State leadership also assigns responsibility for establishing a communication system to ensure that information about the CFSRs is released to the media and the public throughout the review process.
This chapter describes the roles and responsibilities of the teams that conduct activities in each of the two review phases and the PIP process. Later chapters describe the responsibilities of these teams in the context of the review process: the Statewide Assessment Team's responsibilities in chapter 3; the Federal, State, and Federal contractor responsibilities in conducting the onsite reviews in chapters 4 and 6; and the PIP Development Team's responsibilities in chapter 7.
A. Composition of the Review Teams
The following factors should be considered in forming the Statewide Assessment, Onsite Review, and PIP Development Teams:
- To the extent possible, teams should be diverse in their membership, comprising individuals who represent the major populations served by the State.
- The Children's Bureau encourages cross-system coordination and consultation in States through the CFSR, as promoted by the title IV-B, subpart 2, planning process. To that end, the State should involve, on all three teams, individuals from outside the State agency who represent the planning team that developed the State's Child and Family Services Plan (CFSP) and Annual Progress and Services Reports (APSRs). To manage effectively the release of information to the media and the public throughout the review process, the State should consider keeping the State public information office or media relations staff apprised of the activities of the Statewide Assessment and PIP Development Teams.
- The State should select review team members who do not have potential conflicts of interest. Members of the Onsite Review Team, for example, should have no prior casework or supervisory responsibility in the site that they are assigned to review (as described in chapter 1, the onsite review takes place in three local sites; the review in the metropolitan site typically is conducted by two teams).
In addition, State staff are given the opportunity to review the list of cross-State participants and consultant reviewers selected to supplement the Federal component of the Onsite Review Team. This allows the State to ensure that these individuals do not have a conflict of interest with participating in the onsite review (for example, a consultant who was previously employed by the child welfare agency being reviewed or resides in or conducts child welfare-related business in the State under review).
- Staff or consultants of the Children's Bureau-funded National Resource Centers (NRCs) may not serve on the Onsite Review Team. (The NRCs can provide technical assistance [TA] to a State as part of the CFSR process.) This ensures that no conflict of interest exists if, following a review, the State receives TA from the NRCs. The same applies to other TA providers that the State may hire to assist in developing a PIP or conducting other child welfare improvement planning following the review.
All review team members receive an orientation to and/or training about key components of the CFSRs. The State child welfare agency provides an orientation to the Statewide Assessment and PIP Development Teams. The Children's Bureau Regional Office and Children's Bureau-funded National Resource Centers can provide support to the State child welfare agency in preparing these orientations. The Children's Bureau, via the Child Welfare Review Projects, provides training to consultants and the State members of the Onsite Review Team regarding the onsite review.
The following sections describe the structure and functions of each of the review teams.
B. Statewide Assessment Team
B.1. Structure
States must include broad representation from within and outside the child welfare agency in forming a team to conduct the Statewide Assessment. The team should include representatives of organizations consulted in developing the CFSP and APSRs and who are expected to be involved in developing and implementing the PIP. States also should consider including on the Statewide Assessment Team individuals from within and outside the State child welfare agency who have the skills and background to serve as case record reviewers and interviewers and who are available to serve on the Onsite Review Team.
The following are suggested participants in the Statewide Assessment Team:
- Administrators and program specialists from the State and local child welfare agencies
- State and local agency staff with expertise in areas examined during the Statewide Assessment, such as information systems, quality assurance, training, and licensing
- Local child welfare agency staff who have knowledge of front-line practice and supervisory issues
- Judges and other court-related personnel, especially staff of the State's Court Improvement Program (CIP)
- Representatives of the major domains outside child welfare that are addressed in the Statewide Assessment, such as education, health, mental health, substance abuse treatment, domestic violence prevention, and juvenile justice
- Tribal representatives
- Legislative personnel who focus on child welfare issues or funding issues that affect child welfare
- Advocacy groups and consumer representatives, including children and youth in foster care or the groups that represent them
- Service provider representatives, including foster and adoptive families
- University or research-related partners of the State involved in data collection and analysis, training activities, or other relevant areas
- Partners that represent the diversity of the State's population, especially in relation to those served by the child welfare system
B.2. Functions
Members of the Statewide Assessment Team may engage in the following types of activities:
- Participate in training or orientation sessions
- Attend meetings related to the Statewide Assessment or the review process
- Analyze the data related to outcomes and systemic factors
- Collect additional data as needed
- Gather information pertaining to the agency's performance, such as conducting or participating in focus groups, surveys, or interviews
- Develop, review, and comment on drafts of the Statewide Assessment
- Participate in conference calls with Federal staff during the Statewide Assessment process (Statewide Assessment Team leadership only)
- Make recommendations pertaining to the onsite review, such as sample composition, site selection, and Onsite Review Team composition
- Identify the State's strengths and areas needing improvement on the basis of data and information gathered for the Statewide Assessment
- Explore strategies for possible program improvement efforts in areas identified as needing improvement, and make preliminary recommendations to the State's PIP Development Team
C. Onsite Review Team
C.1. Structure
The Onsite Review Team comprises both Federal and State staff, with trained consultant reviewers supplementing the Federal component of the team. Federal staff select the Federal and consultant reviewers, while State agency officials choose the State Review Team members, who may be State agency staff or external representatives. The overall team is divided into four local site teams; two teams operate at the metropolitan site and one each at the other two local sites. The chart below shows the structure of the Onsite Review Team.

d link
The following are the roles of the Onsite Review Team members:
- Team Leaders:
- National Review Team (NRT) Team Leader: A Federal agency representative who provides overall leadership for the onsite review and is a member of the NRT. The NRT comprises staff from the Children's Bureau Central and Regional Offices who provide leadership to the review teams in planning and conducting the CFSRs.
- Children's Bureau Team Leader: A Children's Bureau Regional Office representative who assists in providing overall leadership for the onsite review.
- State Team Leader: A State agency representative who serves as the State's lead representative for the onsite review.
- Local Site Leaders:
- NRT Local Site Leaders: Four Federal representatives from the NRT, each of whom provides overall leadership to a review team in one of the three local review sites (there are four onsite review teams; the review in the metropolitan site typically is conducted by two teams).
- Federal Local Site Leaders: Additional Federal representatives may assist in providing leadership to a review team in one of the three local sites. One of the individuals filling this role at each site may be a high-performing and specially trained Consultant Co-Local Site Leader identified by the Children's Bureau.
- State Local Site Leaders: Four State agency representatives, each of whom serves as the State's lead representative for a review team in one of the three local sites. The State Local Site Leaders work closely with the NRT Local Site Leaders during the onsite review. While most State Local Site Leaders participate in the quality assurance reviews of completed Onsite Review Instruments and participate in stakeholder interviews, their role on site otherwise varies by review. The NRT Local Site Leader and State Local Site Leader jointly determine the best role for the State Local Site Leader at each site.
- Reviewers: Review team members who conduct case record reviews and case-related interviews at one of the three review sites. Reviewers include the following:
- Federal agency representatives or specially trained consultants with skills and experience in the child welfare field
- State representatives, who are State child welfare agency staff or representatives of the agency's external partners in the CFSP planning process
The Onsite Review Team also may include the following individuals, who supplement the Federal Review Team during the onsite review:
- Children's Bureau Regional Office staff from Regions other than the one responsible for the State being reviewed: Children's Bureau Regional Office staff may participate in reviews outside their own Region for training purposes (at the expense of the Children's Bureau Regional Office) or to provide specialized experience to the review. The Children's Bureau Regional Office forwards requests to have staff participate in a review in another Region to the Children's Bureau Central Office through the Children's Bureau Regional Office responsible for the State that is being reviewed.
- State child welfare staff from States other than the State being reviewed (cross-State participants [CSPs]): States preparing to conduct reviews may find it helpful to send CSPs (especially staff who will serve as the State Team Lead for the review) to reviews in other States (at the expense of the CSP's State) to help prepare and train them for their own reviews. States that are interested in sending a CSP(s) to a review in another State should make CSP requests to the Children's Bureau Regional Office in the Region in which they are located. CSPs must be experienced in child welfare; accountable to the State child welfare agency that they represent on an administrative, policy, or training level; and engaged in assisting the State agency in planning for and managing the CFSR.
The participation of Children's Bureau Regional Office staff from other Regions and CSPs is subject to the approval of the Children's Bureau Central and Regional Offices and the availability of reviewer positions on the review team; CSPs also must participate in training provided through the Child Welfare Review Projects.
In all, the Onsite Review Team comprises approximately 65 people. About half of these are Federal representatives, and half are State representatives: 23 Federal reviewers, 23 State reviewers, 3 Team Leaders (2 Federal representatives and 1 State), and up to 4 Local Site Leaders for each of the 4 teams that operate in the 3 sites. These numbers are subject to adjustment, depending on variations among State reviews, such as the sample size and logistical issues, and the structure of the review team in the State's largest metropolitan site.
In addition, State staff members in each of the three review sites serve as Local Site Coordinators, who are responsible for setting up interviews, making logistical arrangements, and ensuring that case records are available to be reviewed. (Local Site Coordinators are not members of the Onsite Review Team.)
C.2. Functions
C.2.1. Team Leaders
As mentioned above, a designated NRT member leads each onsite review, serving as the NRT Team Leader. A Children's Bureau Regional Office staff person and State child welfare agency staff person serve as the Children's Bureau Regional Office Team Leader and State Team Leader, respectively. Team Leaders have the following responsibilities:
- Serve as liaisons with the State leadership in planning review activities. (The Children's Bureau Regional Office Team Leader assumes primary responsibility for this function.)
- Ensure that the State completes the Statewide Assessment, using the Statewide Assessment Instrument found on the Children's Bureau Web site, and sends it electronically to the Children's Bureau Regional Office 60 days before the onsite review. (The Children's Bureau Regional Office Team Leader assumes primary responsibility for this function, in collaboration with the NRT Team Leader and the Children's Bureau data staff.)
- Provide guidance to the State Team Leader during the Statewide Assessment process, and review and comment on drafts of the Statewide Assessment. (The NRT Team Leader and Children's Bureau Regional Office Team Leader assume primary responsibility for this function.)
- Prepare an analysis (Preliminary Assessment) of the Statewide Assessment 30 days before the onsite review, and record it on the Summary of Findings Form. (The Children's Bureau Regional Office Team Leader prepares this document.)
- Plan the details of the onsite review with the Children's Bureau Central and Regional Offices, State child welfare agency, and Child Welfare Review Projects, including arranging conference calls as needed and transmitting review instructions and procedures to State liaisons. (The Children's Bureau Regional Office Team Leader assumes primary responsibility for this function.)
- Participate in a series of review planning conference calls with the Children's Bureau Central and Regional Office staff, and the Child Welfare Review Projects. Formal review planning calls are held with each State to discuss State data issues, the Statewide Assessment, and onsite review planning functions. The Children's Bureau Central and Regional Office staff also host additional calls with the State to discuss other issues, as needed. (The NRT Team Leader, Children's Bureau Regional Office Team Leader, State Team Leader, and other designated State staff participate in the conference calls.)
- Work in collaboration with the State Team Leader to ensure that the stakeholders to be interviewed at the State and local sites include agency staff who are responsible for, or have firsthand knowledge about, the systemic factors that will be evaluated during the onsite review, such as training and quality assurance. (The Children's Bureau Regional Office Team Leader assumes primary responsibility for this function.)
- Ensure the participation of all State Review Team members in a 1-day training offered by the Children's Bureau via the Child Welfare Review Projects. (The State Team Leader assumes primary responsibility for this function.)
- Work with the Child Welfare Review Projects to ensure that all documents needed for the review are sent to the NRT Local Site Leaders at a predetermined location. (The Children's Bureau Regional Office Team Leader assumes primary responsibility for this function.)
- Ensure that the Local Site Leaders are aware of their responsibilities during the onsite review; see the Local Site Leader responsibilities below. (The NRT Team Leader assumes primary responsibility for this function.)
- Conduct State-level stakeholder interviews during the onsite review. (The NRT Team Leader, Children's Bureau Regional Office Team Leader, and State Team Leader jointly participate in interviews.)
- Perform and/or oversee a statewide quality assurance review of all completed Onsite Review Instruments to identify missing information and/or inconsistencies in completing the Instruments and ensure that the items and outcomes are rated correctly. (The NRT Team Leader, Children's Bureau Regional Office Team Leader, and State Team Leader participate in quality assurance reviews of Instruments.)
- Provide leadership for the statewide exit conference that is held during the onsite review. (The NRT Team Leader assumes primary responsibility for this function; see chapter 4, section F, for information on exit conferences.)
- Identify possible promising child welfare approaches (See chapter 4 section E, for information on promising approaches.) (The NRT Team Leader and Children's Bureau Regional Office Team Leader share this responsibility.)
- Serve as the primary liaison with the Federal contractor responsible for preparing the initial draft of the Final Report, review and comment on drafts, rate systemic factors and indicators, determine whether all ratings in the report are accurate, transmit a courtesy copy of the Final Report to the State, and secure needed Children's Bureau Central and Regional Office approval of the report. The Children's Bureau Regional Office transmits the official copy to the State and other required parties. (See chapter 6, section D, for information on the distribution of the Final Report.) (The NRT Team Leader and Children's Bureau Regional Office Team Leader assume responsibility for this function.)
- Act as the liaison between the Children's Bureau Central and Regional Offices and State in the event of discrepancies between information in the Statewide Assessment and the onsite review findings, and work toward resolution of the discrepancies. (The NRT Team Leader and Children's Bureau Regional Office Team Leader work collaboratively on this function; see chapter 5, section C, for more information on resolving discrepancies.)
- Work with designated State staff to develop the PIP. (The Children's Bureau Regional Office Team Leader assumes primary responsibility for this function, in collaboration with the Children's Bureau Central Office.)
C.2.2. Local Site Leaders
The Onsite Review Team is divided into local teams that are assigned to the three review sites. A designated NRT member leads each local team, serving as the NRT Local Site Leader. Children's Bureau Regional Office staff and a specially trained consultant may serve as Co-Local Site Leaders with the Federal Local Site Leader, and a State child welfare agency staff person serves as the State Local Site Leader. Local Site Leaders have the following responsibilities:
- Participate, as asked, in a series of review planning conference calls with the Children's Bureau Central and Regional Office staff. (The NRT Local Site Leader, Federal Local Site Leaders, and State Local Site Leader participate in this function.)
- Conduct the local entrance conference to provide an overview of the review week and highlight State-specific issues before beginning review activities, and conduct the local exit conference to share preliminary findings. (The NRT Local Site Leader assumes primary responsibility for this function.)
- Conduct stakeholder interviews at the local site. (The NRT Local Site Leader, Federal Local Site Leaders, and State Local Site Leader share this responsibility.)
- Provide strong, positive leadership to the team by setting the pace of work for the week, assisting members in resolving problems with their schedules or in reviewing their cases, supporting the team in completing its work, and promoting a positive and objective approach to the review. (The NRT Local Site Leader, Federal Local Site Leaders, and State Local Site Leader share this responsibility.)
- Coordinate the review schedule with local agency staff, including ensuring that the Local Site Coordinator: (1) confirms interview arrangements and (2) adjusts schedules, including those for local entrance and exit conferences, as needed. (The State Local Site Leader assumes primary responsibility for this function, in collaboration with the NRT Local Site Leader and Federal Local Site Leaders.)
- Provide leadership for any scheduled focus groups, open forums, or discussion groups arranged as part of the local review week, such as facilitating discussions, debriefing teams, summarizing findings, and presenting findings at the local exit conference. (The NRT Local Site Leader assumes primary responsibility for this function.)
- Coordinate team members' responsibilities for case record reviews, case-related and stakeholder interviews, and other review activities at the local sites, distributing the workload to ensure that all tasks are accomplished on time. (The NRT Local Site Leader, Federal Local Site Leaders, and State Local Site Leader share this responsibility.)
- Assist reviewers in resolving issues pertaining to completing the Onsite Review Instruments and gathering information for the Instruments. (The NRT Local Site Leader, Federal Local Site Leaders, and State Local Site Leader share this responsibility.)
- Identify possible promising child welfare approaches. (See chapter 4, section E, for information on promising approaches.) (The NRT Local Site Leader and Federal Local Site Leaders assume responsibility for this function.)
- Perform quality assurance reviews of all Onsite Review Instruments, as they are completed, to identify missing information and inconsistencies in completing the Instruments and ensure that the items and outcomes are correctly rated. In addition, perform second-level quality assurance reviews of Instruments from other review sites, as requested by the NRT Team Leader. (The NRT Local Site Leader, Federal Local Site Leaders, and State Local Site Leader participate in this function.)
- Convene and lead daily debriefings with local team members during the onsite review. (The NRT Local Site Leader assumes primary responsibility for this function.)
- Receive and screen case-related concerns that reviewers may have regarding child safety or the inclusion of the case in the sample, and bring those concerns to the NRT Team Leader and an appropriate staff member of the local child welfare agency. (The NRT Local Site Leader, Federal Local Site Leaders, and State Local Site Leader share responsibility for this function.)
- Coordinate the completion of the Summary of Findings Form for the local team at the end of the review week, including providing all local team members with opportunities for input into the document. (The NRT Local Site Leader assumes primary responsibility for this function.)
- Ensure that team members complete all assigned review functions, including completing and submitting all review Instruments, before departing the review site. (The NRT Local Site Leader, Federal Local Site Leaders, and State Local Site Leader share this responsibility.)
- Submit local review team members' completed Onsite Review Instruments and Stakeholder Interview Guides, and the team's Summary of Findings Form, electronically to the designated Team Leader before the statewide exit conference. (The NRT Local Site Leader assumes primary responsibility for this function.)
- Ensure that a process for shredding all case-identifying materials is in place and used by reviewers before leaving the State. (The State Local Site Leader assumes primary responsibility for this function.)
C.2.3. Reviewers
Onsite Review Team members have the following responsibilities:
- Participate in scheduled orientation or training sessions before the review. Children's Bureau Regional Office staff participate in trainings provided by the NRT. Consultants and CSPs attend a training conducted by the Child Welfare Review Projects. State members of the Onsite Review Team attend a 1-day training about the onsite review, which is provided by the Child Welfare Review Projects approximately 2 weeks before the review.
- Review the completed Statewide Assessment and Preliminary Assessment, and other materials provided by the Children's Bureau Regional Office through the Child Welfare Review Projects, in preparation for the onsite review.
- Remain present and free of other responsibilities at the review site for the entire week of the onsite review, including participating in all scheduled review activities, from the entrance conference through the final local site debriefing held on Thursday, unless otherwise instructed by the NRT Local Site Leader.
- Conduct all assigned activities associated with reviewing case records and conducting case-related interviews, including completing the Onsite Review Instruments.
- Work closely with Federal or State partners in making decisions regarding distributing the workload and completing the review instruments. (Case record reviews are conducted by pairs of reviewers, comprising one Federal Review Team member and one State Review Team member.)
- Participate, as requested by the Local Site Leaders, in focus groups, open forums, or other group meetings scheduled during the onsite review.
- Attend daily debriefings of the local review teams, and present information on cases reviewed.
- Notify the Local Site Leaders about previous casework or supervisory responsibility for a case being reviewed by any member of the team, and refrain from attending debriefing sessions when the case is debriefed.
- Notify the Local Site Leaders if a child's safety might be at risk.
- Bring to the attention of the Local Site Leaders any cases that appear to be in the sample in error; for example, a foster care case in which the child and family actually were receiving in-home services.
- Assist in compiling a summary of the team's findings during the onsite review.
- Be alert for promising child welfare approaches during case record reviews and case-related interviews, and inform the designated Local Site Leader about promising approaches identified. At the final review team debriefing at each local site, the NRT Local Site Leader facilitates a discussion of promising approaches that the review team identified.
- Submit completed review instruments on assigned cases to the designated Local Site Leader before departing the review site at the end of the onsite review.
- Remain available for telephone consultation following the review, if needed, to clarify or supplement information recorded on the instruments.
C.2.4. Local Site Coordinator
The State assigns a Local Site Coordinator to manage the logistical arrangements for each of the three review sites. These Local Site Coordinators are State staff members whose functions are different from those of the State Local Site Leader; the Local Site Coordinators are not members of the Onsite Review Team, but are the review team's liaison to the child welfare agency at the review sites.
The Local Site Coordinators have the following responsibilities:
- Orient local child welfare agency staff about the review.
- Manage the process for selecting cases at the local site.
- Schedule review week activities, including the local entrance and exit conferences and local stakeholder interviews, and prepare reviewer schedules.
- Plan and manage the review week logistics, such as booking sleeping rooms for State Review Team members and arranging transportation to and from interviews; and coordinate with the Child Welfare Review Projects regarding logistical arrangements.
- Reserve space for all onsite review activities, including meetings, case record reviews, debriefings, and some interviews, as necessary.
- Assemble the case records selected for review.
- Handle the rescheduling of interviews as necessary during the onsite review.
- Provide general support to the Onsite Review Team.
D. PIP Development Team
D.1. Structure
The State should consider which individuals will serve on the PIP Development Team, as necessary, when composing its Statewide Assessment and Onsite Review Teams. This will enable the State to begin focusing on the program improvement process during the two phases of the review process. Moreover, the PIP Development Team then will comprise individuals who have experience with, and knowledge about, the overall review process and findings.
The following are suggested participants on the PIP Development Team:
- Administrators and program specialists from the State and local child welfare agencies
- State and local quality assurance staff
- Local child welfare agency staff with front-line supervisory and practice experience
- Representatives of major domains outside child welfare that will need to be addressed through the PIP, including the courts, law enforcement, education, health, mental health, substance abuse, domestic violence, and juvenile justice
- Researchers and evaluators, who have a child welfare focus, from within State government and/or on loan from a local university
- Tribal representatives
- Advocacy groups and consumer representatives, including children and youth in foster care or the groups that represent them
- Legislative and policy personnel who focus on child welfare issues and/or funding
D.2. Functions
PIP Development Team members have the following responsibilities:
- Participate in training or orientation sessions for the PIP Development Team members.
- Stay apprised of the review process.
- Attend meetings related to the PIP process.
- Analyze the Final Report to determine areas needing improvement that will need to be addressed in the PIP (see chapters 6 and 7 regarding the Final Report and PIP process, respectively).
- Assist in developing and implementing the PIP, including planning the PIP evaluation process and benchmarks and monitoring PIP progress, as appropriate.
1 The Child Welfare Review Projects supports the Children's Bureau in administering the CFSRs. back
Return to Table of Contents