Identify and hire more high-performing Child Support Enforcement staff.

Child Support Enforcement (CSE) employed a process known as Strategic Assessment Research (STAR), developed by the Georgia Merit System, to evaluate and develop selection processes to hire CSE Agents. CSE developed strategic measures to measure the performance of CSE Agents, to identify high performers, to develop a competency profile of high performance, and to develop and validate new selection processes to hire high performers (i.e., CSE Agents who achieve CSE objectives). The new selection process include a reading and arithmetic test and a competency-based behavioral interview. CSE evaluated this new approach against the effectiveness of the three selection processes previously used by CSE including: 1) a written, multiple-choice, job skills and knowledge test, 2) a video-based test of customer service and interpersonal skills, and 3) an unstructured process applied at the discretion of each hiring manager.

The profile of successful CSE Agent performance was identified as the who, what, why and how, of the high performing CSE staff. Through intensive interviews, observation, and other assessments, it was determined how successful CSE Agents differed from those who were less successful. Differences in competencies (i.e., behaviors, actions, strategies, knowledge, thoughts, motives, values, and self-concepts) were measured. This information was used to develop a competency profile of high performance to recruit, identify, select, train, coach, and manage CSE Agents to achieve strategic objectives. Using STAR, processes were developed to hire new CSE Agents who approach their work in a manner similar to that known to characterize high performers.

The project also examined program factors such as caseload size, caseload specialization, process activities, and administrative procedures to determine their effects upon outcomes achieved by CSE Agents. These factors were evaluated in terms of their effectiveness in achieving agency objectives, cost-benefit, and return on investment.

CSE Agents collected child support payments ranging from $172,107 to $1,677,705. The average amount was $802,237 per CSE agent. Performers in the top 13.5% collected at least $1,143.673. That is 42% more than average. Conversely, low performers, those in the bottom $13.5% collected 42%, or $341,436, less than average.

We compared the three previously-used selection processes and found that Agents hired using the video-based test of interpersonal and customer service skills averaged $194,998 more in collections compared to those hired by the two other methods. In two years, that resulted in over $21,839,776 more collected for Georgia's children. Approximately 40% of that amount would have been paid out in welfare payments had the money not been collected. In only two years, this test saved Georgia over $8,735,910 TANF payments.

We verified that the newly developed reading and arithmetic test accurately predicts CSE Agent performance. We obtained a .45 correlation with collections and a .47 correlation with supervisors' ranking of the CSE Agents' contribution to achieving strategic objectives. When combined with the video-based test, the multiple correlation is .70, an extremely strong prediction of who will be good at the job. Plans are underway to re-introduce the video-based test in combination with the new test.

We constructed a competency-based behavioral interview using the profile of high performance developed through the STAR process. We piloted the interview protocol and found it to have an 80% accuracy rate in identifying top performers. All supervisors have been trained to hire new Agents using the written test and the interview process. About 100 CSE Agents will be hired in the following year. Because top performers collect 42% more than average agents, and because the interview process has a proven 80% accuracy rate, we expect this performance-based selection strategy to translate into $23,900,520 annually in increased collections and salary savings. We will evaluate the actual effects upon collections next summer.

We used the performance measures to show that CSE Agent turnover in one year cost the state over $70,948,400 due to a combination of lost productivity when positions are vacant and while new hires are being trained, reduced productivity due to disruption of team activities prior to departure, and salary costs due to wasted time and resources of these people.

As part of the STARS process, we also discovered that a software system designed to automate computations related to collecting child support payments was faulty and agents spent an average 18% of their time doing manual calculations. The faulty software has been repaired, freeing CSE Agents to increase collections by up to $28,880,532 per year.

We also compared caseload size to average collections per case, percentage of paying cases, and percent of current cases and found that each of these outcomes were maximized at caseload sizes between 600 and 799 cases. In this range, collections per case are 21% greater, the percent of active cases is 18% greater, and the percent of current cases is 39% greater compared to caseloads with 800 or more.

The total cost for the research and the development of this customized selection process was less than $160,000, consisting of analysts' and subject matter experts' salaries and travel expenses. It was determined that if the STAR process was only half as successful as is it appears, the benefits of the selection process will exceed $62 million per year.

Georgia STAR was developed and implemented entirely with in-house state staff.

This program was supported by the operating budgets of the Georgia Division of Child Support Enforcement and the Georgia Merit System, funded by the Georgia legislature. Additional matching funds were provided by the Department of Health and Human Services, Office of Child Support Enforcement, Administration for Children and Families, Office of State and Local Assistance.

Performance measurement begins with understanding the organization's mission and strategic objectives. If these are not clear, then it is not clear what you are trying to achieve or predict. Take steps to clarify and understand the organization's mission, strategic plan, and strategic objectives. Use these to develop strategic measures to determine whether the objectives - the results your organization is trying to achieve - are being accomplished. Focus on efficiency, effectiveness, outcomes and results. Do not worry about input and process measures unless the other types of measures are impossible to obtain.

In defining performance objectives, begin with the top level of the organization. Then, cascade the objectives and measures down to each successive organizational level proceeding down to the level of work teams and individual contributors. Done correctly, this cascading process takes about four months.

The following concepts and sources were used:

Performance measures developed should support overall organizational objectives. Choose measures carefully since they will be what people do on the job. Wrong measures will lead to wrong results. In order to determine how high performers go about doing their jobs, it is first necessary to identify high performers, and this in turn depends completely upon the measures that are used. When it comes to program and process evaluation, including selection processes, "you get what you measure." It is critical that the performance measurement step is done correctly.

Charles Brooks, Georgia Merit System Workforce Planning & Best Practices, Suite 418, West Tower, 200 Piedmont Avenue, Atlanta, Georgia 30334. Telephone: (404) 725-1347 Fax (404) 657-4958; E-mail: bro@gms.state.ga.us. or stoneyb@mindspring.com