Using Data-driven Reflection to Improve Program Quality: New York City’s Human Resources Administration Redesigns Its Upfront Assessment Process for Youth and Families Receiving Cash Assistance

The purpose of this brief is to describe how the NYC HRA used evidence-based decision making and a data-driven reflection process to systematically test and refine their upfront assessment process for youth and families receiving TANF. This brief highlights lessons learned through this process and aims to help TANF and other human services agencies generate ideas of how to use data-driven reflection (testing) to make decisions and continuously improve their programs.

In laying the groundwork for the Improve phase, HRA gathered input from the agencies that they contract with to provide employment and training services (providers) during the Learn phase and identified several problems with the current assessment, including the extensive length, the transactional tone it set for the relationship between staff and participants, and the limited use of the findings for case planning. Next, during the Innovate phase, HRA and contracted service provider leaders and staff adapted an existing assessment tool and process used in other programs to develop a new tool called “Stepping Stones to Success.”

In the Improve phase, HRA designed a road test, implemented the Stepping Stones assessment tool and process, and executed the road test. After executing the road test, they sought input from providers to make meaning of the findings and use those findings to make evidence-driven decisions. Overall, staff and participants liked Stepping Stones, sharing that it was simple, easy, and enjoyable to use. Staff said it helped them be more participant-centered and goal-driven during the assessment process. Participants said that it helped them focus on and feel excited about their goals.

Reflecting on the overall data-driven reflection process (Improve phase), HRA shared the following lessons:

• Designing and implementing a successful test required teams to consistently collaborate
• Create the structure and analytic tools to make testing a change feel doable
• Create a culture that inspires growth and learning within the organization

HRA conducted a four-week pilot test of the Stepping Stones assessment tool between May and June 2021 with 12 provider staff—two staff members from each of the six provider agencies. Each staff member used Stepping Stones with at least five participants per week. Data collection included three primary activities:

• Staff completed a brief four-item online survey after each exchange with participants about their experiences completing the revised assessment tool and process (total of 188 surveys)
• Participants completed a brief four-item online survey after each exchange with their case manager about their experiences completing the revised assessment tool and process (total of 121 surveys)
• Supervisors at each of the six provider agencies conducted 30-minute focus groups during the second and fourth weeks of the road test for a total of 12 provider focus groups

Derr, Michelle K., Erin Welch, Angela Ghesquiere, Edith Kealey, Dina Zagari-LiMandri, Lyndsey Richardson, Genesis Reyes, Samantha Palermo, Alexandra Stanczyk, Joseph O’Brien, Brittany Tabora (2022). Using Data-driven Reflection to Improve Program Quality: New York City’s Human Resources Administration Redesigns Its Upfront Assessment Process for Youth and Families Receiving Cash Assistance, OPRE Report #2022-66, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.