Introduction
Research Questions
- How do goals fit into the pathway toward self-sufficiency?
- Why should employment programs measure goal-related skills, behaviors, mindsets, and outcomes?
- What should programs measure, and how and when should programs collect data?
Employment programs are inherently goal-oriented. They seek to help participants achieve self-sufficiency, typically through education; work readiness and job training; job search assistance or requirements; supportive services (such as assistance with child care and transportation); and access to employment opportunities. Programs have been working to improve participants’ employment outcomes for decades, but the field continues to seek greater effectiveness. New evidence from neuroscience, psychology, and other behavioral sciences suggests that employment programs may be able to improve participants’ long-term outcomes by enhancing their ability to set and pursue their own goals, and that specific skills, behaviors, and mindsets related to self-regulation are critical to goal achievement.
Some employment programs are beginning to offer services that explicitly seek to improve participants’ ability to set and pursue their own goals. Determining whether such services are succeeding can be challenging. Even when programs compare employment outcomes of their participants with outcomes of a comparable group of non-participants, they often lack valuable information about whether participants are reaching more intermediate goals, who is benefiting most from which services, and whether and how services are changing participants’ goal-related skills, behaviors, or mindsets. More directly measuring the skills, behaviors, and mindsets needed for goal achievement can help programs track critical steps participants may be taking toward self-sufficiency and better assess the promise of goal-oriented employment services for adults with low incomes.
Purpose
As programs begin investing effort in boosting participants’ self-regulation and goal achievement, they will likely want to determine whether their efforts are effective. This brief provides guidance on how employment and human services programs can measure self-regulation and goal achievement. It describes considerations for selecting measures and deciding when and with whom they might be used. Depending on the types of measures chosen, data on participants’ self-regulation and goal achievement can inform ongoing program improvements and provide information and evidence about goal-oriented programming.
Key Findings and Highlights
Highlights from this brief include: (1) Programs can support participants’ self-regulation skills, behaviors, and mindsets that are central to the goal achievement process. Depending on a program’s focus, measurement could focus on skills, behaviors, mindsets, or the goal achievement process; (2) Measuring goal-related skills, behaviors, mindsets, and outcomes offers programs the opportunity to learn more about their participants, monitor program performance, show who might benefit most from an intervention, and guide program improvement. In the context of a rigorous experiment, it can help programs assess the effectiveness of their interventions; (3) Deciding what to measure and how and when to collect data depends on what the program is trying to accomplish and how and what it is interested in learning. A program theory of change can help answer these questions. A variety of measurement tools exist (including self-and observer-report surveys). Administrative data collected during normal program activities can also provide useful information about self-regulation skills, behaviors and mindsets. (4) To prepare for measurement, programs should facilitate staff buy-in; delineate responsibilities (for instance, who will collect and analyze the data); and consider the audiences for measurement results. This brief also offers references to additional resources for learning more about particular measures, and about measurement more generally.
Methods
This brief is informed by research conducted under the Goal-Oriented Adult Learning for Self-Sufficiency (GOALS) project, funded by the Office of Planning, Research, and Evaluation (OPRE) at the U.S. Department of Health and Human Services, Administration for Children and Families (ACF). Activities that informed this brief include: (1) a synthesis of the literature on the relationship between self-regulation, goal attainment, and the environment and on how programs have been or could be adapted to promote goal attainment; (2) exploratory site visits to programs implementing interventions to improve participants’ self-regulation and goal achievement; and (3) assessment of the implementation experiences of several goal-oriented pilot interventions.
Citation
Cavadel, Elizabeth W., Jacqueline F. Kauff, Ann Person, and Talia Kahn-Kravis (2018). New Perspectives on Practice: A Guide to Measuring Self-Regulation and Goal-Related Outcomes in Employment Programs, OPRE Report #2018-37, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- Self-regulation:
- The ability to control actions, thoughts, and emotions .