Outcomes Measurement
Logic Models
and Program Theory |
Data Collection Methods
Performance Indicators
Page: 1 | 2Creating Appropriate Indicators
- Attempt to "define" the characteristics of an outcome
as a way of identifying possible indicators. For example, the
characteristics of "increased fund development capabilities"
could include the amount of money obtained in additional support
and the diversity of those funding sources. The specific measures
related to these two characteristics for "increased fundraising
ability" might be: 1) the number and percent of FBCOs
who raise additional funds this year as compared to last year
and 2) the number and percent of FBCOs who show an increase
in the number of sources of funding for their programs.
- Look for indicators that are "indicative" of an outcome
rather than a predictor or a result of an outcome. If the relationship
between an outcome and its proposed indicator sounds like an
"If…Then" statement, then it is probably not
the right indicator for that outcome.
For example, "If" an organization attends grant writing training, "Then" it is more likely to bring in additional grant funding. In this example, attending grant writing training is not an indicator for increased fund development capabilities but may rather be a predictor of increased success. A more indicative indicator of increased grant funding would be "the number and percent of organizations whose budgets show an increase in the number of grants and/or an increase in the amount of support from grants."
- Apply the "means that" rule in assessing your indicators. In theory, the accomplishment of an indicator "means that" you have achieved an outcome. For example, if an organization has completed, submitted and obtained approval for 501(c)(3) status (indicator), it "means that" it has a stronger organizational structure (outcome). In contrast, an organization having an expanded service area for more clients (indicator) does not mean the organization has improved its coordination and/or collaboration in service with others (outcome).
One to three indicators is usually a realistic number for each outcome you have identified. Some straightforward outcomes can be quantified easily through the use of only one indicator. Other more complex outcomes will necessitate two or three.
|
||||||
In the second example, one indicator alone will not provide sufficient information. As you look at what indicators you need to adequately describe your progress in achieving your intended outcomes, it’s important to use the fewest number of outcomes possible. It takes time and money to gather and analyze the data for each one. What’s important is not quantity but quality. What’s the best way to see, hear or read about the change?
Here are some things to consider as you prioritize your indicators:
- Select indicators for which you can get the data
- Select the indicator that is most useful to you and gives you the most useful information about the outcome
- Select the indicator that is most feasible for staff to measure within the time and money resources available to you
When developing indicators, you may wonder about setting a specific level of achievement. This is called a "target." In addition, you may be interested in comparing your performance on the indicator to other organizations or time periods. This approach is known as a "comparative standard." Here is how outcomes, targets and comparative standards fit together in outcome measurement.
Outcomes statements: describe an increase, decrease, maintaining, etc. of a behavior, skill, knowledge, etc.
| Example: | Improve fund development capabilities |
Target statements: specify a particular level of achievement related to the outcome
| Example: | Raise $20,000 in foundation funds during 2004 |
Comparative statements: include comparative targets, usually related to standards, other time periods or organizations
| Example: | Increase the organization’s funding from non-governmental sources by 30% as compared to last year |
Targets and comparisons will be useful to you over time as you gain a better sense of where your client organizations are at the beginning of your technical assistance process. It’s important to establish a baseline for the indicators you plan to measure over time. A baseline is information collected to give you a starting place (something to measure against in the future). A baseline will allow you to set reasonable targets and measure the FBCOs’ progress against them. For organizations that have not collected data on indicators in the past, the first round of data collection will establish the baseline.
|
|
Similarly, comparisons are useful when you want to measure your performance against something—either another time period, another organization or an established set of standards. Although there are comparative standards in some fields (health, for example) where infant mortality rates and other morbidity rates have been established nationally, there are few standards in human services that organizations can use for comparison purposes. Some national human service organizations are beginning to establish standards specific to their own programs. More likely, however, you will use your previous performance as a standard, comparing your success last year to this year or this year to next year. | ||||||||||||||
| At the outset, it’s important to stick to the basics of outcome measurement—defining outcomes and the indicators that make them measurable. You’ll have time to move on to targets and comparisons later. A worksheet to assist you with identifying indicators is in the Appendix. | ||||||||||||||||
Page: 1 | 2
Logic
Models and Program Theory |
Data Collection Methods