Increasing the Capacity of Individuals, Families and Communities

Outcomes Measurement

Data Collection Methods | Summary

Creating a Plan for Your Outcome Measurement System

Page:  1 | 2

Once you have identified the data collection methods you intend to use, you need to decide when you will collect the data and how often. Then consider the procedures you need to put in place to ensure your outcome measurement system is sustainable and produces quality data.

Frequency and Scheduling of Data Collection
The table below describes the five approaches or designs you are likely to use for your data collection.

Data Collection Designs Type of Design Collection Frequency Example Post-only Measures Data are collected once: at the end of the program, service or activity Level of participant knowledge on a survey after a training workshop Pre/Post Measures Data are collected twice: at the beginning to establish a baseline and the end of the program Comparison of an organization’s documented fundraising success before and after receiving technical assistance Time Series Data are collected a number of times: during an ongoing program and in follow-up Monthly observations of an organization’s collaboration meetings to track changes in partnership development and communication Measures with a Comparison Group Data are collected from two groups: one group that receives the intervention and one that doesn’t Comparison of data on skill development from individuals who participated in training and those who have not yet taken your workshop Measures with a Comparative Standard Data are collected once: at the end of the program, service or activity and are compared with a standard Comparison of this year’s data on organizations’ success in fundraising as compared to last year’s data (Did it benefit the community where the client was located?)

The first three approaches in the table are fairly self-explanatory; the last two approaches need a little discussion. Comparison groups can be very useful in demonstrating the success of your intervention. The main question is can you find a group of people or organizations that is just like the group with whom you are working? In order to provide a valid comparison, the two groups must have the same general characteristics. A similar group may be difficult to find. However, if you are working with different groups at different times and the groups are similar, this approach may work for you. Or if, as the example of using a comparison group shown above, you have people on a waiting list for training who are similar to the people who have trained already, they may make a good comparison group. Or perhaps you might compare FBCOs receiving sub-awards with those who applied but did not receive them.

Comparative standards are standards against which you can measure yourself. There are standards of success in some fields (e.g., health mortality and morbidity rates, student achievement scores, teen birth rates). For intermediaries, however, there are unlikely to be many regarding your program outcomes or indicators. You can, however, compare your results for one time period to an earlier one, as shown in the example of using comparative standards in the prior table. You collect data for the first time period as your baseline and use it as your standard in the future.

When considering which approach is best, these questions may help you make a decision:

Checklist on Data Collection When would change occur—immediately, gradually or over a long period of time? Are there milestones that can be measured along the way to the outcome you are trying to achieve? What is the frequency of contact with the organizations with which you are working—once, weekly, monthly or at some other interval? When will data be available? (When are some of your internal records available)? Are there any groups that might serve as comparison groups? Do you have baseline data you can use as a standard? Are you accountable for short-term outcomes or longer-term impacts?

Page:  1 | 2

Data Collection Methods | Summary