Increasing the Capacity of Individuals, Families and Communities

Outcomes Measurement

Data Collection Methods | Summary

Creating a Plan for Your Outcome Measurement System

Page:  1 | 2

Data Collection Procedures and Planning
To finish your planning work, a number of procedural issues need to be considered. First, who will collect the data? How will you recruit these data collectors? What steps will they need to take to collect the data? How will you train them? Finally, who will be responsible for monitoring the data collection process to ensure you are getting what you need? It’s important to answer each of these questions during your planning. You don’t want to be surprised halfway through the process to discover your three-month follow-up surveys were not mailed out because you didn’t identify who would do so!

Have you prepared your clients (the FBOs and CBOs you work with) for data collection? Do they know that you will be collecting data, either at the time of service or in follow-up? Do they understand why it is important to you and how you intend to use the data? Organizations often have outcome reporting requirements themselves, so they usually are responsive if they have been alerted to your needs ahead of time. Advising them in advance about your data collection plans will help increase their willingness to participate during implementation.

Finally, when you put together your outcomes, indicators, data collection methods, frequency and schedule of data collection as well as your procedures, you will have created your plan for outcome measurement. Worksheets to assist you with your evaluation planning are in the Appendix.

Outcome Measurement Plan OUTCOME INDICATOR WHAT DATA COLLECTION METHOD? WHEN WILL DATA BE COLLECTED? WHO WILL COLLECT DATA? WHAT WILL THEY DO? HOW WILL DATA COLLECTED BE MONITORED? Increased ability of FBCOs to raise funds for program services # and % of organizations who put new fundraising practices in place Survey of organizations receiving fundraising technical assistance Once: three months after completion of technical assistance services Technical assistance staff Mail survey to organizations 3 months after the completion of technical assistance services. Send reminder postcards to organizations who do not return the survey within two weeks. Program manager will track completion of services, mailing of surveys and response rate to the survey and notify staff if additional follow-up is needed Increased ability of FBCOs to raise funds for program services # and % of organizations whose budgets demonstrate new $ raised Record review of budget information of organizations receiving fundraising technical assistance Twice: during technical assistance and at the beginning of the year following technical assistance Technical assistance staff Request submittal of prior year’s budget during technical assistance and subsequent year’s budget at beginning of the year after TA Program manager will track receipt of budgets and notify staff if follow-up is needed

Confidentiality and Informed Consent
Anonymous and confidential do not mean the same thing. Anonymous means you do not know who provided the responses. Confidential means you know or can find out who provided the responses, but you are committed to keeping the information to yourself.

You must ensure that you protect the confidentiality of any individual’s data or comment. It is easy to make your surveys anonymous, but if you want to track people over time, you’ll likely need to attach ID numbers to each person from whom you collect, keeping a list of the names and numbers in a locked file.

It is important to inform people that you are measuring your program’s outcomes and may use data they provide in some way. You must let them know that their participation is voluntary and explain how you will maintain the confidentiality of their data.⁹

Planning Checklist 10 Is the data collection method (survey, interview, observation, internal records, official records) appropriate for the outcomes and indicators? Does the plan rely on external sources of data or require collaboration with other agencies? If so, will it be possible to get the data? Have the tools/questions been pre-tested? Are the measurement approaches and tools culturally relevant? Can the approach/tools likely be implemented within available program resources? Does the data collection schedule include time point(s) that follow completion of services or program activities? Does the frequency of data collection match time points when realistic progress can be expected from participants? Does the frequency of data collection match the level of contact with participants? (e.g., greater frequency with greater level of contact) Are the roles and responsibilities clear for all the staff involved in collecting data? Is there a staff person (or staff team) responsible for managing and monitoring the process to assure the work is completed on time and is of high quality?

Page:   1 | 2

Data Collection Methods | Summary