What is an ANA impact visit?

Why does ANA conduct impact visits?

The Native American Programs Act (NAPA), which launched the Administration for Native Americans, mandates ANA to assess the impact and the effectiveness of the projects it funds.. The first full-scale exploration of ANA’s portfolio occurred in 2006; since then, ANA has visited over 480 projects.

How many visits does ANA do each year?

ANA visits approximately 70% of all ending projects each year, which amounts to roughly 70-80 projects. All visits are conducted by an Impact Evaluator and one of our Training and Technical Assistance Providers. Sites are chosen based on a variety of variables, including geographic distribution, funding amount, grant area, and frequency of past visits. An impact evaluator can visit up to 4 sites a week, which unfortunately does not allow much flexibility for the date of the visit.

What is the process for an impact evaluation?

About 45 days prior to the visit, a notification letter is mailed to the authorizing official and scanned and emailed to the project director which indicates the date of the visit. Once the date is confirmed with the project director, an outline of the project assessment tool is emailed to the project director. The outline indicates the specific information that will be gathered on the date of the visit including challenges faced, best practices and lessons learned. A week before the visit the impact evaluator will call the project manager to confirm the date, time and place of the meeting and answer any questions the grantee may have.

What information is collected during the evaluation?

The impact visit is an opportunity to share the story of the project, including what worked and what did not work so well. The impact evaluation does not review activities in the OWP, since the evaluator has read the quarterly reports prior to the visit. The evaluation will concentrate on the Results and Benefits Expected section and the Criteria for Evaluating Results and Benefits section of the OWP. There is also an opportunity for the grantee to provide feedback on how well, or not well, ANA supported the implementation of the project.

Grantees tend to be a bit nervous when ANA first arrives in their communities and/or their project. But across the board, ANA has heard that the evaluation process is a positive experience for grantees. Grantees are working hard to accomplish their project and really enjoy the opportunity to sit back and reflect on their achievements.

How long can we expect the visit to last?

Visits usually last around 6-8 hours.

What does ANA do with the information that is collected?

Most importantly, ANA compiles the information collected from each grantee to report to Congress on what ANA funded projects are accomplishing – both qualitatively and quantitatively. Doing so meets ANA’s mandate and encourages Congress to continue funding the Administration for Native Americans. The information is also used to make changes to internal policies and procedures, including the quarterly reporting process.

How do we prepare for an impact visit?

Objective Work Plan (OWP)

For this section, you should be prepared to discuss your OWP and any methods used to track your ability to meet your project objectives.

Project Management

ANA would like to learn about staff and consultants hired to work on the project. The evaluator will also ask questions relating to the project’s financial management.

• Identify staff and consultants who worked on the project and the total number of hours/weeks worked. For consultants, please have total hours worked available, the dollar amount paid (per hour and in total), and the reason for hire.
• The person who handled the project’s finances will be asked several questions, including the tracking process for the non-federal share of the project, questions about the drawing down of funds, and inquiries regarding budget modifications.
• You will be asked about the record-keeping of the non-federal 20% contribution to the project, budget modifications, and ANA funds left over at the end of the project.

Community Involvement and Outreach

ANA appreciates input from beneficiaries and community members about their involvement in the planning and implementation of the project.

• Review any marketing or outreach materials developed for the project.
• You will be asked about the degree of community involvement in the planning and implementation stages of the project; the number of elders and youth involved, and the extent of their participation; and the type of activities and events held to share the project with the community.

Project Results and Impact

ANA is interested in learning about the results and impact the project has had on the community.

• Program revenue generated by the project (including supporting documentation).
• Jobs created by the project (including supporting documentation) as well as how the positions were paid (e.g. ANA funds, project-generated revenue, leveraged funds). This will include the types of positions and the total hours/weeks worked.
• Businesses created by the project (including supporting documentation).
• Job-related training resulting from the project. This will include the course/workshop titles, hours needed to successfully complete the training, and the number of people who successfully completed training.
• Leveraged Resources (including any supporting documentation). This includes resources mobilized/attracted to the project (above and beyond the 20% non-federal match) as well as their dollar value.
• Partnerships formed as a result of the project.
• Discussions of project impact with project beneficiaries, and how the project has improved the social, economic, and/or environmental conditions of the community.

Grant-Specific Questions

ANA will ask questions regarding your specific type of project.

• Products developed, surveys conducted, codes and/or ordinances developed, language teachers trained, etc.

ANA Services

ANA is hoping you will share any suggestions, concerns or comments that will help ANA better serve you.