< Back to Search

What is an ANA impact visit?

Published: April 9, 2012
Audience:
All
Topics:
Grantee Resources, Performance and Accountability
Types:
Informational Sheet
Tags:
Impact Evaluation

Why does ANA conduct impact visits?

The Native American Programs Act (NAPA), which launched the Administration for Native Americans, mandates ANA to assess the impact and the effectiveness of our projects.  The first full-scale exploration of ANA’s portfolio occurred in 2006; since then, we have visited over 540 projects.

How many visits does ANA do each year?

ANA visits approximately 70% of all ending projects each year, which amounts to roughly 70-80 projects annually.  Sites are chosen based on a variety of variables, including geographic distribution, funding amount, grant area, and frequency of past visits.  An ANA staff member can visit up to four sites a week, which unfortunately does not allow much flexibility for the date of the visit.

What is the process for an impact evaluation?

About 45 days prior to the visit, ANA will mail a notification letter to the authorizing official, and email the letter to the project director.  The notification letter indicates the date of the visit.  Once the date is confirmed with the project director, ANA will email an outline of the impact evaluation tool to the project director.  The outline indicates specific information that will be gathered on the date of the visit including challenges faced, best practices and lessons learned.  

An ANA staff member will call the project manager to confirm the date, time, and place of the meeting and answer any questions the grantee may have.

What information is collected during the evaluation?

The impact visit is an opportunity to share the story of the project, including what worked and what did not work so well.  The impact evaluation does not review activities in the OWP, since the ANA staff member conducting the visit will have read the quarterly reports.  The evaluation will instead concentrate on the “Results and Benefits Expected” and the “Criteria for Evaluating Results and Benefits” sections of the OWP.  

Grantees tend to be a bit nervous when an ANA staff member first arrives for the evaluation.  Please don’t be!  We have heard the evaluation process is a positive experience for grantees.  Grantees are working hard to accomplish their project and often enjoy the opportunity to sit back and reflect on their achievements.

How long can we expect the visit to last?

Visits usually last around 6 hours.

What does ANA do with the information that is collected?

Most importantly, we compile information collected from each grantee to report to Congress on what ANA funded projects are accomplishing – both qualitatively and quantitatively.  The information is also analyzed in order to provide current and prospective grantees with information to improve their projects.  We will also use the information to make changes to internal policies and procedures.

How do we prepare for an impact visit?

There are several sections to ANA’s impact evaluation tool.  When a grantee is contacted for an impact visit, we will provide them a detailed outline on how to prepare for the visit.  

During the visit, we will discuss your project’s:

Objective Work Plan (OWP)

For this section, you should be prepared to discuss your OWP, and your achievement in meeting project objectives.

Project Results and Benefits

ANA is interested in learning about the results and impact the project has had on the community. We will discuss:

  • The extent to which your project addressed the original problem statement as given in your application
  • Partnerships
  • Leveraged resources
  • Volunteer contributions
  • Businesses, if any, created through the project
  • Activities that generated income
  • Trainings and workshops

Community Involvement and Outreach

ANA appreciates input from beneficiaries and community members who participated in the project.

You will be asked about the degree of community involvement in the planning and implementation stages of the project; the number of elders and youth involved, and the extent of their participation; and the type of activities and events held to share the project with the community.

Project Impact

We will ask you to discuss your project-specific impact indicator(s),who benefited from the project, and the impact on each beneficiary.  We encourage grantees to invite project beneficiaries to these discussions.

Sustainability

We will ask you to reflect on what steps you have taken to ensure your project continues after ANA funding ends, and share what your next steps will be to continue project services.

Project Challenges

We will gather information on the challenges encountered during project planning and implementation.

Staffing

We will ask you about project staff, consultants, and others who were paid to work on the project so we can learn about the jobs created by the project.

You will be asked about the record-keeping of the non-federal 20% contribution to the project, budget modifications, and ANA funds left over at the end of the project.

Financial

Though the impact visit is not an audit of the project, we will ask some questions about how the project’s finances were tracked and managed.  We use this information to inform our Training and Technical Assistance processes as well as to analyze challenges and best practices.  We invite financial staff to attend this part of the visit.

Program Area-Specific Questions

ANA will ask questions specific to the competitive funding area of your project: Social and Economic Development Strategies, Native Languages, or Environmental Regulatory Enhancement.