In my last blog post I described why we started assessing our performance as a foundation and how we developed an Annual Performance Report that balances analysis of our grantmaking with tracking of overall institutional effectiveness. As we have engaged in institutional assessment work, we have encountered three broad challenges I will explore in this post:
- The need to distinguish between reporting on activities and describing outcomes and impact
- The difficulty of summarizing complex social change
- The inherent conflict of assessing the past in a forward-facing enterprise
Activities vs. Impact
We are always looking for quantitative ways of describing and analyzing our work. Since the grants we award (and the ways we track those grants) are clearly quantifiable, they have become a prominent part of the Annual Performance Report. For example, we examine trends and changes in our grantmaking over time, explore the geographic distribution of our funds, and examine the populations served by our grants. While this provides an easily quantifiable way of examining our grantmaking, we realize that describing where our resources go is not the same as conducting an assessment of impact.
The same challenge applies to other sections of the report, where we describe financial and investment performance, summarize reports we have published, or discuss how we have refined our strategies based on what we are learning. While all of this together provides a comprehensive picture of our activities in a given year, it may or may not provide a complete assessment of institutional performance.
We have, of course, focused explicitly on the impact of our grantmaking reporting on evaluations of specific initiatives or clusters of grants focused on a common strategy or goal. In the Annual Performance Report, we summarize current evaluations and provide one-page descriptions of key findings and next steps we are pursuing. This section of the report probably comes the closest to an assessment of impact, and we continue to grapple with ways to report comprehensively on activities that we believe contribute to our impact as an institution while ensuring that the report goes beyond the descriptive.
Summarizing Complex Social Change
The very notion of a single report intended to describe an institution’s performance over one year suggests that it is possible to distill and to present such information concisely. We have not yet landed on the right formula here, but we are eager to keep trying and learning as we do.
We are certainly not the first foundation to confront this challenge, especially given the range of social problems that many of our institutions have chosen to address. The particular challenge we face in our performance assessment work is reporting on progress within a one-year period on a set of broad and aspirational goals that have a longer time horizon.
For example, within our California Democracy program, we support efforts that aim to improve our system of governance at the state and local levels. This is a long-term goal, requiring us to collaborate with a range of partners and stakeholders and acknowledging a dynamic political context that is ever-changing. It is unlikely we can summarize effectively all of the complexities, nuances, and challenges of carrying out this work, as we aim to deliver a report that is concise and accessible.
And yet, this particular challenge should not keep us from trying. We are looking at ways to focus our reporting on our performance related to shorter-term objectives and measures of progress that we articulate at the start of each year.
For example, two such indicators for 2010 in our Youth program are the establishment of ten certified Linked Learning pathways and the development of new integrated curriculum in key industry sectors. We will report back to the board at the end of this year on achievements against these indicators in a much more concise format than the full-blown Annual Performance Report.
As we do this work, we will also continue to explore how to present complex information concisely without sacrificing the richness and nuance of the content. Here again, we are not the first foundation to confront this challenge, so we are interested to learn from others.
Assessing the Past in a Forward-Facing Business
The last challenge relates to understanding the core business of a private foundation, which is to make grants. As a result, the nature of our enterprise continually focuses us on looking forward to the next set of grants, while there are far fewer incentives to look back. To respond to this reality, we have placed a high premium on evaluation, both to understand our impact, but also to inform grantmaking. Fortunately, our board holds this value as well.
A related structural challenge can be dedicating resources to performance assessment. Foundations generally focus most of their staff resources on grantmaking programs and related administrative functions, which means that there are additional costs to doing evaluation and foundation-wide assessment.
The question for each institution to consider is whether these additional costs enhance the foundation’s ability to achieve its mission. I would argue that the more evaluation and performance assessment is weaved into the culture of the foundation and viewed as an essential ingredient for social impact, then the costs can be viewed accordingly. Of course, that’s not an excuse to avoid making hard choices about resource allocation, but we must acknowledge the costs of doing this work and the related benefits.
I sought here to outline some of the broad challenges we have encountered in assessing our foundation performance, and some of the solutions we’re exploring. I would welcome your perspectives on all of this, including other challenges we should be considering as well as potential solutions to address these.
My next blog post will describe feedback we recently received from our board about our 2009 Annual Performance Report.
Jim Canales is President of the James Irvine Foundation.
