Perhaps your reaction after reading the latest CEP report, Assessing to Achieve High Performance: What Nonprofits are Doing and How Foundations Can Help, was similar to mine — skepticism. Perhaps you read “almost all nonprofits report collecting performance assessment data…and they want more data,” and you thought — what nonprofits are these? Surely not the ones we fund! This was my reaction when I first read this report. My experience in 15 years of working in philanthropy has been quite the opposite, and I started to dig in, looking for ways to challenge the methods of the study and trying to figure out where CEP went wrong.
But, like all CEP studies, the methods and analyses were sound. I had to look instead into what I really knew about the organizations we fund at The Colorado Trust. Perhaps more than most foundations, we have made considerable investment in evaluation over the years. Almost every grant over the past 30 years has included a funded evaluation component, sometimes costing as much as 25% or more of the total strategy budget. Working with independent third-party evaluation firms, grantees have received seemingly endless hours of technical assistance and evaluation support.
Not once, however, do I remember asking grantees what kinds of information they collect to understand their performance. We didn’t ask what data were important to them for their own decision-making. Never did we ask them about their capacity to collect data meaningful to them. Rather, we usually approached grantees with our questions and provided resources to collect data for our answers.
I looked at the findings in Figure 9 on page 15 of the report showing that a full 25% of respondents reported the performance data required by their funders was “a little or less useful” to them, and I had to admit: this report is very much about the nonprofits we fund. Discussions with grantees about their assessment work? For the most part, only as it relates to the data the foundation needs to answer our questions. Providing financial support for grantee assessment efforts? Very rarely, unless that support would answer our questions.
The more times I read this report, the more I saw our grantees in these answers, and the more I realized we must dramatically shift our focus when it comes to how we work with grantees. And what better way to do this than to model what we believe is important to succeed in overcoming the most complex social problems.
The report indicates only 41% of nonprofits surveyed share what they have learned with other organizations. How much of our own evaluation investment is shared? How many of us have commissioned lengthy reports, asking grantees to provide data, only to say the findings are “internal; for the funder only?” There are examples from a few remarkable foundations that have openly shared their mistakes and learnings, but only a few. It’s time to ask ourselves what better way to encourage grantees to share their learnings than to do this ourselves. Can we, as foundations with a commitment to evaluation and learning, agree to share publicly one mistake we made? Unless we answer yes, we can hardly expect grantees to feel comfortable doing the same.
The report’s findings also show less than half the respondents reported receiving support from foundations for their performance assessments. Foundations have heard over and over the importance of providing general operating support. And many of us do give operating support. Yet, apparently it is not adequate enough to provide resources for assessment, something grantees in this study clearly feel is important.
Can we as foundations commit to providing not only operating support, but enough operating support for nonprofits to improve and enhance their evaluation work? We need to ask critical questions of grantees: what are your evaluation questions? What do you want to know to improve your work? How will you use these results? Once we understand what questions keep grantees awake at night, can we commit to helping them answer these questions — and to helping them share the answers with other nonprofits to help them in their work, as well?
We as foundations have a responsibility to continually evaluate and learn. We have an obligation to assess our own foundation’s impact on the social problems we choose to tackle. To do this we need to ask grantees to collect and contribute data to a larger evaluation than their own nonprofit assessment. As funders we have the right to ask for this. And yet, we also have the responsibility to be sure we are asking of grantees only what we really need for our assessments.
When only a quarter of the nonprofits report the data foundations require of them is useful, it indicates foundations are not talking with grantees, and not helping them to understand how their own work contributes to the larger goals of the foundation. This mirrors findings from a review of past Colorado Trust grantee final reports — when asked what we could do to improve our grantmaking, grantees across the board told us to be clearer with them about how their work and what they accomplish fits into the overarching foundation theory of change. If we spend time answering that question with grantees, we will all be more likely to ask only for data that is relevant to grantees and to the funder.
We need to encourage our grantees to continue the evaluative journey from performance assessment to purposeful learning. We must encourage risk taking and reward adaptive learning in ourselves and our grantees. We must continually ask different questions and share accountability for the answers publicly. We must be willing to make new mistakes and encourage and trust our grantees to do the same, while rewarding creativity and new solutions to old problems.
As CEP’s report suggests, we can engage in deeper conversations with grantees to understand their needs regarding evaluation, continue to provide general operating support, and, with that, encourage time to review results, reflect, and adapt. We can encourage grantees to share what they have learned and provide resources and assistance for them to do so, and do the same ourselves. As funders, we should jump on the opportunities to encourage our grantees to embrace a culture of evaluation and learning that results in seeing problems and solutions differently. And always, we must do ourselves what we ask of grantees.
Nancy Baughman Csuti is the director of research, evaluation and strategic learning at The Colorado Trust. She is an advocate for asking, learning and evaluating in all that the foundation does, always seeking opportunities to push boundaries and make new mistakes.You can follow her on Twitter at @NancyCsuti.
