Is there really a “march to metrics?” Or is it more of a sparsely populated – if much trumpeted – stroll?
What do donors want?
This is the question posed in a thoughtful, nuanced piece in the Nonprofit Quarterly by Cynthia Gibson and William Dietel. They argue, I think rightly, that many donors may not be as interested in data as we assume. They also say that what they see as a “march to metrics” needs to be tempered by a recognition of the complexity of measurement in the nonprofit sector and by an acknowledgment of the role of “heart” in philanthropy. They suggest that the right balance “falls somewhere between data and desire.”
The piece is well worth reading, and its authors speak from years of experience in institutional philanthropy. It is a welcome addition to the discussion of metrics and data because it mostly eschews the kind of over-simplification of these issues that is all too common. The authors are not foes of data-driven decision making by any means, but they worry that its emphasis is coming at the expense of other important factors.
Although I see their point, I am not convinced. While I find much to like in the piece, I find myself differing with Gibson and Dietel on a couple of key issues.
First, I think they fall, at least partly, into the trap of assuming those who promote the thoughtful use of data in philanthropic decision making don’t see a role for passion, heart, or what they term the “art” of philanthropy. They position these forces as in tension. But my experience is that it is usually passion that actually motivates the quest for data: people want to know what works because of their burning desire to make a real difference on an issue or problem about which they care deeply.
Gibson and Dietel note, correctly, that “human beings bring to the philanthropic process values and feelings and historical experiences that no data set or analytical technique can replace.” But who is calling for replacement? Why does this get positioned as either-or?
The most effective philanthropists and foundations I have seen are driven by heartfelt concern and also very much informed by good data, used thoughtfully and in context. Passionate commitment and dispassionate analysis can live together, in harmony, side by side and mutually reinforcing. While I think this is in fact what Gibson and Dietel would like to see more of, I think they undermine their own argument by positing a dichotomy where there isn’t one.
Second, I think Gibson and Dietel confuse the rhetoric about assessment with the reality of practice. They worry about “formulae” or “mathematical models” driving philanthropy but, in CEP’s work over the past nine years studying and providing assessment tools to hundreds of foundations, I have seen precious few formulas or mathematical models. I think Paul Brest of the William and Flora Hewlett Foundation (disclosure: Hewlett is a major funder of CEP and subscriber to our assessment tools) has it right, responding to a Sean Stannard Stockton blog post calling for more “deviant” philanthropy, when he asks, essentially, what is actually deviant?:
“My guess is that most WRITING about philanthropy in the past decade has advocated for ‘strategic’ philanthropy, from which [those who question the value of strategy] are ‘deviants.’ But if you look at the actual PRACTICE of philanthropy, whether in terms of numbers of foundations or dollars granted [the critics of strategy] are completely in the mainstream, and strategic philanthropy is deviant. As an advocate for strategic philanthropy, I certainly hope that it becomes mainstream–but that hope is a long way from being realized.”
CEP’s research on foundations suggests a similar gap between rhetoric and reality when it comes to strategy and performance metrics: In our 2009 report, Essentials of Foundation Strategy, we reported that just 26 percent of foundation leaders had indicators or metrics in place to assess all their strategies. Yet the overwhelming number thought their foundations effective.
Yes, there are a number of foundations that marry passion with strategy and metrics and, to me, exemplify what it means to be effective. But their numbers remain too few, the gap between rhetorical embrace and practice too wide. Our 2007 report on foundation strategy is titled, simply, Beyond the Rhetoric: Foundation Strategy.
My colleague Kevin Bolduc notes frequently, with wry humor, that the title of that report could apply to virtually any of CEP’s research reports. I think that’s true, sadly. And I think we see that kind of gap everywhere, not just in Foundationland. It happens all the time at companies, and not just the egregious examples, like Toyota. (As Stanford’s Jeffrey Pfeffer has noted, “companies don’t make very many decisions based upon the evidence.”) Or, think of a more mundane example of a gap between rhetoric and practice, like what you say when your doctor asks how much you exercise.
When we are able to get beyond what is written in annual reports, the trade press, or blogs, we see that most leaders of large foundations actually think there isn’t enough attention paid to assessment. In a survey conducted by LFA Group for CEP earlier this year, 70 percent of CEOs and program officers say more effort is needed to assess funder performance. There is great unease within many foundations about whether decision making is informed by enough good data about what works — what’s really effective — and what doesn’t.
So my concern is that Gibson and Dietel may be cautioning against drinking too freely from the fountain of good data when we are still only sipping. Dehydration remains a far greater risk than over-hydration. I have seen many large foundations up close and first-hand and I am pretty sure heart alone — even whimsy — is still informing a whole lot of (I’d argue, too many) philanthropic decisions.
Is there really a “march to metrics?” Or is it more of a sparsely populated — if much trumpeted — stroll?
