I was en route to the Center for Effective Philanthropy’s (CEP) bi-annual conference in Boston when an in-flight magazine article prompted some thinking about issues of measurement in our sector. The article references a 15-year old essay by Paul Krugman and compares research trends in development economics to the evolution of the mapping of Africa. Krugman describes how cartographers came to plot the continent’s coastline with growing accuracy and how by the eighteenth century, Africa’s outline on a map was essentially indistinguishable from today’s maps. However, the interior was a different story. As Krugman writes,
The weird mythical creatures were gone, but so were the real cities and rivers…. Improvement in the art of mapmaking raised the standard for what was considered valid data…. Only features of the landscape that had been visited by reliable informants equipped with sextants and compasses now qualified [as valid data]…. In the end, the rigor of modern cartography led to infinitely better maps. But there was an extended period in which improved technique actually led to some loss in knowledge. I suspect there may be parallels to consider in our philanthropy work today.
Improvements in the art of philanthropy have also raised the standard for what is considered “valid data.” With their Grantee Perception Report and other tools and research CEP has played a leading role in this. Thanks to resources like Foundation Center, Guidestar, and the National Center for Charitable Statistics, a growing bank of longitudinal and comparative data on the U.S. charitable community is now available to mine for all kinds of uses. Similar efforts to upgrade the way we quantify, assess, and map the work of philanthropists, charities, and aid agencies are happening around the globe. As a foundation, we’re pleased to support many of these efforts through our Charitable Sector Support initiative and other grantmaking. Yet Paul Krugman’s essay suggests an important caution. Might there be a risk of “losing knowledge” – as he suggests happened in mapmaking? Might we too casually dismiss the second-hand stories that yield “mythical creatures” but also serve as important navigational guideposts? How helpful or distracting are ideological or technical debates over the validity of one measure or method over another – like mapmakers arguing over the quality of their sextants? These are the kinds of questions that come up at CEP’s conferences. The conference always draws a crowd of foundation CEOs, philanthropy advisors, and others interested in bringing greater rigor and measurement to the work of the charitable community, and foundations especially. This year’s sold-out conference was no different. At the Bill & Melinda Gates Foundation, we’re wrestling with questions around measurement and evaluation. Our Guide to Actionable Measurement laid down some initial thinking. Our Strategy Lifecycle Overview describes how we use various sources of information to develop, review, and refresh our strategies. But as Jeff Raikes, the Gates Foundation CEO, shared at the CEP conference, “Our sector is just at the beginning at what will be an accelerated wave to improve the way we measure the impact of our work and use feedback loops to lead to continuous improvement.” We include ourselves. In the absence of market signals, we need to borrow old measures and adopt, adapt, and invent various new means to make sure that we’re continuing to learn and adjust. Ultimately, we should seek to understand the “interiors” of our work, and not just the accessible coastlines. In the end, it is that kind of rigor and measurement that has the potential to lead to better philanthropy.
Darin McKeever is senior program officer, charitable sector, for the Bill & Melinda Gates Foundation
