Rigorous Evaluation Versus Trust-Based Learning: Is This a Valid Dichotomy?

Social change takes time, and it is never a linear process. It also happens within an ecosystem, and rarely comes down to a single grantee partner or foundation. If we are honest, change is entirely out of a foundation’s control. Despite all of this, funders continue to engage in evaluation as if the precision and volume of what we ask from grantees will improve our giving, and help improve lives.

While most of the philanthropy field continues down a business-as-usual evaluation path, a new approach to learning is rising — one that is rooted in trust, equity, and learning for impact. Some funders are suspicious of this reimagined framework; many believe (falsely) that it is not concerned with outcomes or based in rigor. Few understand that it can hold the key to the long-term, systemic change that most of us are working toward.

Respectfully, those who are cynical about what this innovative approach offers do not fully understand its value. I know because I have been practicing trust-based philanthropy with rigor, impact, and strategy and measuring progress along the way.

Measuring What Matters – to Who?

Before I dive into a trust-based approach to evaluation, let me share some of the underlying beliefs and assumptions that fuel philanthropy’s antiquated (yet current) approach to evaluation. Most funders currently track data with more or less the same mainstream approach because they believe:

• The main audience for their evaluation work are board members and leaders who want to see “proof” of impact;
• Traditional ROI approaches from the business community apply, based on the experiences of many foundation leaders and business’ influence on philanthropy;
• Foundation dollars alone are causing positive social impact as written into grants;
• Evaluation and learning can be defined narrowly, based on white-dominant frames around impact, such as funder-driven metrics; and
• Grantee reporting is, and should remain, a one-way stream.

This is all problematic, to say the least. This way of thinking about evaluation reinforces funders as the central figure in this work. It sets up short-term metrics that allow funders to pat themselves on the back, rather than see, and actually address, the real challenges communities face, including the time they waste unnecessarily reporting to funders. It also fails to capture actual learnings that could be useful to grantees, communities, or foundations.

These traditional approaches to learning and evaluation also ignore very real discrepancies between groups with resources to put toward grant reporting, and those without. These differences often run along lines of race, gender, gender identity, class, and other social factors. Smaller groups with less support tend to have less capacity to track and report their impact along traditional business models.

When grantee partners are able to report back, the funder-initiated song and dance continues. Organizations share what they believe funders will want to hear, and staff then pass along the information they think boards will want to hear. It’s a game that many know is being played, but that our sector refuses to call out. Many funders, my former self among them, ask grantees for reports and data that never or rarely get read.

A New Chapter for Evaluation

I believe that if we funders were honest with ourselves, we would all agree that the old ways of working are not getting us to the place we need to reach as a sector, or as a society. Before embarking on shaping this new framework, I too followed traditional approaches to learning and evaluation, for nearly two decades. I used the evaluation frameworks everyone else used, the same rigorous grant reports – not because they felt useful or right, but because that was the expectation.

I first joined Headwaters Foundation at a time in my life when, because of my growing discomfort with the role of funders of making change, I had started to wonder whether I should remain in philanthropy. Headwaters was a brand new health conversion foundation, and I was tasked with creating its structures from the ground up. It was an opportunity to reinvent how philanthropy does its work.

Taking the helm there proved to be the first time in my life that philanthropy felt rooted in the love of humanity instead of the foundation’s ego. When it came to evaluation, my work was about listening to communities and acting on what I was learning, not making grantees jump through structural hoops or report back in traditional ways. A new way of thinking about learning and evaluation began to take shape.

Three Trust-Based Pillars of Evaluation

Headwaters and other trust-based foundations are experimenting with a fundamentally different approach to learning and evaluation rooted in three pillars:

1. Learning for Accountability

While traditional evaluation consists of funders looking at what grantees have been doing and saying, learning for accountability turns the magnifying glass into a mirror. This aspect of learning requires funders to see accountability as a core part of our work and role — to ourselves, and to our grantee partners. Are we showing up authentically? Are we living our values? What do we do when we realize we’re showing up poorly? Learning for accountability recognizes one of the few areas that funders can truly have influence over — how we show up. These questions should be part of every evaluation approach, yet are part of almost none.

Chicago Beyond is a family funder and learning organization that has built accountability into their evaluation structures. They view the impact of their investments through a holistic lens, including “our ability to hold ourselves accountable for our actions.” Learning is a central part of how they approach their work, including regularly assessing the health of their partnerships along with the value they are contributing beyond the check.

2. Learning for Decision-Making

This aspect of learning starts with the understanding that grantee partners, not funders, best understand how to advance their work. By listening to their expertise, funders play a different and more impactful role within the nonprofit-funder ecosystem. Funders do not listen for the stories we’ve grown accustomed to asking for; instead, we ask grantee partners to truly shape what the work looks like, and tell us what they need to succeed, what’s working and what’s not, and how we as funders are supporting our partners in meeting their goals.

Satterberg Foundation in Seattle has worked to embed an intentional learning mindset into the work of the organization. In addition to conducting regular insight circles to collect feedback from grantee partners, they view grantee conversations as opportunities for program staff to come together and reflect on trends and the implications for their grantmaking. Most recently, the staff came together and noticed there was a recurring trend of burnout among many of their grassroots partners. They were able to respond by instituting a series of wellness grants dedicated to supporting the health and wellness of grassroots leaders.

3. Learning for Long-Term Impact.

Most funders are working to make change in the communities they care about. Yet, the answer to whether or not we are succeeding cannot be found in our current evaluation measurements. That’s because change doesn’t happen at the pace of our grantmaking. Rather than focus on the narrow view of a single grant, our evaluation must consider a long view, across ecosystems. We must assess change over the course of years and decades, not one to two year chunks. We must engage wider, cumulative community efforts, not the success of a single organization’s goals.

Health funder bi3 works to reduce health disparities throughout the Greater Cincinnati Area, including the disparity between the rate at which Black babies are dying relative to other racial and ethnic groups. bi3 has taken a decidedly different approach to evaluation, working directly with grantee partners to co-create metrics and indicators of success, alongside publicly available data on infant mortality. They remain flexible and adaptable; they do not want their grantees to be so focused on achieving pre-defined outcomes, which may become outdated over the course of the project, that grantees miss new opportunities to achieve even greater impact. This bigger picture approach is working: Black preterm birth-related deaths (preterm birth is the leading cause of infant mortality in Hamilton County) have been nearly halved after years of coordinated, community-wide efforts to reduce infant mortality.

Critical to these pillars is that an organization has laid the groundwork for adopting them. Foundations have necessary cultural work to do before this emergent approach to learning and evaluation can begin. It involves shedding old notions and embracing a new, more vulnerable philosophy. At Headwaters, we did this by building a culture of trust and learning versus a culture based on accountability and impact alone. This required shifting paradigms of why we were evaluating, who the main audience for the learning was, who defines success and who bears the responsibility for collecting and analyzing data.

Before foundations engage in these new pillars, there must be a recognition, and perhaps a reckoning, that this new approach changes the role of funders and proactively works to shift power.

What is it about trust-based evaluation that traditional funders don’t understand?

In rapid time, we have seen trust-based philanthropy grow from a seed of an idea to an often-referenced term in our sector’s largest conferences and publications. Many events in the last few years have fueled a new and necessary urgency for funders to do this work differently, and more impactfully. Meanwhile, evaluation has been a step behind in the swift uptake of trust-based philanthropy. I am grateful for this progress and eager to ensure that trust-based evaluation is adopted as part of the real changes our sector makes.

As trust-based learning and evaluation begins to catch up, it has been falsely dichotomized against rigorous evaluation because of a few fundamental misperceptions. Some narratives have oversimplified what a trust-based approach to learning and evaluation is, characterizing it as lacking rigor and not being concerned with impact and evaluation. Traditional funders who stop here miss the opportunity to explore the many examples of innovative, grantee-centered, anti-racist evaluation approaches our colleagues in philanthropy have created and shared with the sector.

In addition to these misperceptions and missed opportunities, there is a place which I suspect may be the origin of at least some of the confusion. While trust-based philanthropy has been around for a long time, much of the framework was named and laid out by the Whitman Institute (TWI) in 2014, based on feedback from their grantee partners. When TWI (which has since sunset) first shared their framework, it was not explicit about evaluation and learning. Because TWI was the leader in defining trust-based philanthropy at large, it is possible that some who are familiar with that original approach may not have followed its evolution to include the rigorous, values-aligned learning and evaluation framework we have today.

Building the Field as We Go

Trust-based learning and evaluation are a critical, evolving body of work. Very few foundations have shifted fully, maybe none. I believe that Headwaters Foundation may be the furthest along, but we’re not the only ones who took our dissatisfaction with the way evaluation works and the limits of what it can teach us to build something that is better, more values-aligned, and that helps us be better funders.

The Robert Sterling Clark Foundation created CHAT (Check-in Analysis Tool), which replaces site visits and written grantee reporting as an annual conversation with grantee partners. The Trust-Based Philanthropy Project offers this self-reflection tool. American Jewish World Service has created an in-depth outcome monitoring tool that is filled out by programmatic staff (based on conversations with partners) and maps outcomes alongside their contributions, the grantees’ contributions, the context, and the challenges — in order to capture and document a more complex and realistic understanding of the barriers and contributors to advancing systems change.

Headwaters Foundation’s annual Learning and Evaluation Data Book is an example of how our team has brought a trust-based learning approach to life. Comprised of oral and written reporting data, internal goal-tracking metrics and stories of lessons learned, this publicly available tool helps us determine what is working, what isn’t, and how we can adapt to better serve partners and communities. The learning book also helps track the big-picture impact of our work and funded projects and pinpoints questions to guide future learning. It is an example of how to talk about impact while eliminating the traditional power dynamics and ego that are baked into traditional evaluation practices.

Moving to Impact, Learning, and Evaluation

Philanthropy has an evaluation problem. What funders have come to look for and focus on through traditional evaluation is not what will help us improve our giving, or help communities thrive. We don’t have to give up rigor to live into our values. We must not dismiss progress because of evaluation, however wide or narrow we understand it to be. It doesn’t have to be either/or, as some have tried to claim it is. It must be both. And every foundation will have to take the wisdom others have offered and forge their own path based on the values they hold dear.

I am clear that I do this work to serve communities and to have greater impact. The only way that feels possible is through a radically reimagined and reconfigured philanthropy, including a rigorous, trust-based approach to learning and evaluation. It will take time and work, but I believe it will get all of us — funders, grantees, and communities alike — to a place of more honesty, greater impact, and community-informed transformation. I hope more funders will join me in learning about, building and adopting this new framework that our sector, and our world, needs.

Learn more about this topic in Trust-Based Philanthropy’s recent webinar, “What Does Evaluation Look Like in a Trust-Based Context?”

Brenda Solorzano is CEO of the Headwaters Foundation, which works side-by-side with Western Montanans to improve the health of communities.