Nonprofit organizations — and foundations — talk a lot about accountability. And one of the best ways to be accountable is to demonstrate impact. While you or I would be hard pressed to think of a nonprofit organization that does not want to demonstrate impact, the all too true situation is that the vast majority of nonprofits struggle to do just that. At the heart of the problem are five practices that stymie nonprofit (and philanthropic) attempts to collect and use data to make important decisions, to improve outcomes, and ultimately to demonstrate impact.
Done right, evaluation works.
We know from the examples in Assessing to Achieve High Performance that evaluation can be meaningful and useful to nonprofit organizations. Because of data and evaluation, Stages Repertory Theatre was able to dramatically increase single ticket and subscription sales, and the YWCA of North Orange County can document the length of time from diagnosing to treating breast cancer to improve health outcomes. Additionally, there are 75 Leap of Reason ambassadors who are committed to building high performance organizations built on a foundation that includes measurable results, internal monitoring, and external evaluation. (For more information, check out The Performance Imperative.) The question is not if evaluation has value to nonprofits — the question is if your nonprofit values evaluation.
So what is the challenge keeping more nonprofit organizations from valuing evaluation?
It’s certainly not lack of interest — 71 percent of nonprofit leaders surveyed in Assessing to Achieve High Performance want more data — “more detailed data, a larger volume of data, or more frequently collected data” (Buteau, p10). Looking at the issue from a few angles, there are five grantmaker and nonprofit practices that undermine or limit the ability of nonprofit organizations to fully engage in evaluation:
- Pervasive culture of one-year planning: Only 28 percent of funders provide multiyear grants “often” or “always” — leaving the vast majority of funders providing one-year support (McCray, p7). Given the relatively short-term nature of grant agreements, nonprofit leaders are often hesitant to make investments in the long-term infrastructure necessary for tracking, monitoring, and using data.
- Consistent partial funding of efforts, especially evaluation: For a variety of reasons, funders rarely cover the full cost of the programs they intend to support (Nonprofit Finance Fund, p17; McCray, p13). All too often, the easiest expenses to cut from project budgets are investments in data collection and related staffing. As a result, grant funds and other revenues do not reliably cover these costs. Compounding the problem is that revenue sources other than foundations and philanthropies are less likely to specifically support evaluation (Reed, p9). Foundations and philanthropies are more likely than other funding sources to provide support specifically for evaluation, but it is unfair to ask foundations and philanthropies alone to take on the Sisyphean task of wholly supporting nonprofits in this regard.
- Systemic flaws in nonprofit financing: Compounding the challenge of investing in evaluation is a broader systemic problem of unstable nonprofit finances. Many nonprofits are challenged to attain long-term financial sustainability through no fault of their own. As a result, revenues may ebb and flow, organizations may have limited or no cash reserves, and generally, there may not be extra cash on hand to pay for costs that are not directly covered by project budgets. (Yes, there are nonprofits that are poorly managed, but not all nonprofits who struggle with financial sustainability are poorly managed.)
- Evaluation is understaffed, especially in smaller organizations: Smaller nonprofits are less likely to employ full-time staff dedicated to data and evaluation. In Assessing to Achieve High Performance, organizations with budgets less than $1.4 million were less likely to “employ staff dedicated to working on assessing the organization’s performance” (Buteau, p12), and in Innovation Network’s State of Evaluation 2012, 53 percent of large organizations (annual budgets greater than $5 million) had a full-time employee dedicated to evaluation — compared to 9 percent of small organizations (annual budgets less than $500,000) (Morariu, p8). Eighty-two percent of nonprofit organizations have annual budgets of less than $1 million — challenging many organizations to staff evaluation (Roeger, p2).
- Focusing on communications rather than performance management: Too often nonprofit leaders focus more on reporting results to Boards and funders than using evaluation for performance management and improvement. Many organizations want to use evaluation for learning and improvement (Buteau, p10), but often those desires are waylaid by reporting deadlines. On the other side of the equation, funders are more likely to use evaluation data internally rather than share it with their nonprofit partners on the front lines to drive practice improvements and impact (McCray, p20). Data and evaluation are far from fully utilized across the social sector.
What’s to be done about it?
We’ve created the problem, and we can do something about it. Funders and nonprofits can choose to make a few key changes that will make it much more likely the social sector can value evaluation:
- Provide multiyear grants: Funders should consider using multiyear support when they are, in fact, supporting multiyear endeavors. Longer-term agreements could be used to support infrastructure investments that are otherwise off-the-table in one-year grant cycles.
- Provide general operating support: Grantmakers can also provide general operating support to their trusted, high-performing partners. Flexible funding can be used to cover the full cost of programs that are otherwise underfunded, giving nonprofits the full range of staffing and tools they need to develop, implement, and manage ongoing efforts.
- Agree on metrics and data: Funders and nonprofit leaders need to discuss metrics and data as part of the proposal or project planning process. Nonprofit leaders should be ready to recommend metrics and data that are aligned with the organization’s ongoing data and assessment efforts. Funders need to be prepared to listen to grantees’ recommendations, encourage grantees to collect and use appropriate data, and respectfully work with grantees to contribute to the funder’s data needs.
- Demand data from managers: Nonprofit leaders must do more than promote a culture of data and results — they must demand that managers and staff collect and use data for decision making and good management. Evaluation is management. If a manager does not have data about the work they are managing, what are they managing to?
- Just do it: Nonprofit leaders must set the tone from the top that collecting and using data is a given. Follow up with policies and guidelines to clarify and support implementation, like an organizational evaluation policy. (For a great example, check out TIERS: A Tool for Allocating Evaluation Resources at Nonprofit Agencies by Rachel Albert and Laura Beals.) Without exception, all organizations should track basic information about activities, services provided, and/or deliverables. Additionally, when feasible, nonprofits should attempt to capture data that reflect the outcomes of their work — the changes that they’re able to bring about for their clients, constituents, communities, and other stakeholder groups. These types of data collection can and should be done by staff (or trusted volunteers). Third party evaluators can then be used to build on and extend these most basic forms of data collection.
Johanna Morariu is the Director of Innovation Network, a consulting firm that supports social sector organizations to collect and use data for decision making. You can find her on Twitter at @J_Morariu.
Since 2010, Innovation Network has conducted research into nonprofit evaluation capacity and practice through the State of Evaluation project). State of Evaluation 2015 will be released later this year.
Buteau, E., Gopal, R., and Glickman, J. (2015). Assessing to Achieve High Performance: What Nonprofits are Doing and How Foundations Can Help. Center for Effective Philanthropy. To download the report, visit cep.org
McCray, J. (2014). Is Grantmaking Getting Smarter? Grantmakers for Effective Organizations. To download the report, visit geofunders.org
Morariu, J., Athanasiades, K., and Emery, A. (2012). State of Evaluation 2012: Evaluation Practice and Capacity in the Nonprofit Sector. To download the report, visit stateofevaluation.org
Nonprofit Finance Fund. (2014). Nonprofit Finance Fund: 2014 State of the Nonprofit Sector Survey: National Results. To download 2014 State Of The Nonprofit Sector Survey analysis and products, visit nonprofitfinancefund.org/state-of-the-sector-surveys
Reed, E. and Morariu, J. (2010). State of Evaluation 2010: Evaluation Practice and Capacity in the Nonprofit Sector. To download the report, visit stateofevaluation.org
Roeger, K.L., Blackwood, A.S., and Pettijohn, S.L. (2012). The Nonprofit Almanac 2012. To download the flyer, visit nccs.urban.org
