As I read CEP’s Assessing to Achieve High Performance report, several thoughts came to mind. First, “Ooh, these graphics are so pretty!” Second, “Yeah, funders need to provide more money to nonprofits so we can do more and better evaluation.” Third, “Data needs to be more culturally competent — for example, being disaggregated into different groups, since there is danger in assuming findings can be generalized to all populations” (something I blogged about recently in “Weaponized data: How the obsession with data has been hurting marginalized communities”).
Fourth, and what I want to talk about here, is the fact that we are overlooking some important data. The majority of the data we are collecting and disseminating is known as “summative data,” information that sums up how awesome a program or nonprofit is at the end of (usually) a year, when everything has been done. This is the type of data we report out to our communities: “This year, 85% of the kids in our program advanced by one grade level in STEM skills,” etc.
But we are ignoring a rich source of data that could add texture and critical information to our sector: formative data. Broadly the information and key lessons that we collect along the way, including our mistakes and even failures, formative data helps shape programs and organizations’ practices — and yet we don’t pay enough attention to it. Or we collect it but never share it, thinking of it as an internal process rather than something that could benefit the entire field.
My start-up nonprofit, Rainier Valley Corps, has been hard at work building a model to get more leaders of color into the nonprofit sector. It is a multicultural model in which emerging leaders of color are trained and mentored throughout the year while being placed to work full-time at an ethnic nonprofit to develop that organization’s capacity. We are still in our ramp-up year, so there is no summative data yet. But the formative data — the lessons we have learned, the screw ups we have made — are rich and vital to our work moving forward.
For example, our first quarterly community gathering was held at a shared workspace that has rotating works by local artists on display. This particular month featured beautiful paintings on the theme of female body image and society’s expectations. The entire room was adorned with paintings of nude women. The day before the event, a community leader looked around and was shocked that we were going to have the gathering in this room. None of us had even considered that these paintings, touching on an important topic, would upset anyone. It would have been a logistical nightmare to change venue. I made frantic phone calls negotiating between the artist and the community leaders who, for religious and cultural reasons, would refuse to attend this meeting if the paintings were not taken down. The discussions on gender dynamics and cultural competency, and the lessons learned from this incident, have been helpful as we plan not only other events, but also our curriculum — since we are developing the next generation of leaders, these are dynamics they must understand.
The tendency is for nonprofits to collect and share summative outcomes, while keeping formative lessons internal, except maybe during happy hour with colleagues (“Dude, my org totally screwed up and were culturally incompetent on booking the venue for our first community gathering, even though we’re trying to teach emerging leaders to be culturally competent. I need a Long Island Iced Tea.”) But by doing that, the rest of the field misses out on all the informative key lessons we gather along the way. For nonprofits, I am proposing:
We set aside time and resources to do more formative assessments. It doesn’t have to be as complicated as summative assessments. Maybe once a quarter during or after a team meeting, set aside an hour to answer these questions: “What were our successes this quarter? What were our mistakes or failures? What key lessons did we learn that would help us and other nonprofits doing similar work?” In addition, survey board members, key volunteers, and donors to get their input.
We share the results of formative assessments broadly with the community. Maybe write a blog post mentioning the insights gained. Or form a gathering and invite funders, donors, volunteers, and even clients to talk about the successes, challenges, and lessons learned in doing the work. Or even better, “Formative Assessment Happy Hour!” (It may need a better name.)
We be transparent about our mistakes and failures. They yield great lessons, and may prevent others from going down the same path. Most of us believe in creating a culture of transparency and constant learning. Yet, we still have a fear of disclosing our mistakes, thinking that stakeholders will look down on us. I think it’s usually the opposite, and that people tend to appreciate honesty coupled with actual learning and improvement. If we only focus on successful outcomes, we are missing out on half the data.
Our sector is extremely complex, and all of us are constantly trying to figure things out. There is a trove of interesting and useful formative data out there. One of the nonprofit leaders in CEP’s report says, “I think the only way to actually continue to make effective programming is to know exactly what the impact is.” I would add that to create effective programming, we also need to know how we got to the impact.
Vu Le is the executive director of Rainier Valley Corps in Seattle. He is also the author behind the popular blog, Nonprofit with Balls. Follow him on Twitter at @Nonprofitwballs.
