“There are many things of which a wise man might wish to be ignorant,” wrote Ralph Waldo Emerson. What’s behind the research you consume is not one of those things. It seems every other day there is a new article or blog delivering solutions or guidelines for what to eat, how much to sleep, or how to be an effective leader. Figuring out how the authors of those solutions and guidelines arrived at them is often a guessing game.
The leading organization of public opinion and survey research professionals in the U.S., the American Association for Public Opinion Research (AAPOR), is heading the charge to change this guessing game. In October of 2014, AAPOR kicked off its Transparency Initiative, which is meant “to encourage broader and more effective disclosure of research methods by all organizations.” AAPOR states that its goal “is to promote methodological disclosure through a proactive, educational approach that assists survey organizations in developing simple and efficient means for routinely disclosing the research methods associated with their publicly-released studies.”
The Center for Effective Philanthropy (CEP) has always been an advocate for transparency and open communication. Such values are the very basis of the tools we provide to foundations and the research we conduct. In the publication of our research, and in our communications with individual client foundations, we share the methods we have used, how we selected our samples, any bias we could detect in who did and did not participate, how we arrived at the conclusions we are sharing, and what we see as shortcomings of our work.
Because conducting surveys is a core element of CEP’s work, we applied to become a charter member of AAPOR’s Transparency Initiative. In an email welcoming us to the Initiative, Michael W. Link, President of AAPOR wrote, “We believe you and your organization are providing a strong example to others as a ‘good citizen’ who values and practices openness and transparency….Your participation will help to achieve our goal of an open science of survey research.”
Too often I find myself reading an article or a paper recommending how foundations or nonprofits should change the way they work to have more impact – yet, there is little to no information about how those authors arrived at their recommendations. Sometimes it is apparent that data was collected, but little information is provided about how it was collected, or how it was analyzed. It should never be difficult to understand the methodologies and reasoning upon which recommendations for improving practice are based. Readers should be able to tell if recommendations for improving practice are grounded in rigorously collected data or questionably collected data – or rigorously analyzed data or questionably analyzed data. All of us who publish data from surveys – or from any type of research – with the intention of influencing practice through our work have a responsibility to provide the relevant information that allows readers to judge the quality of the research for themselves.
Through the Transparency Initiative, AAPOR is doing its part to encourage research organizations that conduct surveys to adhere to this maxim. We at CEP share this value, and we’re excited to be a charter member of such an important and necessary movement.
Read the full press release here.
Ellie Buteau, PhD., is vice president, research, at CEP. You can follow her on Twitter at @EButeau_CEP.
