When we developed the Opportunity Index in 2011 with our social science research partner, Measure of America, we asked ourselves a lot of questions. As a data-driven organization, we understand the power – and limitations – of research. So we weren’t surprised when our key questions aligned with the five simple questions outlined in Phil Buchanan’s Chronicle of Philanthropy piece from last year about evaluating philanthropic research. We hope other nonprofits will also use these “five questions” – or questions like them – when they review new and existing research and data sources.
What was the methodology used?
Updated annually, the Opportunity Index is a composite tool that measures opportunity in communities using 16 interrelated economic, educational and civic indicators. The Opportunity Index provides Opportunity Scores for all 50 U.S. states and Washington, DC, and Opportunity Grades for more than 3,100 counties and county equivalents, home to 99.9 percent of the nation’s population. Instead of including factors beyond one’s control – such as race, IQ or family background — the Index focuses on conditions present in different communities that are susceptible to policy change and public and private sector actions intended to improve outcomes for residents.
It’s important to us to be transparent about our data sources. In fact, a complete explanation of Index methodology is listed on OpportunityIndex.org with links to all of the original data sources, including the U.S. Census Bureau, U.S. Department of Labor and the U.S. Department of Education, as well as information collected at the state level, such as the percentage of adults who volunteer and are members of community organizations.
Upon request, we also make the entire spreadsheet of raw data available to policymakers and media outlets, so that they can run the numbers in specific ways, such as comparing the performance of neighboring counties or states.
Is the conclusion warranted?
In most cases, we have not used Index findings to draw conclusions. Instead, we use the information to point to a potential area of focus and to encourage cross-sector collaboration among our diverse network.
For example, we found that one of the indicators with the greatest correlation to overall Opportunity Scores is the number of 16- to 24-year-olds who are out of work and out of school. The lower the number of disconnected youth in a region, the higher the region’s Opportunity Score. This finding has informed much of our policy work, which is geared toward ensuring more teens and young adults get their fair shot at the American Dream.
There are always risks associated with the collection and framing of social science data, in particular the danger of assuming causation instead of correlation. For example, are communities that invest in other opportunity-related factors – such as access to the Internet, healthy food and medical care – less likely to let their youth fall through the cracks? Or do employed and educated youth raise other aspects of a region’s Opportunity Score, such as the employment rate and household income?
What we do know is that the correlation between opportunity and engaged youth pointed to an area where we and our partners could focus our efforts in a meaningful way. With that in mind, we worked with our partners to develop a bipartisan Shared Plan to strengthen pathways to education and career for young adults.
Is this really research at all?
Yes. The Index is a data-driven tool that strives to be objective and accessible to the field. It’s a new way of looking at opportunity via community conditions, using the best comparable data available.
Has other relevant research been done on this topic?
After getting feedback and input from the field from hundreds of organizations, we learned that there were many tools that measured opportunity at the individual characteristic level, but there were no nationally comparative tools that measured opportunity at the community level. That was the impetus for us to create the Opportunity Index and ground it in real data.
Since then, we have started to see additional work on opportunity and mobility, which is exciting for us and our partners. For example, we were thrilled when a team led by Harvard Economist Raj Chetty launched The Equality of Opportunity Project in the summer of 2013. This project examines how geography affects intergenerational mobility by analyzing individual earning records.
We view these developments as complementary to our efforts, and a validation of the need for more tools and resources to promote the expansion of opportunity. High-quality research and indices raise awareness about key aspects related to upward mobility. We closely track and share companion work. In some cases, we collaborate with the researchers and organizations responsible for these projects.
On that note, we’d be remiss not to mention some other awesome tools and research you should check out:
- Pew’s Economic Mobility Project
- The Boston Indicators Project
- Young Invincibles study on youth unemployment: In This Together
- US News/Raytheon STEM Index
Who paid for it?
Opportunity Nation maintains a strong commitment to the Opportunity Index as an independent tool created for use by the field and policymakers. The Index is part of our annual budget and is paid for from unrestricted funds from individuals and foundations. We are fortunate to have partners who believe in our mission and trust us to execute it to the best of our ability.
High-quality research and data-driven tools are instrumental in the work that nonprofits do. It is incumbent upon all of us to divulge our information sources, methodologies and biases – and to make data and research as accessible as possible.
As nonprofits and leading organizations, we should never be afraid to ask ourselves these simple, but critically important questions, and share our answers with others. In a world of information overload, we should also strive to be proactive about communicating what it is we are trying to accomplish with data and research tools, and acknowledge their limitations. Most of all, we must all make an effort to become data connoisseurs – no matter what our roles – so that we know the simple and smart questions to ask and answer.
Sarah Beaulieu is a senior advisor at Opportunity Nation. You can find her on Twitter @SarahBeaulieu.
