On Monday, I posted a blog about the March cover story in Harvard Business Review (HBR), on work-life balance, arguing that the conclusions drawn by the authors were not credible. I took issue with the low sample size of their survey and an interview methodology that seemed questionable, at best.
Most stunning was the survey data, which formed the basis for claims about gender differences but included responses from only 24 women – and 58 men. I argued that “the article provides a stark illustration of a problem I am growing increasingly agitated about.”
What is the problem? “The tendency for respected publications to present research ‘findings’ without sufficient transparency about methodology and limitations – and in a manner that can lead readers to believe that there is much more ‘there’ there than is the case.”
I emailed my blog post to the article’s primary author, Harvard Business School (HBS) Professor Boris Groysberg, as well as to HBR, and I look forward to their responses.
Meanwhile, also on Monday, I noticed a new piece from the Stanford Social Innovation Review. It is a short article by Mark Kramer and Kate Tallant of FSG Social Impact Advisors that introduces a transcript of – or highlights from – a gathering of “seventeen corporate and global education leaders” in Davos focused on education. The body of the discussion is interesting, I think, and will certainly generate a lot of conversation. For example, Michael Porter, also of HBS, argues that we should “apply the capitalist model to education,” a statement that, for me, raises some questions about what, exactly, that means.
But my issue with the SSIR article is not so much that it seems like a bit of a puff piece for the companies.
My issue is with the article’s first sentence, which reads: “Companies across industries – including education, technology, financial services, agriculture, and more – are playing a new role in education, one that improves learning outcomes while driving shareholder returns.” [Italics are mine.] In a comment to the SSIR article on the SSIR Web site I asked to know the basis for this claim.
Tallant replied on the SSIR site, citing anecdotes of two companies doing work she says is effective and explaining that many smart people are increasingly discussing the notion of a “new and improved role for companies in education.” That didn’t exactly answer my question.
The HBR piece I wrote about earlier in the week was at least based on research, however flawed. But this assertion in SSIR appears to have no basis. The sentence could have read “companies are seeking to improve learning outcomes,” but it didn’t. It said that corporate involvement leads to better outcomes.
Look, we won’t make progress on tough issues just flinging assertions around as if they are supported by data when they aren’t. Making a claim about outcomes – even in a short article introducing a transcript – is a big deal.
Or it should be, at least.
Phil Buchanan is President of the Center for Effective Philanthropy. You can find him on Twitter @PhilCEP.
