What is the Worth of Investigative Journalism?

In Democracy’s Detectives: The Economics of Investigative Journalism, James T. Hamilton gives a bird’s eye view of the ecosystem of investigative reporting and addresses its costs, benefits and impact.

The need for investigative reporting has seldom been more acute. Even as investigative journalism thrives by way of cross-border collaborative projects such Panama Papers, pessimism about the profession itself continues. James T. Hamilton’s Democracy’s Detectives: The Economics of Investigative Journalism, is a helpful guide for those of us positive about the future of investigative journalism.

The book gives a bird’s eye view of the ecosystem of investigative reporting – the people who make it work, the oil it needs to run well, the nuts and bolts of what is needed to keep the machine going and the forces that make it harder to work.

This review is aimed at distilling key insights from the book and providing a brief description of how more developed and mature media markets such as the US have dealt with sustainable funding for investigative reporting. The book is for serious media entrepreneurs, particularly those considering the non-profit model and others looking to set up investigative teams.

It opens with the raison d’etre of investigative journalists, “Fraud, and its unravelling, can both build a business,” and goes on to demonstrate the various challenges that journalists and their funders face in uncovering fraud.

Hamilton, the Hearst Professor of Communication and the director of the journalism program at Stanford University, raises a fundamental question, “Who’s paying the bill for those who investigate the operation of democracy?” He also asks who hires investigative reporters to police the operation of institutional power.

“Democracy is not a person, foundation, group or agency and it does not act as a hiring agent for journalists.”

What the book addresses

Largely, the book tries to map the economics of investigative journalism – demand, supply, choices, costs and returns. It looks at indicators including investigative stories, awards for journalism and congressional hearings to document the extent of the success of investigative reports.

The author constructs a new data set based on the thousands of submissions to the awards by Investigative Reporters and Editors (IRE) between 1979-2010. IRE is a grassroots non-profit organisation and its awards are one of the most prestigious badges of honour for investigative journalists. The book draws insights out of this extensive mapping of work in the IRE community and beyond.

The data shows what kinds of stories were reported, what was the impact of these stories was and how these stories were produced and supported. For anyone looking for granular details will come away impressed. There are lots of interesting tables and charts including top stories by topic, policy agenda coverage (stories covering the operation of a policy against stated goals), stories categorised by medium and by the kind of sources and track record of media owners in investigations via their news outlets. The information is also classified according to the impact of the stories categorised as individualistic (people getting fired), deliberative (resulting hearings, audits) and substantive (changes in law).

One of the chapters documents the life’s work of Pulitzer-winning reporter Pat Stith and addresses the impact a single reporter can have during 40 years of investigative reporting. In a chapter called ‘Accountability and Algorithms,’ the author lays out how computational journalism can sustain accountability journalism, which is sure to interest data geeks, reporters and funders.

For readers who wish to get a peek into the world of investigative newsrooms, how stories are produced, what the compulsions are, who supports these in-depth investigations, the book has a couple of chapters looking closely at these issues. If you have watched Spotlight and were intrigued, this book will answer some of your questions.

The dynamics of commissioning an investigation

The author says that the internet, the decline of major metro newspapers, a drop in data processing costs and an increase in non-profit news outlets, have impacted investigative reportage in the previous decades. It is well known that the death of local newspapers means the loss of thousands of years of local knowledge. Diminishing advertising revenues has led to a broken business model of journalism.

Investigative reporting has long been recognised as a public good, at least by journalists and their funders, if not so much by the readers. The author recognises that public goods and positive externalities are market failures that will be “underfunded in the marketplace”.

Positive externalities as a result of investigations could be changes in policies or government behaviour – say, better schools or reduced toxic exposure. The benefits, although enjoyed by the society, do not get captured or are not monetised by news organisations.

While using economic theories to explain investigative journalism, the author highlights three questions faced by editors – the costs of commissioning a story or an investigation, whether to pursue the story at all and why publishing the story could create a problem.

“Incidence, content and impact are key questions.”

Fixed costs, transaction costs and positive externalities determine the kind of stories that get commissioned. Costs could also include seeking legal advice. Further, he analyses a variety of factors that decide the future of the story including audience interests and “political economic boundaries of the institutions” in a story.

Measuring impact: benefit vs cost

There is a whole chapter dedicated to “impact,” much to the glee of frustrated fund-raisers wishing to write business cases for investigative journalism.

The most important headline figure for journalism funders and governments, which emerges from the author’s analyses, is that the investment of a single dollar in investigative journalism can result in hundreds of dollars in social benefits when policy changes occur as a result of the investigation. The related assumptions and calculations are discussed in the book.

He also draws attention to the dilemma faced by newsrooms – every successful investigation that is uncovered confers benefits even to the non-subscribers of the newspaper. Hence, in some sense, the returns are diffused, not accruing directly to the newspaper that generates original content at a great cost. In other words, the news organisation is unable to plough back those benefits into additional reporting sources.

What is at stake for parties in this business – the readers, the funders and the journalists?

There is a detailed discussion on the cost-benefit test from a society’s perspective, which I think is fairly central to this conversation. The reader plays a pivotal role. What trade-offs are the reader willing to make in order to make investigative reporting sustainable, the author asks. What utility does the consumer get from reading an investigation? Why should one read a long investigation as opposed to watching television?

He lists a few reasons why people would be interested in a certain kind of investigative story – it can improve the work they do (become informed about a particular industry), a story might affect consumer choice (food safety), it could entertain (scandals) and finally readers may want to be involved in civic engagement. Self-appointed media marketing geniuses, who are sure of what will not “sell”, will heartily agree.

This is a problem that he delineates well. There is a gap between what people as citizens need to know and what they want to know as readers, he says. The “private value” that people attach to being informed is lower than expected benefits. As a result, “rational ignorance” will prevail. People do not cast informed votes because people feel that the outcome of their single vote on an entire election is negligible.

From the perspective of media owners, especially those of the non-profit organisations, editors directly pursue these noble goals of informing the reader, since it might align with the mission of the funder. For a newspaper, doing investigative journalism might be an incentive to build a brand and visibility.

The cost-benefit analysis is clearly complex. But for anyone to sit up and take notice, the story must lead to policy changes. Social benefits will outweigh costs, even if it does not yield positive returns for the media company.

That is not all. The author takes the discussion a step further by highlighting real world outcomes as a result of investigations. What happens years after a story has been published? While mostly journalists have moved on to other stories, organisations do not have resources to track outcomes. He also highlights difficulties in drawing up counterfactuals or baselines from where journalists can draw “neat conclusions” on the impact of their work.

To illustrate the impact, the author has devised a few case studies based on investigative stories. Each one is deconstructed, in terms of findings of the story, costs, returns for the outlet, policy changes, policy impacts and returns to the society. There is a brief discussion on the estimates of lives saved and calculation of a value of a life, say, as a result of increased public safety, as a consequence of an investigation.

Among the many notable examples cited, ProPublica’s impact calculation is impressive. According to the author, ProPublica is driving conceptualising and measuring investigative work. ProPublica’s tracking report keeps a close watch on the impacts its stories are having. With a $10 million a year budget, its stories result in eight-ten significant reforms. This has to be one of the most successful investigative outfits today.

The future

In order to secure the accountability function of investigative journalism, the author pushes for government transparency. Public authorities must have policies of sharing information that recognise the “failures of public affairs reporting.” Governments can control what kind of created content will get subsidised and distributed as a result of their policies. Other interventions suggested include reviewing tax status for non-profits involved in investigative journalism.

His emphasis on the delights of computational journalism is studied and guarded. By computational journalism, he means “set of practices of reporters using larger datasets and more sophisticated algorithms to find patterns… using knowledge from social science.” According to the author, computation and journalism interact to change how stories are discovered, presented, aggregated, monetised and archived.

This innovative journalism can reduce costs of production in terms of finding stories and help personalise stories. There is particular emphasis on how the reportage around campaign finance can change as a result of using technology for stories.

For example, ProPublica used natural language processing and machine learning techniques to reverse engineer emails and find out how demographic factors affected campaigns seeking support.

In the post-Snowden era, algorithms take a key role in investigations. “The use and misuse of algorithms by markets and the government will need a counter set of algorithms to detect, describe abuse and deception,” he says.

India has its own data wars to resolve in the present and in the future ranging from privacy concerns to the truth about monies recovered from demonetisation.

If you feel lost in this brave new world of tech journalism, there is some sobering advice in the book as well.

According to Jeff Leen, a Washington Post editor who is quoted in the book, an ideal combination of sources that go to make a story is 50% sources, 30% documents provided by a tip, 10% from databases, 2% from social media and the rest from other sources.

There are limits to tiresome curation and breathless emphasis on multimedia. Good old-fashioned reporting is here to stay.

To be sure, this costs money, if that wasn’t already emphasised enough.

The author highlights the need for further research to combine insights from behavioural economics, crowdsourcing, philanthropy to find out how the donation of money, time and attention will affect accountability work. One example he suggests is for entrepreneurs to do well and do good is by using a “dual stock structure adopted by some tech and media companies allowing individuals with higher voting power, usually founders who can pursue objectives beyond profit maximisation in determining what kind of content to produce.”

This book says new age journalism will involve decisions by a diverse set of private actors including computer scientists, philanthropists, entrepreneurs, academics and reporters.