A case study comparing two community newspapers finds that the paper more focused on audience metrics published fewer stories about civic issues, used fewer sources and let reader traffic guide news judgment to a greater degree than the paper that viewed analytics as a secondary consideration.
“I think journalists everywhere — at the management level, at the reporter level — are struggling with this tug of war between the need to create traffic and the need to fulfill the civic mission of journalism,” lead author Tom Arenberg said in a phone call with Journalist’s Resource. “Sometimes those two go hand in hand. Most times they don’t.”
“The Impact of Web Metrics on Community News Decisions: A Resource Dependence Perspective” was co-authored by Arenberg, an instructor at the University of Alabama, and Wilson Lowrey, a professor there. Their paper, published in Journalism & Mass Communication Quarterly, takes a mixed-methods approach to its analysis, incorporating interviews, survey responses, newsroom observation and analysis of the publications’ content.
The authors selected two similar news organizations to study, which were both given pseudonyms in their paper: “One organization, The Heavy Metrics Herald, as part of a chain-wide strategy, freely incorporates metrics into its operations, both in creating content and in assessing results after publication,” they write. “The other organization, The Light Metrics Ledger, also uses an analytics software program, but analytics is a secondary consideration, due to financial limitations and to ownership/management preference.”
In terms of location and newsroom size the organizations were similar. They were both dailies based in the southeastern United States, in cities with populations between 50,000 and 55,000. The Herald had a newsroom of 18 (including four editors/managers), while The Ledger had 15 on staff (including three editors/managers). Both papers were owned by media chains, though The Herald was owned by a large, national chain, while The Ledger operated as part of a small, family-owned chain. In 2016 The Herald had 24.7 percent Sunday print penetration in its home county, a rate comparable with The Ledger’s 25.1 percent.
The researchers observed the newsrooms’ meetings and everyday work for five consecutive weekdays during the summer of 2016. In addition, they examined documents, including the news published during the study period as well as metrics reports and daily story budgets. Surveys and interviews probed the journalists’ views of what makes a story newsworthy, what they perceived as their paper’s priorities and how deep their stories typically go.
Here’s what they found:
- “Metrics appear to play a greater role in determining story newsworthiness at The Heavy Metrics Herald than at The Light Metrics Ledger.”
- For example, the researchers asked journalists at both papers whether they agreed or disagreed with the following statement: “I am not inclined to pursue a story if I think it will generate low traffic numbers.” At The Herald, five journalists disagreed with the statement, while 12 disagreed at The Ledger.
- In interviews, Herald journalists gave further evidence of making news decisions because of numbers, saying: “[If] nobody’s reading [particular public-issue topic], why should we pursue it?” and “If people aren’t following the Chamber of Commerce, in a year or two we won’t cover them.” Interviews with Ledger staff did not yield such sentiments. Further, “All 15 Herald respondents to the brief questionnaire agreed or strongly agreed that their outlet prioritizes stories that gain reader clicks, compared with 6 of the 15 Ledger”
- Further evidence of the influence of metrics on publishing decisions at The Herald included the paper’s practice of “seeking follow-ups to popular stories, reducing follow-ups to numerically disappointing stories, de-emphasizing a coverage beat or a particular institution on a beat, and changing the nature of stories sought on a beat.” However, Herald managers also inquired about the newsworthiness of follow-ups, which suggests that traffic was not their only consideration. At The Ledger, “routine reviews of top traffic stories did not lead managers to request additional stories on popular topics.”
- Across the board, journalists said poor metrics wouldn’t keep them entirely from pursuing stories that could help serve the public. “Overall, 12 of 15 journalists at The Herald and 13 of 15 at The Ledger agreed/strongly agreed that expected low traffic numbers would not stop them from pursuing a story that could benefit their communities,” the authors write.
- All told, The Ledger produced a higher percentage of civic and public issue stories (33.3 percent) during the study period than The Herald (25.5 percent).
- The Ledger’s staff stories included 50 percent more sources than The Herald’s – 2.23 sources per story compared to 1.49. For the purposes of this study, sources included human sources, documents, attributions to other media and citations to social media. A similar trend held for sourcing in issue stories. Quantitative survey results suggest that the staff at these organizations were aware of these patterns: “In the survey, only 4 of 15 Herald staff said they disagreed/strongly disagreed that their stories included too few sources, compared with 9 of 15 at The Ledger.” The authors suggest a few explanations for these differences, including the fact that the Ledger had a rule mandating at least two sources for print stories. Additionally, Herald staff produced more content and a few interviews mentioned “low-hanging fruit” stories used to generate metrics and keep up posting volume.
One possible limitation noted in the paper stems from the difference in ownership of the two papers. However, the paper suggests there was “no evidence” that ownership had an impact on the metrics and content strategy of the paper, “such as one paper having a stronger advertising unit or closer editorial-marketing relations.”
And while the study is limited in that it is a direct comparison of only two outlets, Arenberg suggested that the mixed methods help to validate the findings. Incorporating observation and interviews into the analysis allowed him “not just to measure results, but to sort of witness how this came about, what the mechanisms were,” he said.
Further, Arenberg cited his background as anecdotal support for his findings: “Before I began my second career in academics, I was a professional journalist for 35 years. In recent years, of course, all newsrooms have begun to lean on online metrics as part of their operation, as part of their business model,” he said. “I’ve witnessed this, and I hear similar issues — these same issues I’ve found in the two organizations I studied.”
“I think it’s fairly clear at this point that the idea of building a business model upon selling digital advertising based on a large audience has not worked nearly as well as the journalism industry had hoped,” he continued. “Clearly there’s some rethinking going on, but at least for the moment, and the foreseeable future, there’s no new business model.
“Numbers decide what’s news and what’s not,” Arenberg said. “It’s essential journalists be aware of it.”
Expert Commentary