News stories published mostly by small- to medium-sized outlets boosted Twitter conversations about those topics by about 63% across the political spectrum, suggests a five-year experiment released on Thursday.
Twitter has 328 million active users — journalists and news hounds, politicians and public agencies, activists and Russian trolls — a real-time cacophony that has become the water cooler of the internet.
The platform has also become a laboratory for social scientists trying to tap into the national conversation. In the new study, published in the journal Science, researchers enlisted 48 news outlets to participate in a five-year experiment in which they agreed, on certain weeks, to publish stories about topics chosen by the scientists, in a bid to understand the impact of their work.
As it turned out, the impact was significant: The stories were not only talked about, but shifted people’s opinions ever so slightly toward the political outlook of the news outlets, the study found.
“What this tells us is that journalists bear considerable responsibility for a crucial part of American democracy,” study lead author Gary King of Harvard told BuzzFeed News. “This is not just another job. It really matters.”
Although it’s common sense that the news has some effect on society, measuring its precise impact on political chatter has been tough, King said. “This was much bigger than we had realized or previous scholars had understood.”
The participating media ranged from general-audience outlets like The Huffington Post to niche publications such as Yes! Magazine, Feministing, and Alternet. Most skew liberal, according to the researchers.
On 35 separate weeks, several outlets would collaborate with each other on a story about one of the selected topics, such as abortion, immigration, jobs, and climate change, as well as publishing their own articles. Then the researchers measured all of the tweets that linked to or mentioned the articles, whether favorably or not.
All of the stories were real news the outlets produced and promoted in normal fashion. Apart from telling the outlets which topics they were focusing on in a given week, the study team did not influence what, or how, the news was reported. (BuzzFeed News was not part of the experiment.)
King said most of the study team’s time was was spent negotiating cooperation with the news outlets. "I learned every time they started to talk about 'journalistic integrity' I had crossed a line, and needed to start the conversation over," he said.
To make everybody happy, the researchers eventually followed the collaborative model of the 2016 “Panama Papers” investigations, in which a consortium parceled out documents detailing dodgy offshore investments to various outlets, allowing them to freely pick ones for stories, an effort that won the Pulitzer Prize in April. The funders of King’s study, The Media Consortium, a “progressive” media association, also paid outlets for their participation.
“This is the kind of research design most of us would only dream about launching.”
Pulling off a study this long, complex, and expensive was a feat, other researchers said. “This is the kind of research design most of us would only dream about launching,” political scientist Michael Jones of Oregon State University told BuzzFeed News.
Overall, a burst of stories increased tweets on the topic by 63% — an extra 13,000 posts over the selected week — compared to a week without them, the study found. The tweets were spread evenly among users from all over the country, of all genders and political allegiances, with few or many followers — arguing against the notion of an “echo chamber” bottling up news to like-minded communities, the researchers said.
The tenor of the political opinions voiced in these tweets was graded by an automated text analysis computer program that detected supportive, critical, or indifferent words in those posts. The stories shifted the balance of opinions about the subject area by about 2.3% toward the ideology of the news outlets (as rated by the study authors). It was a small effect, the researchers admitted, but one that could add up over time.
“Basically what they have put a number on is an assumption that many in [journalism] already take for granted, but there is something to be said for them actually being right,” Jones said. “Plenty of times people think something is obvious but it turns out they are wrong.”
The results support the conventional view in political science about news outlets setting the agenda in national political discussions, King said, but not driving the outcome of political debate.
One caveat of the study is that Twitter is only a small part of online political discourse, compared to Facebook’s 2 billion monthly active users. Facebook has had a starring role in the ongoing debate over Russian interference in the 2016 presidential election, with fake news stories and troll farms exploiting divisions in the US electorate. But Twitter is part of it, too: One Russian troll account claiming to represent the Tennessee Republican Party since 2015, for example, acquired 136,000 followers on Twitter by tweeting out bogus news stories.
Communications scholar Dietram Scheufele of the University of Wisconsin applauded the new study’s design, but questioned how much it really measured genuine shifts in opinion. “The assignment really tells us that if we put stories out, they’ll get retweeted, if we don’t, they don’t,” he told BuzzFeed News by email. “If they get retweeted, it’s likely by people that agree with it.”
Still, King argued that talking about something, on Twitter or elsewhere, was at the very least an essential step to getting politicians to pay attention to a problem, making the study results relevant: “What are the odds of them passing a bill about something that nobody ever talks about?”
UPDATE
The original version of this article stated that Public Radio International participated in the new Science study. That was based on an error in the study that the journal is now correcting. (PRI republished an article from one of the media outlets involved in the experiment, but PRI did not know about the experiment.)