Shoddy Coronavirus Studies Are Going Viral And Stoking Panic
Scientists are rapidly posting findings about the new coronavirus outbreak online, accelerating the speed of scientific discoveries — and of misinformation.
Last Friday morning, after a week in which the coronavirus outbreak had been declared a global public health emergency, a group of scientists from India posted a paper online. A handful of genetic sequences in the new coronavirus matched those found in HIV, they reported, suggesting that this “uncanny similarity” meant the two diseases were linked.
A scientist in India blasted out the provocative finding to his more than 200,000 Twitter followers: “They hint at the possibility that this Chinese virus was designed [‘not fortuitous’]. Scary if true.” A Harvard researcher with tens of thousands of followers called it “very intriguing.” The official-looking, highly technical paper whipped dozens of onlookers into a frenzy, declaring on Twitter and at least one blog that it showed the virus was “man-made” and “not natural” and “prob. not random.”
For the second time in as many weeks, a segment of social media was freaking out over a coronavirus study that hadn’t been reviewed by experts or published in a journal. It was a “preprint,” or a preliminary draft, published on BioRxiv (pronounced “bio-archive”), a free repository that hosts thousands of unvetted papers about the biological sciences.
Preprint servers bypass the long, arduous timelines of traditional, peer-reviewed scientific publishing, and can lead to lightning-speed information sharing during outbreaks like this one. But the coronavirus is also bringing to light the pitfalls of this new system for the first time, as everyone from bad actors to naive ones grasp for new information in a panic-driven climate.
The “uncanny” paper was withdrawn by its authors on Sunday, putting an end to an undeniably messy situation that spread misinformation about a little-understood virus that has so far sickened upward of 20,600 people and killed more than 420, the vast majority near the outbreak’s epicenter in Wuhan, China.
Yet it was also a case of new research being shared, debated, and corrected in real time — something that many scientists have long wanted to see happen. Many still say that the benefits of instant scientific publishing far outweigh its downsides.
“This is exactly what BioRxiv is intended to do — it’s intended to be a place for timely discussion of results,” Russell Neches, a postdoctoral researcher who studies viral ecology at the Joint Genome Institute in Berkeley, California, told BuzzFeed News. An advocate for what’s called “open science,” where studies are freely accessible instead of hidden behind paywalls, Neches put a call out on Twitter over the weekend, asking scientists to give feedback on preprints about the new coronavirus, now numbered at more than 300 and increasing by the day.
Since the “uncanny” preprint went up, a swarm of researchers have left critiques in its comment section, posted comments on other preprints, and tweeted out their observations about the findings. Neches has been gratified to watch the corrections to the "uncanny" preprint happen but also wishes more experts had responded in the immediate hours after it went live.
“Part of what made the conspiracy theories really grow legs is the amount of time that passed where that preprint was up and getting a lot of attention, but still had no comments on it from scientists,” Neches said. If the article had been immediately debunked, people “probably wouldn’t have gotten so excited about it.”
The short-lived but fierce furor underscores how all research about the coronavirus right now — whether it takes the form of a preprint or not — is rapidly evolving. Last week, an article in one of the world’s most prestigious journals, the New England Journal of Medicine, made waves with its findings that someone with no symptoms can still be contagious. Fearing that the virus could be spread more widely than previously thought, US health officials cited it when declaring a national public health emergency last Friday. But within days, major flaws in the study became apparent.
Science is always subject to ongoing debate, something scientists proudly hold up as a feature, not a bug. But in an outbreak that’s playing out in unprecedented ways online, preliminary findings — some of which don’t stand up to scrutiny — can suddenly find a huge audience.
Traditional scientific publishing moves slowly. Researchers submit their findings one by one to journals. Then the manuscript goes to a handful of outside experts who give feedback and sometimes ask for more experiments.
If the final draft is accepted, it goes to press, usually appearing online first — albeit behind a paywall, making it inaccessible to many. A lot of scientists complain about the process being too long, sometimes biased, and riddled with inefficiencies. Peer review is notoriously inconsistent at catching errors, and retracting flawed studies can take months, even years.
BioRxiv, founded in 2013, is trying to significantly speed things up. Scientists submit their early drafts to the website, where an in-house team weeds out “obvious spam and clear garbage,” as well as submissions that aren’t scientific in nature, according to cofounder Richard Sever. All the papers that go up have disclaimers about how they’re not peer-reviewed and shouldn’t be treated as definitive.
A newer repository, MedRxiv, operates similarly, although with a few more hurdles to posting given the higher risks of publishing information about people’s health. Scientists must declare they have patient consent and approval from an independent ethics board. And, Sever says, MedRxiv staff screen out more studies that they believe should go through peer review first — for instance, a study that suggests that vaccines are dangerous and could be weaponized by anti-vaccine groups.
Preprints have only been around for a few years, but their benefits to the scientific community are already becoming clear. In 2017, after a cancer researcher spotted a preprint and contacted the scientist leading it, the pair started doing experiments together and published their results — seven months earlier than if they’d waited until the original study appeared in a journal.
And in the case of the coronavirus outbreak, preprints have helped get the word out about existing drugs that might be effective for treating patients.
But the nature of an outbreak unfolding live on social media means that all results, especially the most sensational ones, can go viral before they’re vetted.
One such preprint appeared on Jan. 24 on MedRxiv. It estimated that the coronavirus’s contagiousness, as measured by a variable called R0, was 3.8. That meant that — absent any interventions like quarantines or hospitalizations — every person who caught the disease could give it to almost four other people. That statistic went viral after Eric Feigl-Ding, a health economist affiliated with the Harvard T.H. Chan School of Public Health, tweeted about the preprint.
“HOLY MOTHER OF GOD — the new coronavirus is a 3.8!!!” he tweeted. “How bad is that reproductive R0 value? It is thermonuclear pandemic level bad — never seen an actual virality coefficient outside of Twitter in my entire career. I’m not exaggerating.”
Meanwhile, many other scientists on Twitter — including virology experts, which Feigl-Ding is not — called out the study’s clear methodological issues and missing context, as well as Feigl-Ding’s sensationalist and inaccurate tweets. They pointed out that R0 is also not the final word on how dangerous a virus is, but rather a snapshot in time that changes as measures are taken to limit its spread. And by the time Feigl-Ding started tweeting, unknown to him, the researchers were already revising the R0 paper to have a lower estimate.
Then, on Friday, a group of scientists in New Delhi, India, uploaded to BioRxiv their preprint suggesting that the virus was linked to HIV. Once again, Feigl-Ding, who by this point had accumulated tens of thousands of followers in a matter of weeks, tweeted out the sensational findings. In nine tweets that reverberated around the internet, he quoted extensively from the study, noting that it was “not peer reviewed”: “Let’s not draw conclusions yet.” At the same time, he called the findings “very intriguing” and “bold.”
The HIV-coronavirus paper, too, drew swift criticism. Matthew Frieman, of the University of Maryland School of Medicine, told BuzzFeed News that the overlapping gene sequences the study’s authors linked to HIV are found in all kinds of viruses. And Amesh Adalja, of the Infectious Disease Society of America, said that even if the sequences look similar to ones in HIV, “that doesn’t tell us anything new about this coronavirus.”
Feigl-Ding deleted both of his tweet threads about the studies, though not before they were retweeted thousands of times. And the results had quickly spread to other parts of the internet, too. The blog ZeroHedge cited the HIV preprint, as well as Feigl-Ding’s tweets about it, to advance a conspiracy theory that the coronavirus might be an “artificially created bioweapon.” (The site was recently kicked off Twitter for, according to the social network, “violating our platform manipulation policy.”)
During fast-moving outbreaks like this one, critics say that traditional gatekeepers — like peer review or journals or the press — can get in the way of giving people the up-to-date information they need.
In an interview, Feigl-Ding said he views social media as a crucial way to explain science to laypeople who don’t read, or believe, traditional news sources. “There’s a whole confluence of different reasons that we need to explore alternative ways of communicating directly with the public,” he said.
The events of the last week have made Feigl-Ding come up with a new social media policy, he said, one that identifies preprints as the problem. “I’m not going to tweet about preprints going forward, until someone else who’s an expert has tweeted about it,” he said, adding, “There’s lots of things in life you wish you could take back.” (A Harvard spokesperson declined to comment on Feigl-Ding’s tweets.)
Maia Majumder, a biostatistician and epidemiologist at Harvard Medical School and Boston Children’s Hospital, said that now that the public is showing interest in preprints, scientists could take steps to proactively manage the conversation. For example, preprint pages could be covered with large watermarks saying that they’re preliminary, rather than just having a disclaimer at the top.
And if study authors share their preprints on Twitter, “I believe that myself and my colleagues have an obligation to frame the research in a way that is accessible to the audience at hand,” Majumder said by email.
Steven Salzberg, a professor of biomedical engineering, computer science, and biostatistics at Johns Hopkins University, was among the critics of the HIV paper. That evening, he tweeted: “and the conspiracy theorists have now learned to post papers on @biorxivpreprint … great.”
But even he thought “that this event is a terrific illustration of the value of preprint servers,” Salzberg said by email on Monday, after the study was retracted.
“The paper was written too quickly, to respond to a crisis, and the authors submitted it without giving it the kind of scrutiny it needed,” he wrote. “If they’d gone to a journal, they would have waited months to learn about the flaws.” Online, he pointed out, the correction process wrapped up in just a few days.
Just because scientists may consider the matter to be settled, though, doesn’t mean the public does. Even after the preprint was withdrawn, it’s continued to circulate on Twitter.
On Monday night, one user directed their 18,000 followers to it: “Why is the MSM trying to downplay this information about how the Wu Flu was made in a lab for sure…”
Dan Vergano contributed to this story.