Internal Emails Show Twitter Struggled To Interpret Its Own Verification Rules While Hunting Trolls
Emails suggest that verification was never quite what Twitter said it was and that the company was aware it served as a tacit endorsement long before it admitted so publicly.
Twitter's January 2016 decision to unverify Milo Yiannopoulos, then Breitbart's tech editor, ignited a fierce debate among the company's top executives about how to enforce its opaque and ill-defined harassment and verification rules, according to documents obtained by BuzzFeed News.
The internal confusion and frustration arose after an email Yiannopoulos sent to Twitter CEO Jack Dorsey asking for his verification to be restored in exchange for a detente. The email, which was forwarded by Dorsey to Twitter executives, inspired a debate as to whether the company could or should reinstate the notorious troll’s blue checkmark. The problem? Few inside Twitter seemed to have a clear understanding of the company’s policy on verification or its meaning. Should Twitter reverse its decision or should it ban Yiannopoulos altogether — and which of those moves was best supported by the company’s policy?
“I thought that he wasn't qualified for verification under current guidelines - is that not true?” Vijaya Gadde, Twitter’s general counsel, wrote on the thread. “I want to make sure we are doing the right thing here and not responding to external pressure or attacks from him. We've already taken the PR hit, so let's make sure we are focused on getting this right!”
“If he is as bad as we think on abuse, why don't we just perma-suspend?” the VP of user services, Tina Bhatnagar, asked.
“It has become a cultural status symbol. It influences search ranking. It exempts a user from some spam filters. It gives them priority support treatment.”
“To my understanding, none of the violations taken individually warrant permanent suspension and while we have escalations for repeat offenses of some types, we don't have a blanket 'three strikes you're out'-type policy,” wrote Adam Sharp, then Twitter's head of news, government, and elections.
“I'd like to understand the verification policy and whether or not he is eligible or what would make him eligible. That should be an objective criteria in my view,” Gadde replied.
This exchange and others reviewed by BuzzFeed News provide a rare glimpse into Twitter’s years-long struggle to curb abuse and police trolling on its platform. The messages reveal that Twitter's rules — many of which have undergone changes in 2017 — were so vague and opaque that even the company's leadership struggled to interpret and execute them.
Indeed, the communications reviewed by BuzzFeed News suggest that within Twitter there was no clear consensus on what the company’s verification badge meant. The blue checkmark, first introduced in 2009, was supposed to prevent impersonation. But according to the emails, some inside Twitter viewed verification as both an endorsement and a badge of validity — especially among journalists and celebrities. Other emails reveal that verification bestowed upon users perks and status within the Twitter community. More broadly they suggest that verification was never quite what Twitter said it was and that the company was aware it served as a tacit endorsement long before it admitted so publicly.
“One challenge is how verification has morphed into something so much more than a well-intentioned identity check,” Sharp explained in a January 2016 thread that included top legal and communications team members and Dorsey, among others. “It has become a cultural status symbol. It influences search ranking. It exempts a user from some spam filters. It gives them priority support treatment.”
Such internally voiced concerns, however, appear to contradict Twitter's own public messaging around verification. Nearly two years after the the Yiannopoulos debate — after Twitter verified white nationalist and Charlottesville rally organizer Jason Kessler — a tweet from the company’s support account noted that “verification was meant to authenticate identity & voice, but it is interpreted as an endorsement or an indicator of importance.”
"Keeping a bad ... user verified means awarding them and their abusive content extraordinary privileges and visibility."
One employee argued that Twitter’s own internal metrics suggested a different meaning for the blue checkmark. “[Verification] makes the account measured for Media OKRs [Objectives and Key Results] and contributes to the VIT [Very Important Tweeter] count we report to shareholders,” Sharp wrote in an email to fellow executives, suggesting that verified users were valuable to the company. According to a person in a position to know, Twitter’s earnings reports included references to the number of celebrities and VIP users on platform, meaning that verification would boost those numbers.
“Under the current system,” Sharp continued, “keeping a bad (based on TOS violations, not a commentary on content) user verified means awarding them and their abusive content extraordinary privileges and visibility, as well as a disproportionate impact on how we measure platform quality.”
Reached for comment, Twitter pointed BuzzFeed News to public statements it made in 2017 as part of its pledge to be more transparent about its rules enforcement and interpretation — particularly when it comes to verification and account suspension. It also flagged other statements from executives apologizing for the confusion over verification. In recent months, Twitter has rolled out a public timeline for the implementation of new rules around unwanted sexual advances, nonconsensual nudity, hate symbols, violent groups, and tweets that glorify violence.
The emails also highlight a fundamental tension inside Twitter — the strain between the company’s desire to rid its platform of bad actors and its oft-professed commitment to a maximalist interpretation of free speech. In an email chain dated Jan. 8, 2016 — in which the decision to strip Yiannopoulos of verification appears to have been made — employees expressed their frustration with the company’s inaction in the face of Yiannopoulos’s rule violations.
“Based on my conversation with PartnerOps the other day, it sounds like [Yiannopoulos] technically does meet criteria for verification as a journalist (though we're unable to verify policy sign-off),” a Twitter senior policy manager wrote. “That said, we all seem to be in agreement that he's a notorious troll and that we'd be comfortable de-badging (or at least issuing a warning) if/when he crossed the line again.” They continued with a list of Yiannopoulos’s latest violations:
As a reminder, during the last ~2 weeks alone, he has falsely identified himself as a Buzzfeed Editor in his bio (prompted an emergency Media Team escalation)
-tweeted @KanyeWest naked pictures of his wife and insinuated that she's a whore
- continued his standard practice of writing hit pieces (factually accuracy unclear...they've linked to sites like Encyclopedia Dramatica and NCN content in the past)
- tweeting directly at the target of his piece to taunt them about it (target has typically already has him blocked) and subtly inciting mob harassment AND/OR
- posting screenshots of people who have blocked him, mocking that person, and tagging them anyway to subtly incite mob harassment (see latest example in email below)
- tweeted this https://twitter.com/Nero/status/685148564742389760 (still live) which could be interpreted as an attempt to intimidate/silence women/a protected group... and many more examples.
“It is *extremely* difficult to justify/rationalize his verified status (the fact that he's still on the platform at all is a tough sell too),” the policy manager wrote. “We need to be better than this.”
While some messages show executives agonizing over the interpretation and consistent enforcement of the rules around verification, other messages suggest suspension and de-verification was at times something of a judgment call. In a Jan. 16 email, Bhatnagar, the VP of user services, told Dorsey and other executives that the 2015 suspension of right-wing troll Chuck Johnson could provide a precedent for suspending Yiannopoulos.
“Per our new enforcement policies, [Yiannopoulos] is consistently in violation but never of direct violence (which is what we perma suspend for). So if we can take the stance to debadge, then why can't we take the stance to perma-suspend?” Bhatnagar wrote. “We perma suspended Chuck Johnson even though it wasn't direct violent threats. It was just a call that the policy team made. He is finding loopholes in policy which is almost worse than the people who blatantly have violations.”
"We need to be better than this."
In a subsequent email, Gadde, the general counsel, also referenced a May 25, 2015, email from Twitter's then-CEO Dick Costolo to the company’s operations team, which suggested the decision to make Johnson’s suspension permanent was made at Costolo’s discretion. "As for Chuck Johnson - Dick made that decision," Gadde wrote before copying the text of Costolo's email to the chain.
“To be very clear, I don't want to find out we unsuspended this Chuck Johnson troll later on,” Costolo wrote. “That account is permanently suspended and nobody for no reason may reactivate it. Period. The press is reporting it as temporarily suspended. It is not temporarily suspended it is permanently suspended. I'm not sure why they're mistakenly reporting it as temporarily suspended but that's not the case here...don't let anybody unsuspend it.”
Costolo did not respond to a request for comment.
Despite ongoing debates over Twitter’s rules for policies around verification and suspension, company executives seemed befuddled and, at times, hamstrung when it came to enforcing them. Even after the company decided to strip Yiannopoulos of his verification, Dorsey appeared dissatisfied with Twitter’s interpretation of its own policies. “Think a debate worth having is suspension versus removal of badge per harassment policies. Feels like we mixed the concerns a bit here,” he wrote to other top employees.
Despite Yiannopoulos’s ongoing bad behavior on the platform and violations of Twitter’s rules, it would take another seven months before Twitter ultimately suspended him; the final straw for the company was the targeted harassment campaign he brought against actress Leslie Jones. Shortly after Yiannopoulos’s suspension, Dorsey sent an email to Twitter employees pledging to make safety “a top-level priority of the company.” Signing off on the email, Dorsey tried to rally employees, asking them to stay positive:
“It's natural during times like these to focus on the negative. I was listening to a podcast on my walk to work this morning that referenced this Gandhi quote:
"‘You must not lose faith in humanity. Humanity is an ocean; if a few drops of the ocean are dirty, the ocean does not become dirty.’
"Let's make sure we continue to remember that.”