Kathy Kostrub-Waters and Bryan Denny estimate they've spent more than 5,000 hours over the past two years monitoring Facebook to track down and report scammers who steal photos from members of the US military, create fake accounts using their identities, and swindle unsuspecting people out of money.
During that time they reported roughly 2,000 fake military accounts, submitted three quarterly reports summarizing their findings to Facebook, and even met with Federal Trade Commission, Pentagon, and Facebook employees to talk about their work.
Kostrub-Waters and Denny aren’t security or fraud consultants, nor do they have any official relationship with the company. They do it all in their spare time.
Nearly two years into their efforts, Denny and Kostrub-Waters have become concerned they seem to know more about the problem than Facebook does, and they’re frustrated that the company hasn’t gotten better at solving it on its own.
“It seems like every time we tell them something, they had no idea or didn’t know that was possible,” Denny said. “You can’t tell me that you don’t know some of this. I mean, this is your business, right? This is stuff me and Kathy are doing in our spare time because we are committed to it at this point. But every time Kathy tells them something, it’s like a revelation.”
Facebook told BuzzFeed News it recognizes it has “more work to do.”
“Impersonation hurts real people and goes against what we stand for. It is often driven by organized scammers who are financially incentivized to keep adapting, which is why we are working with law enforcement and investing to help stay a step ahead,” said Scott Dickens, a Facebook product manager, in a statement. “We are very grateful to Kathy and Bryan, who have met with us on multiple occasions, and tested some of the new techniques we are still refining. Their frustration is understandable and we know we have more work to do.”
Denny and Kostrub-Waters are part of a loose global network of victims, researchers, amateur sleuths, activists, companies, military officers, and others who’ve been pressed into service to help clean up the world’s biggest social network. They call out scams using their own profiles and pages, report bots, fake accounts, and hate speech to Facebook’s community standards team, and debunk misinformation. They include Felicia Cravens, a former tea party activist who now spends her free time exposing fake accounts and making videos to show other people how to do it. She and Kostrub-Waters are both regularly in touch with Sarah Thompson, an Indiana woman who runs a Facebook page dedicated to calling out clickbait, false news, and fake accounts.
Along with individuals, the US Army and businesses such as Publishers Clearing House also find themselves dedicating staff resources to removing fake profiles and scams from Facebook.
“We have been drafted — shanghaied — into helping fix their platform,” Denny, a retired 26-year veteran of the US Army, said in an interview with BuzzFeed News.
His photos are used so frequently by romance scammers on Facebook that one publication recently dubbed him “the face of military romance scams.” That’s how he and Kostrub-Waters met: A friend of her mother was scammed by someone using stolen photos of Denny. The impostor pretended to be in the US military and after gaining the woman’s trust began asking for money to help with a series of fake emergencies. He managed to steal $35,000 from her using this scam, which is widespread and has seen $884 million stolen from Americans since 2015, according to data from the Federal Trade Commission and the FBI.
Denny says he’s so inundated with Facebook messages from women who’ve been scammed by someone using his photos that he’s created responses he cuts and pastes into messages. He also reaches out to people interacting with fake accounts, all while holding down a job with a defense consultancy in Virginia.
“I’d see the same [fake] profiles every day, and then try to write the people who were friends with those profiles and say, ‘Hey, you’re talking to a scammer.’ It was really fighting an uphill battle,” the Iraq and Afghanistan veteran said. “I’m not trained to quit fighting, so we continue to do what we need to do.”
Facebook announced last year that it’s doubling its security and safety operations team to deal with people and content that violate its policies. But content moderation teams are primarily focused on responding to reports from users, rather than hunting down bad actors. Facebook’s use of artificial intelligence to detect and stop abuse is also still in its early stages.
This results in people and companies performing unpaid labor for one of the world’s most valuable companies, as well as for other social platforms such as Twitter.
Denny says their contacts at Facebook always express gratitude for the work he and Kostrub-Waters do, but it seems to end there.
“The reality is this isn’t my job or Kathy’s. So I get really tired of ‘Hey, thanks for the work — keep it up.’ I’m like no, damnit, it’s your job to do this,” he said.
Kostrub-Waters recently tested one of the new techniques that Facebook is rolling out to fight impersonator accounts. The company told her and Denny that 22 of his often-stolen photos had been flagged in a way that should prevent them from being used to create new accounts.
So Kostrub-Waters created a fake account with the name “Waters Kathy” and began adding photos of Denny. Not only was she easily able to set up an account with some of Denny’s most-stolen photos, but also the platform suggested friends who nearly all hailed from Nigeria — a country from which a significant number of romance scammers originate.
“When I started building the account more, that’s when I started noticing the Nigerian recommendations and I was like, ‘What in the world?’” said Kostrub-Waters, who lives in Fresno, California, with her family and recently left a job in the health care industry.
Facebook’s top suggestion for the hometown of “Waters Kathy” was Benin City, Nigeria, and one of the suggested places of employment was a Nigerian oil company.
Kostrub-Waters and Denny came to the same conclusion: His photos had become so abused by Nigerian romance scammers on Facebook that the company’s artificial intelligence now linked his image with that country.
Facebook didn’t comment on the record about the Nigerian recommendations but did say that the system used to flag Denny’s photos is in the early stages of deployment.
“The system that Kathy and Bryan have described, which is designed to try to stop new impersonating accounts from being created, is not something we have rolled out broadly across Facebook,” said a company spokesperson. “This is one of several techniques we are using to combat impersonation, and we are still working to refine it and improve its scope and accuracy.”
Denny said he wasn’t surprised that Kostrub-Waters could easily set up an account, even with Facebook’s assurances that 22 of his photos had been flagged to prevent their use in new profiles.
“I’m painfully aware that anybody can establish an account with my name and photos,” he said.
Scammers have also so frequently used photos of Gen. Mark A. Milley, the US Army’s chief of staff, to create fake accounts that Facebook flagged his photos in the same system. But Kostrub-Waters was still able to create a test account with the general’s photos, though she was not served the same Nigerian recommendations.
“We have met with Gen. Milley’s office, along with many others within the military, to discuss these issues, offer training for impostor reporting, and establish channels to receive and act on reports of impersonating accounts,” said a Facebook spokesperson.
As of today, the duties of some US Army Public Affairs officers now include spending time identifying and reporting fake profiles of senior officers like Milley. The Army has also issued public warnings to try to stop people from falling for military romance scams, and its Criminal Investigation Command is actively working to fight impostor accounts.
“We get hundreds and hundreds of calls Army-wide about this,” Chris Grey, the chief of public affairs for Army CIC, told the Daily Press newspaper in Virginia. “Myself, at my desk, I probably get three a week.”
Close to a dozen employees at Publishers Clearing House, which gives away millions of dollars and prizes in real sweepstakes each year, are also on clean-up duty for Facebook. The company has become a favorite target of sweepstakes scammers who steal photos of real PCH employees in order to create fake accounts on Facebook and elsewhere. (Facebook’s CEO also has an impostor problem: The New York Times recently reported on fake Mark Zuckerberg accounts that litter the platform and are used to run sweepstakes scams.)
“We report those accounts on a weekly basis, and sometimes on a daily basis, to Facebook to say, ‘Here’s what’s going on on your platform,’” Christopher Irving, the assistant vice president of consumer and legal affairs for Publishers Clearing House, told BuzzFeed News.
He said sweepstakes scammers have typically used the phone and mail to target victims, but “over the past year it has moved on and included social media, with Facebook being one of the main platforms.”
“You’re saying your site is secure, so then why are you depending on your own consumers to take care of the security?"
Facebook will quickly remove accounts impersonating PCH and its employees when notified, according to Irving. The problem is the fake accounts keep being created, which requires his colleagues to spend more time reporting them.
“Where they have a big challenge is the inability to prevent these from being posted in the first place,” he said of Facebook.
The company says it has rolled out a number of initiatives to automatically identify different types of fake accounts, including facial recognition and machine learning. It also launched a dedicated form people can fill out to report impostor accounts.
But by Facebook’s own admission, automated efforts are not yet working as well as it would like. This means people like Kostrub-Waters and Denny remain critical to Facebook.
The reality of how much the company relies on the labor of its users and others hit home for Kostrub-Waters when Zuckerberg was asked about fake accounts by Sen. Chris Coons during his testimony before Congress. The senator described how he found a fake account in his name on Facebook and had his staff report it to get the impostor removed.
“My core question is: Isn’t it Facebook’s job to better protect its users? And why do you shift the burden to users to flag inappropriate content and make sure it’s taken down?” Coons asked.
Zuckerberg said that “this is an area, content policy enforcement, that we need to do a lot better on over time,” and admitted that “the main way that this works today is that people report things to us and then we have our team review that.”
“You’re saying your site is secure, so then why are you depending on your own consumers to take care of the security? That hit me wrong,” Kostrub-Waters said. “It’s kind of scary.”
Even more alarming to her and Denny was Facebook’s announcement that it would be launching a dating service. “It’s heartbreaking to know that this dating site is supposed to be starting,” Kostrub-Waters said.
“You can’t fix this problem but you can spend time and energy crafting a dating service,” Denny said. “I mean, what’s your priority here? Facebook already is a dating service — it’s populated by scammers taking money from people.” ●