Scientists Are Doubting A Famous Study That Claimed You Can Be Easily Tricked Into A False Confession

On the night of April 19, 1989, Trisha Meili, a 28-year-old white woman, was jogging in Central Park when she was severely beaten, raped, and knocked into a 12-day coma. Five black and Hispanic boys from Harlem were convicted and served time based on confessions that later turned out to be false and, they said, coerced by police.

As in the case of the Central Park Five, people can be tricked into remembering crimes they didn’t commit. But there’s a brewing debate in the psychology field over just how easily this can happen.

In 2015, a highly publicized study found that over the course of three misinformation-laden interviews, researchers convinced a staggering 70% of participants that they’d committed a crime.

But now critics suspect that 70% figure is far too high, arguing in a new paper that the original study poorly defined “false memories” and overlooked signs that the volunteers didn’t actually remember the invented crimes. Other research suggests the rate is closer to 20%.

In an unrelated development, BuzzFeed News has also learned that the 2015 paper, which appeared in Psychological Science, will be formally corrected for statistical errors. These inaccuracies do not affect the study’s major conclusions, however, and the lead researcher, criminal psychologist Julia Shaw, says that she stands by her findings.

There are many ways to define false memories, she says, and hers was not necessarily incorrect. “[O]nly participants can know whether or not they are having the subjective experience of remembering a past experience,” wrote Shaw, an honorary research associate at University College London, in a forthcoming response to her critics.

Kimberley Wade, a psychologist at the University of Warwick in England and a co-author of the critique, doesn’t dispute that it’s possible to develop false memories of committing a crime — but she doesn’t think it’s as easy as Shaw’s paper suggests.

“The problem with reporting inaccurate, inflated results is this: It fuels skepticism of memory research and can detract from how we understand why people develop false memories,” she told BuzzFeed News by email. “It can slow science down and even lead scientists to draw erroneous conclusions about real-world behaviour.”

When Shaw published the 2015 study with Stephen Porter, a forensic psychologist at the University of British Columbia, their 70% finding was covered by the likes of NPR, Wired UK, and the Daily Mail, and Shaw went on to publish a popular book, The Memory Illusion (which she wrote about on BuzzFeed).

“I was incredibly surprised at the rates that I had in terms of successfully implanting these false memories,” Shaw told the PBS documentary Memory Hackers. “And yet there we were, and they just kept coming and coming and coming.”

The experiment involved 60 college students. First, researchers gathered real facts about the participants in their teenage years, like the cities where they lived and their friends’ names, through questionnaires sent to their parents or guardians.

The volunteers then sat through three 40-minute interviews over three weeks. Researchers asked them about a true event in their adolescence (as provided by their parents), then a fake event. Several were told they’d committed a crime that led to the police being called, ranging from theft to assault with a weapon.

The interviewer asked them to explain what happened, while dropping cues — like how old they supposedly were at the time, and what time of year it took place — meant to convince them that the fabricated crime occurred. And when volunteers struggled to remember, the interviewer told them, falsely, that they could remember if they tried hard enough. In the end, they found that 70% of them described false memories of committing the crime.

The researchers labeled a volunteer’s account as a “false memory” only if several criteria were met, such as not remembering the crime at first but recalling it later. The volunteer would also have to give at least 10 details, including more details than the interviewer had provided. And when finally told that the crime hadn’t actually happened, and asked, “Did you believe that you had forgotten the event and that it actually happened?”, the volunteer would have to answer “yes.”

This definition of a false memory is “very conservative,” Shaw and Porter wrote. And their results were “essential in the quest to help prevent memory-related miscarriages of justice,” they concluded.

But others think Shaw and Porter’s definition was in fact too loose. In a critique published in January, Wade and two other researchers pointed to other research indicating the rate is more like 22%.

Shaw shared the experiment’s raw data with Wade’s team, who then had two independent judges grade the interview transcripts by two sets of criteria — not Shaw’s — that have been used to categorize false memories. The judges evaluated whether the subjects spoke with confidence about their memories, such as by saying, “I’m certain this happened” or describing emotions related to the event.

Under the two new sets of criteria, the rates of false memories fell to 26% and 30%, respectively, in line with previous research. Much of what had been counted as richly detailed false memories ended up looking more like “false beliefs,” Wade said.

“We found there were subjects who would say, ‘I think it was this boy and he wore a red shirt and he was my neighbor, but I don’t actually remember. I’m speculating, trying to remember here,’” Wade said. “There’s a distinction between speaking about an event, and genuinely remembering it and thinking you remember it.”

Shaw declined to be interviewed by phone, citing scheduling conflicts. This week, her startup Palace launched a chatbot, based on artificial intelligence and memory science, that lets employees anonymously report harassment and discrimination. (Porter, who happens to be embroiled in sexual harassment accusations, did not return requests for comment.)

But in a response not yet published in Psychological Science, Shaw — who shared the manuscript with BuzzFeed News — said that she purposely did not distinguish between false memories and false beliefs. “[T]his difference may be difficult, if not impossible, to reliably direct,” she wrote.

There are many ways to classify false memories, she wrote, and her own system was not wrong, but intentionally different. She pointed out that when Wade’s team added up the false memories and false beliefs according to the other criteria, the rates were close to 70%.

Two independent psychologists, however, agreed with Wade’s criticisms of the paper.

“What they claimed to be measuring was wrong,” said Deryn Strange, an associate professor of psychology at the John Jay College of Criminal Justice. (Strange helped develop one of the criteria sets that Wade used to re-analyze the study, but was not involved in the analysis.)

“It was published in a top journal, it received a lot of media attention, but the figure didn’t sit with what we know was happening in the rest of the literature for implanting false memories,” Strange added. “It seemed likely, right from the outset, something was wrong.”

And John Wixted, a psychology professor at UC San Diego, said that he would like false memories and beliefs to be categorized with even greater nuance.

“People have a strong tendency to think in binary terms about memory — it’s there or not there,” he said. “Thinking in binary terms like that is not the right way to think about memory. There are shades of gray, there are gradations of memory.”

What’s more, Shaw confirmed that her paper has statistical errors. One of the biggest mistakes means that the average number of details recalled for the false memories was about 65, not 72, according to a draft she shared with BuzzFeed News.

The correction will be published in several weeks, according to Stephen Lindsay, the editor of Psychological Science. “Most of them are small errors and none changes the findings in a qualitative way (although it doesn’t speak well of the authors or of the journal that there were so many errors),” he told BuzzFeed News by email.

Shaw said that Nick Brown and James Heathers, independent researchers who ferret out mistakes in psychology research, brought the mistakes to her attention. She chalked them up to “some small miscalculations, due to an Excel error where a couple of cells were missed for a series of calculations.”