The Ministry of Justice rejected the methodology used in a study that suggested a sex offender treatment programme increased reoffending even though it had accepted the “almost identical” methodology in a study on another subject which had more politically favourable results, an employment tribunal heard on Wednesday.
BuzzFeed News revealed on Tuesday that researcher Kathryn Hopkins is suing the department over the impact on her career of whistleblowing about her research showing its Sex Offender Treatment Programme (SOTP) was associated with higher rates of reoffending.
Appearing in the Central Employment Tribunal in London, Hopkins, who worked for the MoJ’s Analytical Services, described how the department had not acted on her findings about the potentially damaging effect of the programme for five years.
While the case was being heard on Wednesday, the House of Commons Justice Select Committee was asking justice minister Robert Buckland about the lack of evidence for the effectiveness of rehabilitation programmes for offenders.
Despite freedom of information requests for the study going back to 2012, and despite a peer review panel recommending that the report could be published that year subject to changes, a final version of the report was not published until 2017.
The MoJ argued that as well as issues with the initial quality of the research, a general election and purdah caused delay to publication. But Hopkins challenged this, pointing out it was published online five days after a media leak. “After it was leaked to the Mail on Sunday it very swiftly got published,” she said.
The court heard how the research was re-run over several years, eventually changing to a new methodology and a new lead author.
The late change in methodology under a new author was questioned by Hopkins, who said they had been happy with the same methods when they were used for research that had more politically convenient findings.
Hopkins oversaw the publication of a report on the effectiveness another cognitive-behavioural intervention aimed at general offenders back in 2010, called the Enhanced Thinking Skills Programme. The research was managed by Hopkins and undertaken by a consultancy firm.
The study suggested the programme worked, and Hopkins said the prison service was “delighted with the result” and commissioned a similar piece of research to be done of their treatment of sex offenders.
But when the results of the study on sex offenders came back in 2012 suggesting the programme might do more harm than good, Hopkins said her managers and the prison service met them “with anger and disbelief”.
After a peer review panel recommended changes before it could be published, she increased the sample size and said she got similar — though slightly more unfavourable — results. After this, the National Offender Management Service (NOMS) wrote to Analytical Services complaining that there were serious problems with the research and requesting that it be stopped.
Hopkins told the court: “I was commissioned to do the research because of my expertise and then it was decided I didn’t have the expertise after all, but only after the results were known.”
Three years later, when the research had still not been published, a new methodology was suggested. The MoJ argues the new methodology was chosen because it was a more “real life” way of testing the data and would give a more robust result.
Hopkins disagrees. She believes that since they had been happy with the methodology in research where the result was positive, the reason for changing it at a late stage for this study was in the hope of changing the result. “We had just run the [Enhanced Thinking Skills study] which had used the exact same methodology for NOMS. The methodology was almost identical,” she said.
The method used by Hopkins puts all treated sex offenders in the treatment group, regardless of when they receive the treatment. The method used by the MoJ to re-run the data in 2015 keeps later-treated individuals in the comparison group, despite them being treated.
Hopkins said the new method was chosen because it “always or nearly always shows a smaller effect” and she believes the MoJ “thought it would minimise or reduce the results”. The MoJ’s position is that this study method was better for showing the “real world effect” of the programme.
The final study published in 2017 had a very similar outcome to the results she had in 2012, showing sex offenders who had been on the programme were more likely to reoffend, but there was a less dramatic gap between the two groups.
Hopkins argued that changing the methodology using the same data set once they already had a result broke ethical guidelines for research. “It was unethical. It’s a complete breach of research ethics to get the result and then go back and change the design.”
In court papers, the named lead author of the 2017 study, Aidan Mews, argued the new methodology “mitigated against a potential source of bias”.
Sarah Morton, one of Hopkins’ managers at the MoJ from 2014, was the first to give evidence for the department. When asked if they were wrong to re-run the study given that the conclusions were largely the same five years later, she disagreed, arguing the issue was about the quality of the research.
“It’s not about what the eventual findings are, it’s about the methodology and whether it stood up to external scrutiny,” she told the court.
Morton said there were also issues with Hopkins’ behaviour at work, describing an incident two years after Hopkins first raised the alarm about the programme where she became emotional about the extended process of quality-assuring her study.
“Kathryn became quite upset and said to me ‘how can you sleep at night? You’re responsible for children being molested because of your decisions, how would you feel if it was your children?’”