On her popular Instagram feed, Jessie Brebner informs her thousands of followers about tools they can use to become — or avoid becoming — pregnant. Last summer, she posted something worrisome: A scientist was raising serious questions about the Daysy, a $330 thermometer that claimed it could identify whether you’re fertile with 99.4% accuracy.
Women reacted with shock and concern. And Daysy’s manufacturer, Switzerland-based Valley Electronics, promptly blocked Brebner from its Instagram account. When the influencer later chimed in on the company’s Facebook group, where another person had asked about the evidence for the device, she was kicked out.
“I just don’t think it’s fair to market something as 99.4% effective to women if it’s not, because this is literally women’s lives that can be affected by something like that,” Brebner told BuzzFeed News.
The study that was the source of the 99.4% figure was retracted Tuesday from the journal Reproductive Health. It was pulled more than a year after Chelsea Polis, a reproductive health researcher, raised concerns about it — and eight months after the journal’s editor told her that the study would be withdrawn.
In its retraction notice, the journal’s publisher wrote that “fundamental flaws in the methodology” mean “that the conclusions are unreliable.” The publisher also noted that the study’s authors disagreed with the retraction.
Despite hearing criticism for months — and even formally acknowledging that the paper would be retracted — Valley Electronics has continued to tout the device on social media as being 99.4% accurate, although it has removed the statistic from its website. “Travelling with Daysy? No problem!” said a Facebook post on Jan. 23. “Daysy is made to adjust to your unique way of living with 99.4% accuarcy [sic].”
Niels van de Roemer, Valley Electronics’ medical director, said earlier this year that the company has “the right to block accounts that use our platforms to make aggressive advertising for themselves” and was justified in citing the study until it was officially removed. “Since the study was not retracted about 1 year after the publication of the commentary, it is of course legitimate to use the content for educational purposes,” he wrote by email.
But Polis disagrees. “It does not appear this company is very interested in providing accurate information to its potential consumer base,” she said.
Her protracted attempt to fact-check the Daysy, she said, demonstrates how untruths proliferate online far faster than scientific publishing is accustomed to correcting them.
“My concern is that these are overinflated estimates, and that people are being put at risk of unintended pregnancies” because publishing in scientific journals appears to make a company credible, said Polis, a senior research scientist at the Guttmacher Institute (though her research on Daysy was done outside of her role there).
For Polis and other critics, Daysy is just one of the many fertility products now flooding the market without solid evidence to stand on. Many are essentially just thermometers connected to an app. Although they’re marketed as high-tech, the basic concept behind them is antiquated, said Lauren Streicher, clinical director of the Northwestern Medicine Center for Sexual Medicine and Menopause. She said that clinicians realized decades ago that body temperature alone is not a reliable indicator of a woman’s fertility window, and have since recommended far more accurate urine tests.
“To me, the danger is that people are depending on it,” she said of digital thermometers. “Whether they’re depending on it to not get pregnant and end up inadvertently pregnant. Or if they’re using it to get pregnant and they’re not getting pregnant, they’re wasting a whole lot of time.”
For example, Natural Cycles requires users to take their temperature every morning and record that data in an app. Although it is the first app with FDA approval to market itself as a contraceptive, it has been blamed for misleading ads touting its accuracy and for causing unintended pregnancies. A regulatory agency determined that a number of those pregnancies were in line with the company’s statistics, but ordered it to clarify its marketing about the risks.
Along the same lines, the Daysy is an under-the-tongue thermometer that, with a combination of red and green lights, communicates to its user where she is in her ovulation cycle and records that data on a corresponding app.
These “digital birth control” inventions can help women become more aware of their bodies, but Mary Jane Minkin, a professor of obstetrics, gynecology, and reproductive sciences at Yale School of Medicine, said they likely don’t offer much more than that.
“If it helps people and it helps them track better and helps them get organized, that’s perfectly okay,” she said. “But in terms of ‘Is this going to offer something brilliantly new and novel?’ — I just don’t see that happening.”
Nevertheless, the Daysy has a healthy follower count on its main Instagram of more than 43,000, not to mention several additional accounts for different countries. It, along with other devices like the Lady-Comp and the Baby-Comp, is made by Valley Electronics, which describes itself as a family-owned business of about 50 employees founded more than 30 years ago.
Polis started researching the Daysy in the summer of 2017. While she was looking for studies about fertility-tracking apps, she found a pair of papers by Valley Electronics about devices that were predecessors to the Daysy, and whose results were being touted as evidence for the Daysy.
In Polis’s view, the studies raised red flags because they surveyed users about their past experiences with the devices, instead of following women as they used them. What’s more, she believed, the questionnaires were not well-designed or rigorous. So she emailed the company to express her concerns about its shoddy science.
In September 2017, van de Roemer of Valley Electronics responded to say her feedback would be discussed by an internal advisory board, according to an email shared with BuzzFeed News. But in another email, he also wrote, “Personally, I don’t think you can compare a company’s (not representative) surveys with scientific standards.”
The Daysy didn’t resurface on Polis’ radar until March 2018, when a group of scientists, including one from Valley Electronics, published the company-funded study in Reproductive Health. Based on a survey that asked 125 users about their past use of the app and pregnancies, it concluded that the “Pearl Index” — a metric for the number of contraceptive failures from a birth control method — was 0.6, making the device 99.4% effective at preventing unintended pregnancies.
But Polis was alarmed by how the company came to that 0.6 index. This time, she outlined her concerns in a formal rebuttal to the journal, noting, for instance, the survey’s low 13% participation rate. She also pointed out that the study did not count responses from users with fewer than 13 cycles’ worth of data in the app — meaning that it left out the majority of users’ data, and at least 10 of their pregnancies, according to a data table in the paper.
As a result, she wrote, the study served to “severely underestimate unintended pregnancy rates.” She called for it to be retracted “to protect public health, scientific integrity, and potential consumers.”
Polis submitted her critique to Reproductive Health in April 2018, and it was published that June. Three months later, the editor-in-chief José Belizán emailed to say that the Daysy study would be retracted.
“Fortunately, the final decision have [sic] been taken and promptly the retraction will be published in the journal,” Belizán wrote her in a Sept. 25, 2018, email. “It was a painful experience but unfortunately these are situations that journals are exposed to.” In an undated document published on its website sometime that fall, Valley Electronics acknowledged, too, that the study would be retracted.
But that retraction would not actually happen for many more months.
Earlier this year, van de Roemer of Valley Electronics acknowledged to BuzzFeed News the need for the retraction, but said that the Daysy’s accuracy was backed up by three other studies of similar devices by the company.
He also said that the company is working with US scientists on a new study, “as we are convinced that further studies and scientific findings are necessary.” But he declined to disclose any details about that research.
Although Polis knew that retractions frequently take months, sometimes even years, she did not expect that doing something in the public’s interest would be so hard or so time-consuming — for her, or for other potential science whistleblowers.
“I think it’s reasonable after a year to have lost a little bit of faith that this is being taken seriously,” she said.