CR: COVID-19 --Political Talk

[PsychMike 1,550]

This seems bad...and captures some of the exact things that most academicians feared nearly two years ago about quick data releases without peer review. Anyone seen triple lately?

How bad research clouded our understanding of Covid-19

Early studies of Covid-19 therapeutics turned out to be fabricated or suspicious. That’s a huge problem for science. 

By Kelsey Piper  Dec 17, 2021, 9:00am EST

 

The scientific integrity site Retraction Watch has been a bulwark of sound science since its founding in 2010. Its self-proclaimed mission is to act as a “window into the scientific process” by shining a light on academic research that has been retracted.

Retractions come from across all scientific fields, but over the past two years, many of the most consequential ones have revolved around a single topic: Covid-19. And according to Retraction Watch, as many as 200 Covid-19 papers have been retracted since the start of the pandemic for a range of reasons: elementary calculation errors, researchers refusing to provide evidence that the studies were really conducted, or conclusions that aren’t supported by the data.

The problem is especially acute when it comes to studies looking into treatments for Covid-19. Early papers claiming to find stunning results for drug regimens like ivermectin or hydroxychloroquine earned tremendous attention and influence, only to later be retracted, often because substantial evidence suggested that the studies never actually happened or at least never happened as described. 

One ivermectin study included in an influential meta-analysis that found great results for the drug turned out to be based on a data file where the same 11 patients were copied and pasted repeatedly to produce a more robust sample size of a few hundred. When BuzzFeed Newsfollowed up on another ivermectin study with huge results, a hospital where the research had reportedly been conducted said it had no record of such a study happening there. 

As I wrote for Vox in September, the evidence for ivermectin is mixed and limited. Most careful analyses suggest it might have an effect but also that we don’t know enough to say for sure — certainly not enough to hail it as a miracle cure, as many have. Part of the reason the treatment rose to prominence is that early, high-profile studies of ivermectin showed positive results, sometimes huge ones. When other researchers peered under the hood, however, they realized those findings didn’t hold up or had no basis to begin with. But by then it was too late — for some, ivermectin had become an article of faith.

To be clear, it’s not all an epistemic hellscape. There’s sound research being performed on early treatments, including ivermectin. But the damage done by fraudulent research is immense and has contributed to misleading public policymaking and individual decision-making. Narratives about effective cures take hold only for retractions to cloud the picture, abetting bad-faith actors and inflaming our all-consuming culture war.

If we are to finally overcome this pandemic and defeat the next one, it’s essential for us to fix a system where it’s all too easy for bad research to warp scientific discourse and public knowledge. 

A look at papers that show signs of serious error ormisconduct

One of the first high-profile instances of alleged Covid-19 research fraud occurred last spring: A study in The Lancet claimed to have looked at more than 96,000 coronavirus patients across the world and found that — after controlling for age, sex, and how sick the subjects were — patients receiving hydroxychloroquine or a variant of the drug were about twice as likely to die as those who did not.

But scientists immediately started pointing out that something was wrong with the study. It had reported more Covid-19 deaths of enrolled patients in its Australia arm than there were Covid-19 deaths in all of Australia. Further investigation found that hospitals supposedly enrolled in the study had never heard of Surgisphere, the company that conducted it.

The hydroxychloroquine paper was quickly retracted from The Lancet, as was another by the same authors in The New England Journal of Medicine, another highly prominent journal. Further research has demonstrated that hydroxychloroquine is not, in fact, an effective Covid-19 treatment — but it’s also not as deadly as the Surgisphere paper suggested.

In the months since, scientific misconduct, data fraud, and careless mistakes have been repeatedly found in the Covid-19 literature.

Jack Lawrence, a grad student in biomedical science at the University of London, kicked off a broad investigation into ivermectin research fraud when he got an assignment to look at a widely cited ivermectin study. That study, by professor of medicine Ahmed Elgazzar and others at Benha University in Egypt, found huge benefits for Covid-19 patients taking ivermectin.

Lawrence noticed something wrong immediately: The introduction was plagiarized from other papers. He then notified a colleague, Gideon Meyerowitz-Katz, an epidemiologist at the University of Wollongong in Australia. “Jack Lawrence emailed me about the Elgazzar study and said, ‘I know you think it’s low quality but are you aware that most of it’s plagiarized?’” Meyerowitz-Katz recalled. “And that started this whole thing.” 

Data forensics experts found other problems with the Elgazzar paper: patients who died before the study started and yet were counted as enrolled; patient records duplicated; numbers that were too suspiciously clean to have occurred by chance. (Elgazzar says his study is legitimate.)

Soon, the preprint server that hosted the paper withdrew it. But the damage had been done — the Elgazzar study had already started feeding the widespread misconception in some quarters that ivermectin was a Covid-19 miracle drug. 

 

A supporter of Brazilian President Jair Bolsonaro holds a large box of box of ivermectin during a protest organized to show support to his government in Brasilia, Brazil, in May. Bolsonaro has recommended the use of hydroxychloroquine and ivermectin to treat Covid-19.  Andressa Anholete/Getty Images

Lawrence and Meyerowitz-Katz consider themselves part of a loose-knit group of researchers — including Nick Brown of the University of Groningen in the Netherlands, Cipher Skin chief scientific officer James Heathers, and Kyle Sheldrick of the University of New South Wales in Australia — who have taken it upon themselves to check the integrity of Covid-19 research. 

A big part of their work is simply to pore over the data and methods of Covid-19 studies. It was this team that investigated the paper, in the journal Viruses, that found that ivermectin was a highly effective treatment but that turned out to have a data set that was just the same 11 patient records copied over and over.

 

The study authors said, “After revising the raw data we realised that a file that was used to train a research assistant was sent by mistake for analysis.” (Though the authors admitted the error, they have yet to come out with a legitimate data set.) 

The same group of researchers has also raised serious questions about a large randomized controlled trial in Iran that found positive results for ivermectin. 

The problem? The data is “not consistent with a genuine randomised controlled trial,” Sheldrick argued in a detailed blog post laying out some of the suspicious inconsistencies. “This paper claims to describe a trial in which patients were randomly allocated to treatments. This is not true. Extreme differences are seen between groups across multiple variables such as oxygen level, blood pressure, and SARS-CoV-2 test results before they even got their first dose of medication.” 

One simple but effective test for data fabrication is whether numbers have a typical distribution of “trailing digits” — for example, whether they end in “1” about as often as they end in “0.” Humans, when making up numbers, tend to cluster around certain end digits. The randomized control trial from Iran fails this test, with very pronounced digit clustering to a degree that can’t have occurred by chance, according to Sheldrick. (It’s “very normal to see such randomisation,” the lead author rebutted.)

Members of this research group have raised questions, too, about research published by Dr. Flavio Cadegiani on various Covid-19 treatments, including ivermectin and proxalutamide, that exhibited bizarre statistical patterns inconsistent with randomization. They also raised concerns about research into the anti-parasite medication nitazoxanide, which turned out to be riddled with statistical errors and data analysis problems. (Meanwhile, other researchers have noted that Cadegiani’s papers claim a horrifying death rate in the control group. One explanation is fabricated data; another is medical malpractice, prompting a parliamentary inquiry in Brazil to accuse Cadegiani of crimes against humanity.)

Other researchers have been uncovering more dubious work. A randomized controlled trial in Egypt studying favipiravir and hydroxychloroquine was retracted after glaring data inconsistencies were found.

A study that took place in Brazil that found incredible benefits of ivermectin as prophylaxis also failed to withstand scrutiny. Reporters at BuzzFeed News called one of the hospitals where the study was reportedly conducted for more information, only to learn that the hospital said that it hadn’t participated.

The sheer number of problem studies is eye-popping. In one major analysis, Lawrence, Meyerowitz-Katz, and others looked at 26 studies that were part of the evidence base for ivermectin and found that 10 of them had serious invalidating errors or evidence of likely fraudulent conduct. It’s one of the first attempts to estimate the scale of the problem, and it’s not encouraging. 

“I’ve been working in this field for 30 years and I have not seen anything like this,” University of Liverpool’s Andrew Hill, who has been researching Covid-19 treatments, told MedPage Today. “I’ve never seen people make data up. People dying before the study even started. Databases duplicated and cut and pasted.”

How misconduct is interfering with our understanding of Covid-19

To understand how flimsy or even potentially fraudulent small-scale studies can have such a strong influence on broader conclusions about Covid-19, it helps to understand that a considerable part of our knowledge of treatments comes from meta-analyses.

 

A meta-analysis looks comprehensively at published evidence on a topic and figures out which way that evidence collectively points. A small study here and there might not give us findings we can count on alone, but put a bunch more studies together and you’re likelier to get a reliable picture of a drug’s effects. 

But if some of the papers included are error-ridden or fabricated, it can easily throw off the entire meta-analysis.

Meta-analyses of ivermectin research have often combined some papers that found modest or no effects with papers that found dramatic, nearly miraculous effects — and consequently a few of those meta-analyses have concluded that the drug should be expected to work remarkably well for Covid-19. (Quite reasonably, ivermectin’s enthusiasts often point to meta-analyses like these, which are generally considered valid scientific tools.)

That would be fine if all the studies looked at in a meta-analysis were of high quality and had been conducted as the authors described. But when a decent share of them might well be problematic or maybe even faked, aggregating all these studies won’t give you a clear picture of the drug’s efficacy, to say the least. 

The University of Liverpool’s Hill conducted a meta-analysis that initially found positive results for ivermectin, only to conduct a re-analysis without studies that were later identified as suspect that found much worse results for ivermectin. “This has made me more wary about trusting results when you don’t have access to the raw data,” Hill told MedPage Today. “We took them on trust and that was a mistake.”

In the case of ivermectin, the phenomenon is especially striking because the bad papers “are bigger than average, make more dramatic claims than average,” Sheldrick told me. “While this is a minority of studies, they probably form about half of all the trial data we have on ivermectin for Covid.” 

That makes sense — fakers often publish their studies faster, Sheldrick said, because running a real study takes a lot more time than fabricating data does. And they often claim to have run larger and more comprehensive studies. 

“Running a 600-person trial is a hell of a lot more work than running a 50-person trial, but faking a 600-person trial isn’t actually that much more work than faking a 50-person trial,” he said. So in the case of an event like Covid-19 where researchers are rushing to complete trials as fast as possible, the rate of possible fraud tends to be higher in the earliest published research, which disproportionately influences further research, and analyses by number of patients treated will often end up over-weighting studies that didn’t occur as described. 

The preponderance of bad research casts a shadow on the whole research enterprise two years into the pandemic. “The entire scientific community operates on trust,” Meyerowitz-Katz told me. “There is this assumption in research that if someone tells you they have done something, then they have done it.” 

Unfortunately, there aren’t enough guardrails at other points in the process to prevent misleading or flawed work from seeing the light of day. The peer review process typically checks whether the authors of a paper are accurately interpreting their data, whether they’ve appropriately situated their results in the existing literature, and whether their approach to the topic is a good one. What it generally does not do is check whether the data set that the researchers used was faulty or even fabricated.

And publication in top-tier journals isn’t a safeguard against error or fraud. While some of the highest-profile studies that were allegedly fraudulent were published only as preprints or in poorly regarded journals, “some of the studies where we have very serious concerns about fraud are in very high-quality journals,” Meyerowitz-Katz told me.

 

“There are some studies that have found benefitsfor ivermectin that are certainly not fraud,” he added. But those benefits have tended to be small and still need to be confirmed by further research. “Any study reporting massive benefits from ivermectin, at this point, we’ve either found fairly strong evidence for fraud or the authors have declined to share data with us.”

A challenge to how we do science

There has been hardly any let-up for the retraction watchers. Last month saw another high-profile incident: The Journal of Intensive Care Medicineprinted a retraction notice for a December 2020 paper about how to treat Covid-19 patients co-authored by Pierre Kory, who has testified in front of the US Senate about Covid-19 and has been a leading advocate for ivermectin. 

Kory and his co-authors’ paper centered on his so-called MATH+ treatment: a protocol with four headline drugs — methylprednisolone, ascorbic acid (Vitamin C), thiamine, and heparin — plus other treatments. (Kory has since included ivermectin in the protocol, but it was not part of his treatment regimen at the time the paper was published.) 

The paper was retracted because a hospital where the research took place told the journal that patients there weren’t systematically offered Kory’s treatment, contrary to his claims, and that Kory misrepresented outcomes for the patients who did receive his treatment. 

Moreover, officials at the hospital — Sentara Healthcare in Norfolk, Virginia — told me that Kory had ignored their calls when they reached out to ask for the error to be corrected. (Kory has said that it was inappropriate for the journal to retract the paper and that he has offered to correct the study.) 

The long list of dubious studies adds up to a single big picture: Our understanding of ivermectin, and early Covid-19 treatments more broadly, has been badly damaged by studies reporting results that did not really occur as described. 

The situation has implications for how scientific studies are performed and vetted that go beyond Covid-19 research.

 

The researchers who’ve been calling out bad studies have some ideas on how to improve things. In an article in Nature Medicine in September, they made the case for a radical rethinking of meta-analyses. Instead of re-analyzing using “summary data” from original studies, they argue, the scientific community should transition to doing meta-analyses that re-analyze by patient — effectively pooling all the patients across all the studies as if they were all part of one study, and then analyzing that.

Currently, patient data is not generally shared among researchers, both because of privacy concerns and because scientists often feel proprietary about data sets they may have worked for months or years to acquire, clean up, and analyze. But if those norms changed, it would be harder for data fakers to get away with it or for serious mistakes to pass unnoticed — and our processes for aggregating knowledge could improve significantly. 

“If people are unwilling to share their data, that’s the biggest giveaway. There’s a good likelihood there are major, major flaws in the data,” Covid-19 treatments researcher Ed Mills at McMaster University told me. 

Sheldrick told me that some researchers declare in their paper that their data is available and they’re happy to share it, and then avoid scrutiny by not actually responding to requests. There’s no real mechanism to complain about a researcher who said they’d share their data but isn’t following through. “We’ve developed this culture where it’s acceptable to publish a study and go, ‘Our data’s available, just contact us,’ and then not respond when contacted,” Sheldrick said.

Another approach may simply be for researchers to internalize what they’ve learned these past couple of years and more closely scrutinize ivermectin studies — and, more broadly, studies that claim spectacular benefits while obscuring some details of their methodology.

Flavio Abdenur, an independent mathematician and data scientist who has been investigating the case for ivermectin through meta-analyses, told me he has radically changed his approach, examining the ivermectin literature by looking only at studies that have been demonstrated not to be fraudulent. He’s essentially withdrawing the normal practice of giving researchers the benefit of the doubt. 

That’s what Richard Smith, former editor of the British Medical Journal (BMJ), argued for in an editorial in the BMJ this summer. “We have now reached a point where those doing systematic reviews must start by assuming that a study is fraudulent until they can have some evidence to the contrary,” he wrote. 

That would be an extraordinary shift. But there’s no question that something needs to change. Monitoring for mistakes and fraud can’t only be done by journalists calling hospitals, hospitals doing their own research and contacting journals proactively, and researchers with day jobs looking for fraud on the side. 

If fraudulent and faulty research is a systematic, serious problem — and it is — it needs a systematic, serious solution.

[GreenspointTexas 9,600]

From a friend at Houston methodist:

 

Covid outbreak at Methodist was contact traced to a fully vaccinated outdoor party. Y'all be safe over the holidays, seems like this new variant is suboptimal.

 

And they're stopping regeneron treatments because they aren't working
 

 

 

 

 

at least it seems like omicron is pretty mild tho, so itll burn through fast with little spike in hospitalizations?

[Don Johnson 4,753]

LOL.  Shut it all down and come back in February.

[Post Oak 20,210]

16 minutes ago, GreenspointTexas said:

 

 

 

At least it seems like omicron is pretty mild tho, so itll burn through fast with little spike in hospitalizations?

 

[Don Johnson 4,753]

On 12/15/2021 at 2:30 PM, Don Johnson said:

Who doesn't see that?  I see that. 

What I disagree with is the idea that the 3% of the world's unvaccinated in the US not getting vaccinated is why this thing isn't long gone. 

It doesn't help.  Its dumb for them.  But if they changed their mind tomorrow the worldwide pandemic isn't over.

The UK just hit its highest daily total.  That means they have more cases in a day at 75% vaccinated than they ever did with 0%.  In fact, the 78k blew past the previous record of 68k. 

 

Yesterday, the UK blew by the previous day's record of 78K cases with 88k.  Most ever.  Will be interesting to see what today brings.

 

[Don Johnson 4,753]

 

[Doug E. Fresh 452]

2 hours ago, Don Johnson said:

Yesterday, the UK blew by the previous day's record of 78K cases with 88k.  Most ever.  Will be interesting to see what today brings.

 

I thought we’d be over 100k today but it was “only” 93k today so not quite spiking like some though it would. There were predictions of up to 600k per day by this weekend, which won’t even be close.

I had 3 days of congestion and fatigue but it all seems to have passed today. Took the dog out for a walk and Central London is very quiet tonight, although some of that is due to this being the last day of work/school for most people.

[GreenspointTexas 9,600]

3 hours ago, Don Johnson said:

LOL.  Shut it all down and come back in February.

Too late for that

[wildcat09 29,349]

4 hours ago, PsychMike said:

This seems bad...and captures some of the exact things that most academicians feared nearly two years ago about quick data releases without peer review. Anyone seen triple lately?

How bad research clouded our understanding of Covid-19

Early studies of Covid-19 therapeutics turned out to be fabricated or suspicious. That’s a huge problem for science. 

By Kelsey Piper  Dec 17, 2021, 9:00am EST

 

The scientific integrity site Retraction Watch has been a bulwark of sound science since its founding in 2010. Its self-proclaimed mission is to act as a “window into the scientific process” by shining a light on academic research that has been retracted.

Retractions come from across all scientific fields, but over the past two years, many of the most consequential ones have revolved around a single topic: Covid-19. And according to Retraction Watch, as many as 200 Covid-19 papers have been retracted since the start of the pandemic for a range of reasons: elementary calculation errors, researchers refusing to provide evidence that the studies were really conducted, or conclusions that aren’t supported by the data.

The problem is especially acute when it comes to studies looking into treatments for Covid-19. Early papers claiming to find stunning results for drug regimens like ivermectin or hydroxychloroquine earned tremendous attention and influence, only to later be retracted, often because substantial evidence suggested that the studies never actually happened or at least never happened as described. 

One ivermectin study included in an influential meta-analysis that found great results for the drug turned out to be based on a data file where the same 11 patients were copied and pasted repeatedly to produce a more robust sample size of a few hundred. When BuzzFeed Newsfollowed up on another ivermectin study with huge results, a hospital where the research had reportedly been conducted said it had no record of such a study happening there. 

As I wrote for Vox in September, the evidence for ivermectin is mixed and limited. Most careful analyses suggest it might have an effect but also that we don’t know enough to say for sure — certainly not enough to hail it as a miracle cure, as many have. Part of the reason the treatment rose to prominence is that early, high-profile studies of ivermectin showed positive results, sometimes huge ones. When other researchers peered under the hood, however, they realized those findings didn’t hold up or had no basis to begin with. But by then it was too late — for some, ivermectin had become an article of faith.

To be clear, it’s not all an epistemic hellscape. There’s sound research being performed on early treatments, including ivermectin. But the damage done by fraudulent research is immense and has contributed to misleading public policymaking and individual decision-making. Narratives about effective cures take hold only for retractions to cloud the picture, abetting bad-faith actors and inflaming our all-consuming culture war.

If we are to finally overcome this pandemic and defeat the next one, it’s essential for us to fix a system where it’s all too easy for bad research to warp scientific discourse and public knowledge. 

A look at papers that show signs of serious error ormisconduct

One of the first high-profile instances of alleged Covid-19 research fraud occurred last spring: A study in The Lancet claimed to have looked at more than 96,000 coronavirus patients across the world and found that — after controlling for age, sex, and how sick the subjects were — patients receiving hydroxychloroquine or a variant of the drug were about twice as likely to die as those who did not.

But scientists immediately started pointing out that something was wrong with the study. It had reported more Covid-19 deaths of enrolled patients in its Australia arm than there were Covid-19 deaths in all of Australia. Further investigation found that hospitals supposedly enrolled in the study had never heard of Surgisphere, the company that conducted it.

The hydroxychloroquine paper was quickly retracted from The Lancet, as was another by the same authors in The New England Journal of Medicine, another highly prominent journal. Further research has demonstrated that hydroxychloroquine is not, in fact, an effective Covid-19 treatment — but it’s also not as deadly as the Surgisphere paper suggested.

In the months since, scientific misconduct, data fraud, and careless mistakes have been repeatedly found in the Covid-19 literature.

Jack Lawrence, a grad student in biomedical science at the University of London, kicked off a broad investigation into ivermectin research fraud when he got an assignment to look at a widely cited ivermectin study. That study, by professor of medicine Ahmed Elgazzar and others at Benha University in Egypt, found huge benefits for Covid-19 patients taking ivermectin.

Lawrence noticed something wrong immediately: The introduction was plagiarized from other papers. He then notified a colleague, Gideon Meyerowitz-Katz, an epidemiologist at the University of Wollongong in Australia. “Jack Lawrence emailed me about the Elgazzar study and said, ‘I know you think it’s low quality but are you aware that most of it’s plagiarized?’” Meyerowitz-Katz recalled. “And that started this whole thing.” 

Data forensics experts found other problems with the Elgazzar paper: patients who died before the study started and yet were counted as enrolled; patient records duplicated; numbers that were too suspiciously clean to have occurred by chance. (Elgazzar says his study is legitimate.)

Soon, the preprint server that hosted the paper withdrew it. But the damage had been done — the Elgazzar study had already started feeding the widespread misconception in some quarters that ivermectin was a Covid-19 miracle drug. 

A supporter of Brazilian President Jair Bolsonaro holds a large box of box of ivermectin during a protest organized to show support to his government in Brasilia, Brazil, in May. Bolsonaro has recommended the use of hydroxychloroquine and ivermectin to treat Covid-19.  Andressa Anholete/Getty Images

Lawrence and Meyerowitz-Katz consider themselves part of a loose-knit group of researchers — including Nick Brown of the University of Groningen in the Netherlands, Cipher Skin chief scientific officer James Heathers, and Kyle Sheldrick of the University of New South Wales in Australia — who have taken it upon themselves to check the integrity of Covid-19 research. 

A big part of their work is simply to pore over the data and methods of Covid-19 studies. It was this team that investigated the paper, in the journal Viruses, that found that ivermectin was a highly effective treatment but that turned out to have a data set that was just the same 11 patient records copied over and over.

 

The study authors said, “After revising the raw data we realised that a file that was used to train a research assistant was sent by mistake for analysis.” (Though the authors admitted the error, they have yet to come out with a legitimate data set.) 

The same group of researchers has also raised serious questions about a large randomized controlled trial in Iran that found positive results for ivermectin. 

The problem? The data is “not consistent with a genuine randomised controlled trial,” Sheldrick argued in a detailed blog post laying out some of the suspicious inconsistencies. “This paper claims to describe a trial in which patients were randomly allocated to treatments. This is not true. Extreme differences are seen between groups across multiple variables such as oxygen level, blood pressure, and SARS-CoV-2 test results before they even got their first dose of medication.” 

One simple but effective test for data fabrication is whether numbers have a typical distribution of “trailing digits” — for example, whether they end in “1” about as often as they end in “0.” Humans, when making up numbers, tend to cluster around certain end digits. The randomized control trial from Iran fails this test, with very pronounced digit clustering to a degree that can’t have occurred by chance, according to Sheldrick. (It’s “very normal to see such randomisation,” the lead author rebutted.)

Members of this research group have raised questions, too, about research published by Dr. Flavio Cadegiani on various Covid-19 treatments, including ivermectin and proxalutamide, that exhibited bizarre statistical patterns inconsistent with randomization. They also raised concerns about research into the anti-parasite medication nitazoxanide, which turned out to be riddled with statistical errors and data analysis problems. (Meanwhile, other researchers have noted that Cadegiani’s papers claim a horrifying death rate in the control group. One explanation is fabricated data; another is medical malpractice, prompting a parliamentary inquiry in Brazil to accuse Cadegiani of crimes against humanity.)

Other researchers have been uncovering more dubious work. A randomized controlled trial in Egypt studying favipiravir and hydroxychloroquine was retracted after glaring data inconsistencies were found.

A study that took place in Brazil that found incredible benefits of ivermectin as prophylaxis also failed to withstand scrutiny. Reporters at BuzzFeed News called one of the hospitals where the study was reportedly conducted for more information, only to learn that the hospital said that it hadn’t participated.

The sheer number of problem studies is eye-popping. In one major analysis, Lawrence, Meyerowitz-Katz, and others looked at 26 studies that were part of the evidence base for ivermectin and found that 10 of them had serious invalidating errors or evidence of likely fraudulent conduct. It’s one of the first attempts to estimate the scale of the problem, and it’s not encouraging. 

“I’ve been working in this field for 30 years and I have not seen anything like this,” University of Liverpool’s Andrew Hill, who has been researching Covid-19 treatments, told MedPage Today. “I’ve never seen people make data up. People dying before the study even started. Databases duplicated and cut and pasted.”

How misconduct is interfering with our understanding of Covid-19

To understand how flimsy or even potentially fraudulent small-scale studies can have such a strong influence on broader conclusions about Covid-19, it helps to understand that a considerable part of our knowledge of treatments comes from meta-analyses.

 

A meta-analysis looks comprehensively at published evidence on a topic and figures out which way that evidence collectively points. A small study here and there might not give us findings we can count on alone, but put a bunch more studies together and you’re likelier to get a reliable picture of a drug’s effects. 

But if some of the papers included are error-ridden or fabricated, it can easily throw off the entire meta-analysis.

Meta-analyses of ivermectin research have often combined some papers that found modest or no effects with papers that found dramatic, nearly miraculous effects — and consequently a few of those meta-analyses have concluded that the drug should be expected to work remarkably well for Covid-19. (Quite reasonably, ivermectin’s enthusiasts often point to meta-analyses like these, which are generally considered valid scientific tools.)

That would be fine if all the studies looked at in a meta-analysis were of high quality and had been conducted as the authors described. But when a decent share of them might well be problematic or maybe even faked, aggregating all these studies won’t give you a clear picture of the drug’s efficacy, to say the least. 

The University of Liverpool’s Hill conducted a meta-analysis that initially found positive results for ivermectin, only to conduct a re-analysis without studies that were later identified as suspect that found much worse results for ivermectin. “This has made me more wary about trusting results when you don’t have access to the raw data,” Hill told MedPage Today. “We took them on trust and that was a mistake.”

In the case of ivermectin, the phenomenon is especially striking because the bad papers “are bigger than average, make more dramatic claims than average,” Sheldrick told me. “While this is a minority of studies, they probably form about half of all the trial data we have on ivermectin for Covid.” 

That makes sense — fakers often publish their studies faster, Sheldrick said, because running a real study takes a lot more time than fabricating data does. And they often claim to have run larger and more comprehensive studies. 

“Running a 600-person trial is a hell of a lot more work than running a 50-person trial, but faking a 600-person trial isn’t actually that much more work than faking a 50-person trial,” he said. So in the case of an event like Covid-19 where researchers are rushing to complete trials as fast as possible, the rate of possible fraud tends to be higher in the earliest published research, which disproportionately influences further research, and analyses by number of patients treated will often end up over-weighting studies that didn’t occur as described. 

The preponderance of bad research casts a shadow on the whole research enterprise two years into the pandemic. “The entire scientific community operates on trust,” Meyerowitz-Katz told me. “There is this assumption in research that if someone tells you they have done something, then they have done it.” 

Unfortunately, there aren’t enough guardrails at other points in the process to prevent misleading or flawed work from seeing the light of day. The peer review process typically checks whether the authors of a paper are accurately interpreting their data, whether they’ve appropriately situated their results in the existing literature, and whether their approach to the topic is a good one. What it generally does not do is check whether the data set that the researchers used was faulty or even fabricated.

And publication in top-tier journals isn’t a safeguard against error or fraud. While some of the highest-profile studies that were allegedly fraudulent were published only as preprints or in poorly regarded journals, “some of the studies where we have very serious concerns about fraud are in very high-quality journals,” Meyerowitz-Katz told me.

 

“There are some studies that have found benefitsfor ivermectin that are certainly not fraud,” he added. But those benefits have tended to be small and still need to be confirmed by further research. “Any study reporting massive benefits from ivermectin, at this point, we’ve either found fairly strong evidence for fraud or the authors have declined to share data with us.”

A challenge to how we do science

There has been hardly any let-up for the retraction watchers. Last month saw another high-profile incident: The Journal of Intensive Care Medicineprinted a retraction notice for a December 2020 paper about how to treat Covid-19 patients co-authored by Pierre Kory, who has testified in front of the US Senate about Covid-19 and has been a leading advocate for ivermectin. 

Kory and his co-authors’ paper centered on his so-called MATH+ treatment: a protocol with four headline drugs — methylprednisolone, ascorbic acid (Vitamin C), thiamine, and heparin — plus other treatments. (Kory has since included ivermectin in the protocol, but it was not part of his treatment regimen at the time the paper was published.) 

The paper was retracted because a hospital where the research took place told the journal that patients there weren’t systematically offered Kory’s treatment, contrary to his claims, and that Kory misrepresented outcomes for the patients who did receive his treatment. 

Moreover, officials at the hospital — Sentara Healthcare in Norfolk, Virginia — told me that Kory had ignored their calls when they reached out to ask for the error to be corrected. (Kory has said that it was inappropriate for the journal to retract the paper and that he has offered to correct the study.) 

The long list of dubious studies adds up to a single big picture: Our understanding of ivermectin, and early Covid-19 treatments more broadly, has been badly damaged by studies reporting results that did not really occur as described. 

The situation has implications for how scientific studies are performed and vetted that go beyond Covid-19 research.

 

The researchers who’ve been calling out bad studies have some ideas on how to improve things. In an article in Nature Medicine in September, they made the case for a radical rethinking of meta-analyses. Instead of re-analyzing using “summary data” from original studies, they argue, the scientific community should transition to doing meta-analyses that re-analyze by patient — effectively pooling all the patients across all the studies as if they were all part of one study, and then analyzing that.

Currently, patient data is not generally shared among researchers, both because of privacy concerns and because scientists often feel proprietary about data sets they may have worked for months or years to acquire, clean up, and analyze. But if those norms changed, it would be harder for data fakers to get away with it or for serious mistakes to pass unnoticed — and our processes for aggregating knowledge could improve significantly. 

“If people are unwilling to share their data, that’s the biggest giveaway. There’s a good likelihood there are major, major flaws in the data,” Covid-19 treatments researcher Ed Mills at McMaster University told me. 

Sheldrick told me that some researchers declare in their paper that their data is available and they’re happy to share it, and then avoid scrutiny by not actually responding to requests. There’s no real mechanism to complain about a researcher who said they’d share their data but isn’t following through. “We’ve developed this culture where it’s acceptable to publish a study and go, ‘Our data’s available, just contact us,’ and then not respond when contacted,” Sheldrick said.

Another approach may simply be for researchers to internalize what they’ve learned these past couple of years and more closely scrutinize ivermectin studies — and, more broadly, studies that claim spectacular benefits while obscuring some details of their methodology.

Flavio Abdenur, an independent mathematician and data scientist who has been investigating the case for ivermectin through meta-analyses, told me he has radically changed his approach, examining the ivermectin literature by looking only at studies that have been demonstrated not to be fraudulent. He’s essentially withdrawing the normal practice of giving researchers the benefit of the doubt. 

That’s what Richard Smith, former editor of the British Medical Journal (BMJ), argued for in an editorial in the BMJ this summer. “We have now reached a point where those doing systematic reviews must start by assuming that a study is fraudulent until they can have some evidence to the contrary,” he wrote. 

That would be an extraordinary shift. But there’s no question that something needs to change. Monitoring for mistakes and fraud can’t only be done by journalists calling hospitals, hospitals doing their own research and contacting journals proactively, and researchers with day jobs looking for fraud on the side. 

If fraudulent and faulty research is a systematic, serious problem — and it is — it needs a systematic, serious solution.

The fraudsters should be flayed and then thrown into a pen of great whites.

[Sawbonz 27,547]

Will be interesting to see how the next couple of weeks go. With the initial wave of infection, and with the spike with Delta, the hospitals shut down elective cases about 10 to 14 days after the increase in reported infections because they were inundated with Covid patients. It doesn’t really seem like that is happening with omicron at least not yet

Edited December 17, 2021 by Sawbonz

[DDD Dad 14,335]

Just now, Sawbonz said:

Will be interesting to see how the next couple of weeks ago. With the initial wave of infection, and with the spike with Delta, the hospitals shut down elective cases about 10 to 14 days after the increase in reported infections because they were inundated with Covid patients. It doesn’t really seem like that is happening with omicron at least not yet

Thankfully I was able to sneak in to get my rotator cuff repair.

[Sawbonz 27,547]

This feels different to me. Non lethal mutations compete with each other to infect people. The most advantageous mutations to a strain would be to increase its ability to cause infection but minimize the resulting illness. It doesn’t have to happen that way, but it does give such a strain a big advantage. Hopefully that’s what we are seeing here

[GreenspointTexas 9,600]

11 minutes ago, Sawbonz said:

Will be interesting to see how the next couple of weeks go. With the initial wave of infection, and with the spike with Delta, the hospitals shut down elective cases about 10 to 14 days after the increase in reported infections because they were inundated with Covid patients. It doesn’t really seem like that is happening with omicron at least not yet

Its early, but hospitals dont seem to be filling up with COVID patients just yet… at least here in Houston

[Sbbruin 21,844]

[formermav43 4,606]

7 minutes ago, Sbbruin said:

I feel like that analogy plays better to an audience outside of Texas. Or Southern California, for that matter. 

[jimmyjazz 45,903]

3 minutes ago, formermav43 said:

I feel like that analogy plays better to an audience outside of Texas. 

Before Feb 2021?  Probably.

Edited December 17, 2021 by jimmyjazz
unaware of what year it is

[babysdaddy 2,721]

Delete Facebook for your own sake

[4th&Five 41,525]

 

[Don Johnson 4,753]

Never would've believed it until the last 24-48 hours, but after seeing multiple NBA and CBB games cancelled and some NFL games moved, I imagine some bowl games won't be played.   

[Sbbruin 21,844]

20 minutes ago, babysdaddy said:

Delete Facebook for your own sake

Where else (besides surly) am I going to get my news?!

[Don Johnson 4,753]

5 hours ago, Don Johnson said:

 

New record for New York state

[Neonmoon 38,129]

Cases don’t matter. Hospitalizations do. And before someone responds with cases lead to hospitalizations, meh, it’s a little more nuanced than this these days. Vaccines/Boosters and omicron being less lethal are keeping hospitalizations down. Is Omnicron also blocking Delta out? Who knows. But cases are less important metric these days 

[Satchel 6,938]

I favor bringing back Mutual of Obama’s Wild Kingdom and dart vaccinate the recalcitrant from helocopters.

[Don Johnson 4,753]

3 minutes ago, Neonmoon said:

Cases don’t matter. Hospitalizations do. And before someone responds with cases lead to hospitalizations, meh, it’s a little more nuanced than this these days. Vaccines/Boosters and omicron being less lethal are keeping hospitalizations down. Is Omnicron also blocking Delta out? Who knows. But cases are less important metric these days 

100% agree.  And wish everyone did.  

But the early numbers are staggering.  Omicron doesn't fuck around.

2 minutes ago, Satchel said:

I favor bringing back Mutual of Obama’s Wild Kingdom and dart vaccinate the recalcitrant from helocopters.

Omicron is about to "vaccinate" everyone in the next few weeks.

[NWBuck 8,552]

22 minutes ago, Don Johnson said:

New record for New York state

Won't matter on December 31st... the cash flow show must go on

[Longhorn_Fan68 33,869]

7 minutes ago, Satchel said:

I favor bringing back Mutual of Obama’s Wild Kingdom and dart vaccinate the recalcitrant from helocopters.

can I enter a contest where I get to be one of the administrators? man that would be fun

[Satchel 6,938]

Just now, Don Johnson said:

100% agree.  And wish everyone did.  

But the early numbers are staggering.  Omicron doesn't fuck around.

Omicron is about to "vaccinate" everyone in the next few weeks.

I fear you are correct. For the life of me, I don’t understand why people can see that getting vaccinated is our way out of this scourge.

[Don Johnson 4,753]

Just now, Satchel said:

I fear you are correct. For the life of me, I don’t understand why people can see that getting vaccinated is our way out of this scourge.

IMO, times up.  It takes 5 weeks with Pfizer and 6 weeks with Moderna to be "fully vacccinated."  Omicron ain't got time for that.

[Sbbruin 21,844]

16 minutes ago, Satchel said:

I favor bringing back Mutual of Obama’s Wild Kingdom and dart vaccinate the recalcitrant from helocopters.

Just have people walking around Floribamatucky with these.

 

[Satchel 6,938]

Just now, Sbbruin said:

Just have people walking around Floribamatucky with these.

 

Definitely worth a try. Shit’s past being real.

[MC Fresh Breath 4,074]

30 minutes ago, Sbbruin said:

Just have people walking around Floribamatucky with these.

 

“Floribamatucky” truly made me laugh out loud.  Thank you 

[DDD Dad 14,335]

37 minutes ago, Sbbruin said:

Just have people walking around Floribamatucky with these.

 

 

5 minutes ago, MC Fresh Breath said:

“Floribamatucky” truly made me laugh out loud.  Thank you 

"West Floribamatucky"

[PsychMike 1,550]

Doubt this turns into FAFO, but made me LOL nonetheless.

 

 

[MC Fresh Breath 4,074]

32 minutes ago, PsychMike said:

Doubt this turns into FAFO, but made me LOL nonetheless.

 

 

"You are now free to cough about the country."

 

[Walden Ponderer 12,639]

Doctor cleared me from quarantine today. I had a fever for about three days, and a runny nose for a few days more. I'm assuming it was Omicron, based on breakthrough and mild symptoms, but who the hell really knows anything any more.

[babysdaddy 2,721]

 

Feels pretty political.  And here's the declaration if you haven't read it.

https://gbdeclaration.org/

[Foosters 17,850]

dude

[Biff Tannen 45,676]

15 minutes ago, babysdaddy said:

 

Feels pretty political.  And here's the declaration if you haven't read it.

https://gbdeclaration.org/

Is this the group that goes around mutilating squirrels?

[MC Fresh Breath 4,074]

16 minutes ago, babysdaddy said:

 

Feels pretty political.  And here's the declaration if you haven't read it.

https://gbdeclaration.org/

lolwut?

 

[wildcat09 29,349]

27 minutes ago, babysdaddy said:

 

Feels pretty political.  And here's the declaration if you haven't read it.

https://gbdeclaration.org/

I love this tactic of posting things and insinuating that they’re nefarious but without saying so because the thing you’ve posted about is actually very clearly good.

[jimmyjazz 45,903]

50 minutes ago, babysdaddy said:

 

Feels pretty political.  And here's the declaration if you haven't read it.

https://gbdeclaration.org/

Oh come on.

[Sawbonz 27,547]

2 hours ago, Neonmoon said:

Omnicron also blocking Delta out?

I think this is what is happening. If it is much more infectious than Delta, and if asymptomatic people are out and about with it, it should quickly take over the vast majority of cases. I was in NYC last weekend and it was wall-to-wall people just like every Christmas season I’ve been up there. Feel like I must’ve been exposed, but zero symptoms. There are some cases going through my daughters dance company, but no close contacts have tested positive so far and she is symptom-free. In the absence of direct contact with a positive, I would not recommend testing If no symptoms 

[jimmyjazz 45,903]

Stepson just popped positive, as I suspected.  Fully vaxxed, no booster.  Symptomatic Wednesday morning (fever, headache, chills), mostly better by Thursday night.  Negative test Wednesday.  Inconclusive test this morning, positive test this afternoon.

I haven't spent much time around him this week, but 3-4 other family members have been closer than I over that time frame.  Two of them aren't vaccinated (one isn't eligible, the other is an idiot).

Merry f'n Christmas!  We all feel fine, but that means nothing.  No fevers.  Fingers crossed.

[Lurch 5,930]

I love this tactic of posting things and insinuating that they’re nefarious but without saying so because the thing you’ve posted about is actually very clearly good.

Just asking questions!

[jimmyjazz 45,903]

1 minute ago, Lurch said:

Just asking questions!

"Research".

[scottsins 1,966]

[deleted}

[Poolflood 845]

Didn't know where to drop this.

https://reason.com/volokh/2021/12/17/breaking-6th-circuit-dissolves-5th-circuit-stay-in-osha-mandate-case/?comments=true#comments

6th circuit dissolves the 5th's stay on OSHA mandate.

[brakeman 28,359]

https://twitter.com/musicrocksit452/status/1471869085948723201

 

this is fuck'd up. mean people still suck.

[Sawbonz 27,547]

[Incredulity 11,666]

46 minutes ago, Poolflood said:

Didn't know where to drop this.

https://reason.com/volokh/2021/12/17/breaking-6th-circuit-dissolves-5th-circuit-stay-in-osha-mandate-case/?comments=true#comments

6th circuit dissolves the 5th's stay on OSHA mandate.

Heard a news brief on the drive home this evening that Boeing was canceling their vaccine mandate.  Wonder if that was a coincidence or ?

Edit* heres a link

https://www.reuters.com/business/aerospace-defense/exclusive-boeing-suspends-vaccination-requirement-us-based-employees-internal-2021-12-17/

Edited December 18, 2021 by Incredulity