2/n I went over this in my article, which you can find here, but I'll repeat the precise analysis I've done on twitter, because I think it's a fair question https://twitter.com/GidMK/status/1415764372362649601?s=20 … To be clear - I'm not perfect, and not everything I say is right!
-
-
Show this thread
-
3/n Judge for yourself whether it is fair to argue that removing Elgazzar largely eliminates the benefit for mortality here, the headline finding and the primary analysis of the paper
Show this thread -
4/n So, here's the primary results from the Bryant et al meta-analysis of ivermectin that concluded "Moderate-certainty evidence finds that large reductions in COVID-19 deaths are possible using ivermectin"pic.twitter.com/CkCYWknA9X
Show this thread -
5/n Step 1: replicate the findings While there are very minor differences due to the statistical software used, this is the same Dersimonian-Laird IV model run using the metan command in Stata 15pic.twitter.com/TZOgZtAtNb
Show this thread -
6/n The benefit from this model is huge, very statistically significant, and you can see why the authors made such strong recommendations. There is a significant benefit for mild disease, verging on significance for severe, and overall 60% (!) reduction in death
Show this thread -
7/n Step 2: remove only Elgazzar Immediately, the difference is notable. The primary benefit goes from 60% to 44%, is only marginally statistically significant, and there's no benefit in either Mild or Severe subgroupspic.twitter.com/QHjWGPXLOn
Show this thread -
8/n Ok, so that's an issue. What were extremely strong results are now statistically very weak and not beneficial in the major subgroups But there's a bigger issue here
Show this thread -
9/n Step 3: add in new research This study was published shortly after Bryant et al came out. It would certainly have been included in the analysis if it was already out (no fault to the authors of course) so let's add it inpic.twitter.com/Wc8I7Qmf4d
Show this thread -
10/n The significance of the model entirely disappears. Now ivermectin shows no benefit for mortality at all, even in the overall analysispic.twitter.com/QH8tdyXi7U
Show this thread -
11/n But wait, there's more! If you go back to that graph excluding Elgazzar, there's something odd. You see only one study finding a significant benefit for ivermectinpic.twitter.com/2VZvGDOetB
Show this thread -
12/n In fact, it seems like the entire mortality benefit for ivermectin, once Elgazzar is excluded, comes down to this one study: Niaee et al
Show this thread -
13/n If you look at the Niaee et al preprint, there are quite a number of issues. In fact,
@K_Sheldrick noted that the results are worrying, and that the study "should not be included in meta-analyses"

pic.twitter.com/mWV8BvccnJ
Show this thread -
14/n So, Step 4: Sensitivity Analysis Rerunning the original model excluding both Elgazzar and Niaee gives you this result Absolutely no benefit for ivermectin on mortality. CIs include both large benefit and large detrimentpic.twitter.com/tVKlmxJgpp
Show this thread -
15/n Worth noting that removing these two studies from the analysis takes the I squared statistic down to 0% - in other words, the remaining studies are all statistically consistent with each other
Show this thread -
16/n What does this all mean? In short - once Elgazzar is excluded, the results are incredibly uncertain, and any potential benefit rests entirely on the Niaee study, which at least one expert has argued should never be included in meta-analyses
Show this thread -
17/n Moreover, because the result is so marginally significant, including a ~single~ new piece of evidence also results in the benefit disappearing even if you still include Niaee in the model
Show this thread -
18/n Therefore, in my opinion, this means that excluding Elgazzar removes any certainty and a lot of significance from this analysis
Show this thread -
19/n In other words the conclusion - that moderate-certainty evidence found large reductions in death using ivermectin - is entirely reversed. The certainty is gone, and the reduction in death is likely to be very substantially smaller
Show this thread -
20/n Indeed, since the benefit now appears to rest entirely on one very worrisome trial, it is hard to see how we can justify any argument other than that we do not have sufficient information to make a conclusion about ivermectin at this pointpic.twitter.com/39xLhAewpi
Show this thread -
21/n I remain optimistic that ivermectin will indeed prove to be a "wonder drug" as Elgazzar claimed, but I simply don't think the evidence to date supports that assertion
Show this thread -
22/n With large randomized trials ongoing, we can only wait for them to finish before making a strong judgement as to whether ivermectin is beneficial It may yet turn out to be fantastic. We simply do not know
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.