Something that I keep seeing pop up is the idea that meta-analysis somehow eliminates issues with the underlying research This is just confusingly incorrect
-
Show this thread
-
Meta-analysis really is just a simple statistical aggregation of results. Indeed, the most basic way to meta-analyze studies is just to use a mean or median of their point-estimates
2 replies 5 retweets 59 likesShow this thread -
But simple means/medians can be misleading - we don't really want a study of 10,000 to have the same weighting as a study of 10, but that's what a simple average provides
2 replies 1 retweet 49 likesShow this thread -
When we run a meta-analytic model, what we are actually doing is generating a WEIGHTED mean/median, and confidence interval. Essentially, we take all of the means and SEs, and based on that our model weights them with bigger studies generally contributing more to the modelpic.twitter.com/8WrRkjVflu
1 reply 1 retweet 35 likesShow this thread -
You'll notice I've said nothing about the underlying quality of the evidence That's because META-ANALYSIS HAS NOTHING TO DO WITH QUALITY OF EVIDENCE
4 replies 6 retweets 74 likesShow this thread -
You can throw anything into a meta-analysis model. Here's a model I just ran on the ratio of hosting to participating in the summer Olympics. This is meaningless!pic.twitter.com/snEmahYVDg
2 replies 3 retweets 35 likesShow this thread -
We tend to put meta-analyses on a pedestal, but the fact is that statistically aggregating evidence is a total waste of time if that evidence is all bad
4 replies 19 retweets 114 likesShow this thread -
Health Nerd Retweeted Health Nerd
This recent Cochrane review is a perfect example - they looked at the evidence for ivermectin for COVID-19, but because most of it was terrible they only included a few studies in their modelhttps://twitter.com/GidMK/status/1420340231786549253?s=20 …
Health Nerd added,
Health NerdVerified account @GidMKCochrane review on ivermectin just dropped. This is the current gold-standard summary: "the reliable evidence available does not support the use ivermectin for treatment or prevention of COVID‐19 outside of well‐designed randomized trials." https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD015017.pub2/full …Show this thread6 replies 12 retweets 60 likesShow this thread -
This is also where the phrase "garbage in, garbage out" comes from. If your meta-analysis includes numbers from studies that are terrible, the final point estimate is as meaningless as my graph above on the Olympics
7 replies 8 retweets 90 likesShow this thread -
Replying to @GidMK
One of the issues I have with Cochrane in particular is that they throw out a LOT of studies, and what remains mushifies (a word I just made up) the issue instead of bringing clarity. Why publish a analysis at all if the conclusion is "all the studies are crap"?
2 replies 0 retweets 0 likes
Because that's an important finding. If the studies are insufficient to determine a conclusion, we shouldn't be promoting a treatment based on them!
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.