Something that I keep seeing pop up is the idea that meta-analysis somehow eliminates issues with the underlying research This is just confusingly incorrect
-
-
But simple means/medians can be misleading - we don't really want a study of 10,000 to have the same weighting as a study of 10, but that's what a simple average provides
Show this thread -
When we run a meta-analytic model, what we are actually doing is generating a WEIGHTED mean/median, and confidence interval. Essentially, we take all of the means and SEs, and based on that our model weights them with bigger studies generally contributing more to the modelpic.twitter.com/8WrRkjVflu
Show this thread -
You'll notice I've said nothing about the underlying quality of the evidence That's because META-ANALYSIS HAS NOTHING TO DO WITH QUALITY OF EVIDENCE
Show this thread -
You can throw anything into a meta-analysis model. Here's a model I just ran on the ratio of hosting to participating in the summer Olympics. This is meaningless!pic.twitter.com/snEmahYVDg
Show this thread -
We tend to put meta-analyses on a pedestal, but the fact is that statistically aggregating evidence is a total waste of time if that evidence is all bad
Show this thread -
This recent Cochrane review is a perfect example - they looked at the evidence for ivermectin for COVID-19, but because most of it was terrible they only included a few studies in their modelhttps://twitter.com/GidMK/status/1420340231786549253?s=20 …
Show this thread -
This is also where the phrase "garbage in, garbage out" comes from. If your meta-analysis includes numbers from studies that are terrible, the final point estimate is as meaningless as my graph above on the Olympics
Show this thread -
Some people think meta-analysis is impressive because it involves fancy statistical software, but it's entirely possible to implement a Dersimonian-Laird inverse-variance model in Excel with a stats textbook and a few hours of time
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.