A lot of people in the world of evidence appraisal use the phrase "Garbage-In Garbage-Out" (GIGO) to describe bad meta-analyses I thought I'd briefly explain what this means 1/n
-
-
6/n A common method of doing this is to use the inverse of the variance of the studies. This essentially uses the confidence intervals of each study as a weighting tool, with tighter intervals getting a higher weightpic.twitter.com/yW7WF1YY8o
Show this thread -
7/n Here's an example from the ivermectin literature - this is simply a weighted average where the weighting is derived from the inverse of the variance (calculated from the confidence intervals)pic.twitter.com/ux75gk6HFz
Show this thread -
8/n But you can immediately see the issue here - this weighting is DERIVED FROM THE DATA If the data is unreliable, the weighting is meaningless!pic.twitter.com/7Noojnz1iQ
Show this thread -
9/n If I were to, say, fabricate a large trial that had a heavy weight, it would completely mess up the meta-analysis and make the results unreliable For ivermectin, we KNOW that this has happenedhttps://gidmk.medium.com/is-ivermectin-for-covid-19-based-on-fraudulent-research-5cc079278602 …
Show this thread -
10/n So this is what we mean by GIGO. If you incorporate bad numbers into a meta-analysis, by definition the results are also bad, because the model is simply an average of the numbers you input
Show this thread -
11/n This is why most of the work of meta-analysis is to carefully choose the studies you use, because if your model is based on garbage the results will also be garbage
Show this thread -
12/n A good way to think of this for laypeople is to think of a simple, boring average Would you average out the included studies? If not, the meta-analysis probably doesn't make sense
Show this thread -
13/n I think the website ivmmeta is actually a brilliant teaching tool for what not to do here. Can you imagine adding up the time it takes for people to recover and the proportion of people who had symptoms and dividing by 2? What would that even mean?pic.twitter.com/Cd9Bg2rWsV
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.