A lot of meta-analyses aren’t systematic reviews though. They’re like people going to a zoo, average all the animals they see - tigers, monkeys, birds— everything… and return home and tell a confident tale about the opossum they saw.
-
-
Replying to @RickCarlsson @NoahHaber
Polishing a pile of turds to make them shine like one big diamond, is what I always say.
1 reply 0 retweets 1 like -
Replying to @robinnkok @NoahHaber
Best systematic reviews be like:pic.twitter.com/T3Q3pW6ghH
1 reply 1 retweet 4 likes -
Replying to @RickCarlsson @robinnkok
Noah Haber Retweeted Rickard Carlsson
Why not? We can't do this for *publication* reasons, but certainly can for scientific rigor reasons. If the whole literature is a cesspool, the best number of studies to include in the meta-analysis is usually zero.https://twitter.com/RickCarlsson/status/1423013651636707333 …
Noah Haber added,
Rickard Carlsson @RickCarlssonReplying to @RickCarlsson @NoahHaber> so what’s the alternative? Ignore all shitty studies? I wish we could but no, we have to dig into the cesspool of literature we have and carefully summarize the best evidence. And explain why many studies didn’t even make it past abstract, or had bad design or risk of bias.1 reply 0 retweets 1 like -
Replying to @NoahHaber @robinnkok
If we completely ignore it, then others will use it and fault us for ignoring it. We have to deal with it. And yes, it’s not uncommon for eg a Cochrane review to conclude there are no studies of high quality. That’s a very important part of systematic reviews! >
1 reply 0 retweets 1 like -
> however I agree we should not set out to a meta-analysis it’s just a tool at the end IF we have a set of studies…
1 reply 0 retweets 0 likes -
Replying to @RickCarlsson @robinnkok
Which brings it all back around to my point. Being forced to have meta-analyses to try to fight off worse meta-analyses in situations where a systematic review would conclude that there is no meta-analysis-worthy studies available is not a sign of a functional system.
2 replies 0 retweets 2 likes -
Replying to @NoahHaber @robinnkok
Agreed. But once we have k > 3 studies we need to summarize it somehow and I’ve seen alternatives that are straight out of Lovecraft… So we need it much like we need statistical inference of some sort :)
1 reply 0 retweets 2 likes -
But yeah MA has so much misuse that we might as well do systematic reviews with multilevel modeling instead to avoid the term ;)
2 replies 0 retweets 1 like -
Replying to @RickCarlsson @robinnkok
I'll push back, since it's my "whole thing" as it were. We don't "need" to do any such thing, and "need" to do something unreliable and flawed is extraordinarily harmful. We have to learn to be fine doing nothing, because "nothing" is often the right thing to do.
2 replies 0 retweets 3 likes
Unfortunately it's less prestigious to get published in the Journal of Nothing than the Journal of American Moody Angst
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.