Here's a question - why aren't we fact-checking systematic reviews and meta-analyses?
-
Show this thread
-
I'm not talking going through risk of bias assessments or even the meta-analytic code, but just simple things like whether all the studies meet the inclusion criteria or running the search terms and seeing if the numbers match their PRISMA diagram
1 reply 0 retweets 2 likesShow this thread -
The reason I'm so interested is that the first study I tried this on - one that I've cited in my blog before - failed these basic checks They included a cohort study despite specifying only RCTspic.twitter.com/K9a5Vm4wPR
1 reply 0 retweets 5 likesShow this thread -
They also - and this is the kicker - rated the random sequence generation of the trial as "at low risk of bias" despite there being no random sequence generation (the study didn't even use the word "random" or any similar term)pic.twitter.com/b8B1fs5j6Q
1 reply 0 retweets 3 likesShow this thread -
The study is here check for yourself: http://www.annfammed.org/content/11/5/452.long#ref-6 …
1 reply 0 retweets 1 likeShow this thread -
Thing is, that's maybe not a big error. A single matched cohort included in a review that supposedly only looked at RCTs But that was just the first review I picked out. I wonder how common this is?
1 reply 0 retweets 4 likesShow this thread -
Most people don't go through eligibility criteria and comb through studies to check systematic reviews - it's incredibly time-consuming and tedious. I know I don't Maybe we should
1 reply 0 retweets 5 likesShow this thread -
Replying to @GidMK
David Nunan Retweeted David Nunan
So did we. And found major problems: https://twitter.com/dnunan79/status/1092466433290878976 … And currently have a systematic assessment of 24 matched-pair MA’s submitted to
@JClinEpi doing exactly this. Just realised, I should pre-print this. Would this be ok@JClinEpi ?David Nunan added,
David Nunan @dnunan79ON WHY APPRAISAL OF EVIDENCE MATTERS Finally can reveal the story behind the retraction of a Heart BMJ paper and how this makes for a great teaching example on the importance of skills to appraise evidence. https://www.cebm.net/2019/02/salt-reduction-in-heart-failure-where-has-the-uncertainty-come-from/ … pic.twitter.com/HPo2w3Whrn2 replies 1 retweet 3 likes -
Definitely. I'd love to read it. Not even thinking about the in-depth stuff from the salt case, just boring basic checking against inclusion criteria and PRISMA to see if there are clear errors
2 replies 0 retweets 2 likes
Like...not even checking all the complex stuff such as correct ROB assessments. This is just me wondering why I rarely go through and check the basic aspects of the trial to see if they did what they said they did as I would do for a normal trial
-
-
Andrew Althouse Retweeted Ricky Turgeon PharmD
I have a similar example for you turned up by
@Ricky_Turgeon that has since been retracted:https://twitter.com/ricky_turgeon/status/1082524412685910018?s=21 …Andrew Althouse added,
Ricky Turgeon PharmD @Ricky_Turgeon@ESC_Journals Surprised that a meta-analysis with so many errors made it past peer review: https://academic.oup.com/eurheartj/advance-article-abstract/doi/10.1093/eurheartj/ehy833/5272474 … . I hope that by raising awareness of these issues, this article can be withdrawn and revised before it makes it into print 1/nShow this thread2 replies 1 retweet 3 likes -
Replying to @ADAlthousePhD @GidMK and
Andrew Althouse Retweeted European Society of Cardiology Journals
Andrew Althouse added,
European Society of Cardiology JournalsVerified account @ESC_JournalsReplying to @ADAlthousePhD @Ricky_Turgeon3/3 The EHJ has now published your Discussion Forum paper https://bit.ly/2XeY8pR and the Editor's response to your paper https://bit.ly/2GShM5f The scientific record is improved owing to your vigilance, so we sincerely thank you once again.1 reply 0 retweets 1 like - Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.