4/n And if you read the methods section of this review, it's fairly reasonable. Good exclusion criteria, risk of bias assessment, it ticks all the boxes! But as ever, the devil is in the detailspic.twitter.com/qFUyVGbAFV
You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
15/n I haven't gone through every study yet, but every one that I have is similarly worrying. I can't actually find a study that I'm not worried about in this analysis
16/n If I go through the studies, I really can't find any that I would rate as a "low" risk of bias. Most of them are "high", some of them are really worryingly bad
17/n Given this, I'm not sure the conclusions of the review make much sense at all. How can you interpret studies of this quality as moderate/strong evidence???pic.twitter.com/mvlxiUAvJX
18/n I'm also worried about the heterogeneity of the 'usual care' groups across studies. Some of the usual care: - coffee - paracetamol - syrup - diphenhydramine - placebo - prednisolone These are very much not the same!
19/n That's a big issue because you can't really combine the effect of coffee with diphenhydramine and expect it to make sense, but that's what the authors did
20/n Ultimately, I think the only real conclusion you can draw here is that we have very little idea whether honey reduces symptoms for URTI/cough, and that the research is quite problematic
21/n Not something that'll make headlines, perhaps, but sadly that's often how these things go
22/n N.B. this study has already hit ~800 on Altmetric, been covered internationally, and made huge newspic.twitter.com/bj72yqShfA
23/n Something else I didn't mention. Every study that I've looked at so far used a per-protocol analysis, which is a huge and worrying issue All of these should be at a high risk of attrition bias, yet none were rated as such. Most of them were green (low risk)pic.twitter.com/Y6sXlnkJHe
24/n I've gotta say, for anyone teaching students about how finicky bias can be in systematic reviews, this is a beautiful example of getting it wrong
25/n Somehow there are more issues here. This study was included in the risk of bias, but even though it assessed honey vs placebo/salbutamonl (and found no effect) it is not in any of the meta-analyses Very weirdpic.twitter.com/8AaMA85ERm
26/n This is even weirder when you consider that the authors report excluding a study for not providing data So they exclude one study and report it, but another just...disappears? So strangepic.twitter.com/oyrp57yP1Y
27/n Another included study using a per-protocol analysis. This was at least rated correctly as at a high risk of biaspic.twitter.com/t2mRMIq539
28/n Another one. This study was rated at low risk of bias for most domains. Here's how they described their randomization and allocation concealment. What do you think?pic.twitter.com/1NM606xcRg
29/n I'll give you a head-start - if they literally don't report ~how~ patients were randomized, by definition this should be unclear or high risk of bias for the domain of random sequence generation
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.