When we conduct a study looking at negative outcomes, what we usually want to see is if the bad thing happens more in one group than anotherpic.twitter.com/hUbMNj3Jan
-
-
Show this thread
-
There are a number of ways of doing this The most intuitive is the relative risk ratiopic.twitter.com/2VJ3cuO4Cx
Show this thread -
Risk is simple and easy to understand - it's essentially the percentage chance of something happening For example, say 10 in 100 people have an event. The risk is just 10/100 = 10%
Show this thread -
Relative risk ratios are also pretty easy. It's just the risk of one thing divided by another Let's say we've got 2 groups A and B. A has a 10% risk of the event, B has a 20% risk What's the risk ratio of B:A?
Show this thread -
As I said, it's easy and intuitive The risk in group B is double that of A This is how VIRTUALLY ALL DIFFERENCES are represented in the media
Show this thread -
The problem is, sometimes relative risk doesn't make sense to use. In those cases, what's often used instead as an estimate of the risk ratio is what's called an odds ratio
Show this thread -
Odds are fairly simple to calculate, but a bit less intuitive than risks The odds of something happening are the events divided by the non-events So if 10 in 100 people have an event, the ODDS are 10/90 = 0.11
Show this thread -
The ODDS RATIO is the ratio of one odds to another Take the same example - 10/100 events in group A, 20/100 events in group B What's the odds ratio?
Show this thread -
Well, the first odds we've worked out already - 0.11 For group B, it's 20/80 = 0.25 So the ratio of the odds B:A is 0.25/0.11 = 2.25
Show this thread -
So our relative risk ratio is 2 Our odds ratio is 2.25 They are different, but used interchangeably by the press!pic.twitter.com/aq9Aw0jduN
Show this thread -
The thing is, by definition odds ratios are higher than relative risk ratios - it's mathematically certain When the risks are low, this effect is small, but if the risks are big it's very noticeable
Show this thread -
For example, if your two risks are 0.01% and 0.02%, the risk ratio is 2 and the odds ratio is: (0.02/99.98)/(0.01/99.99) = 2.0002 Barely different
Show this thread -
But if the two risks are 20% and 40%, the risk ratio is still 2 (40/20) but the odds ratio becomes VERY different: (40/60)/(20/80) = 2.67 That's a lot higher!pic.twitter.com/73Fc7CxeUU
Show this thread -
Going back to this headline that I picked up - it looks at a study that used logistic regression, which spits out odds ratiospic.twitter.com/8vHaEBaLIS
Show this thread -
The results were reported as odds, with vapers having a 1.83 times higher odds of stroke than non-vapers Given that the prevalence of stroke was 2-4% in the groups, that means that the risk ratio would be a bit lower than 1.83
Show this thread -
In other words, the headline rounds UP from 83% to a 100% increase (double), but in actual fact it's more likely that the risk is somewhere around 75% instead!
Show this thread -
And this is done almost ubiquitously across the board It's not really the media's fault - scientists do it all the time as well!pic.twitter.com/BjxCDlughU
Show this thread -
It's also really hard to tell if the study used risks or odds unless you actually read it, which adds a layer of complexity to the matter
Show this thread -
Honestly, I want to end on a nice easy note, but the fact is that odds are confusing, a lot of researchers get them wrong, and it's unlikely we'll have a solution to this any time soon Hurray!
Show this thread -
Anyway, a reasonable proportions of the headlines you've seen probably overestimate the actual risk because the studies used odds ratios
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.