Recently this paper was published purporting to show that staying at home does not prevent COVID-19 deaths I don't think the evidence provided supports that at all! Some peer-review on twitter 1/nhttps://twitter.com/jhnhellstrom/status/1368585541462208519 …
-
-
5/n The authors ensured that the countries/regions were reasonably comparable by controlling for a few population measures like markers of income and healthcarepic.twitter.com/RczWnQE97n
Show this thread -
6/n Now, the first issue is a pretty obvious one that springs out immediately: Google "residential" mobility data
Show this thread -
7/n Firstly, this is a selected dataset. Only people who use Google services (mostly Android users) AND HAVE LOCATION HISTORY TURNED ON are represented in this dataset Almost certainly not representative of the people who are mostly dying from COVID-19pic.twitter.com/ugcL4oPOvV
Show this thread -
8/n This is mentioned in a sentence in the discussion, but I think it's a fundamental issue that makes this analysis a bit useless. We know that 50%+ of COVID-19 deaths are in the over-65 population, who are the least likely to be represented in this dataset!
Show this thread -
9/n Furthermore, only using the "residential" data*, as the authors did, is a big problem You see, most people already spend most of their time at home *there's also an issue with how opaque the term "residential" is and how this is calculated, but one issue at a timepic.twitter.com/AOwzpHozXC
Show this thread -
10/n Google even points this out in the explainer for mobility data. Most people already spend 12+ of their 24 hours a day at home, so the "residential" category is the LEAST LIKELY to show any increase/decreasespic.twitter.com/nTEo19B0gP
Show this thread -
11/n It makes sense when you remember that Google mobility data tracks CHANGES, not absolute figures. So 50% of the population working from home 100% would reduce office mobility by 50%, but only increase residential by a fraction of that amount
Show this thread -
12/n For example, here is the "residential" vs "workplace" mobility data for the state of Victoria in Australia during their mammoth lockdown. "Residential" never goes above a 25% increase, but "workplace" decreases FAR morepic.twitter.com/iUfR2QGHcz
Show this thread -
13/n What this means is that by comparing "residential" mobility, you are the most likely to find no difference by default. This is called a bias towards the null, and it's not ideal
Show this thread -
14/n Furthermore, remember my asterisk from above? Yeh, turns out that it's really hard to find out what "residential" actually means, how it's calculated, or what the raw figures are based on, presumably because this is proprietary Google analysis
Show this thread -
15/n So the conclusions about staying at home make no sense at all. "Residential" mobility data might not have been different between places, but for all we know that's a meaningless measure anyway that has very little to do with how much people stay at homepic.twitter.com/6BZyfOXqj7
Show this thread -
16/n On top of that, this study suffers from the same drawbacks that most ecological trials do. To their credit, the authors acknowledge this in the discussion, but it certainly hasn't filtered through to the publicpic.twitter.com/XMeDYJanin
Show this thread -
17/n Limitations inherent in this sort of research are many and varied, but as one example it's hard to make any realistic inferences about individuals staying at home when your unit of study is Spain vs the United States of America
Show this thread -
18/n Even within Australia, which was included, the massive Victorian outbreak/lockdown skew the figures enormously, because one state with 1/4 of the population locked down while the rest of the country opened up
Show this thread -
19/n We might actually expect null findings from an ecological trial of this sort, because at the country level heterogeneity in local policy irons out a lot of the impact
Show this thread -
20/n It's also worth noting that the study literally does not address the question of whether government orders influenced COVID-19 deaths. Even if you ignore all the other issues, "residential" mobility data simply can't answer that question!
Show this thread -
21/n There are many reasons that people stay at home, and given the opacity of "residential" data it's hard to say much about the results other than that this is a hard question that we may never answer well
Show this thread -
22/n That being said, the idea that this study disproves staying at home as a driver of COVID-19 mortality is obviously wrong - at best, it is an example of how difficult answering that question can be
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.