2/n The paper is here, and it's an interesting read: https://onlinelibrary.wiley.com/doi/epdf/10.1111/eci.13484 …
-
-
Show this thread
-
3/n The paper takes 10 countries' worth of data, and compares their COVID-19 case numbers against the restrictions they had in place in early 2020, comparing those with less-restrictive non-pharmaceutical interventions (lrNPIs) with more-restrictive NPIs (mrNPIs)pic.twitter.com/9xsiRGG7rP
Show this thread -
4/n The basic findings are that, compared to South Korea and Sweden, the marginal impact of implementing mrNPIs was not possible to discern in this study In other words, "lockdowns" don't have a significant marginal benefit over a suite of other measurespic.twitter.com/ZvhlhvUdJV
Show this thread -
5/n This has actually been shown before, using much larger datasets and more rigorous (but still imperfect) analyses, so it's a bit odd that this particular paper has been seized on so much https://science.sciencemag.org/content/early/2020/12/15/science.abd9338 …pic.twitter.com/6mFveAJKeE
Show this thread -
6/n I should also say I have something of an intellectual conflict of interest here, because the authors conclude something that I've been saying since March - the interventions pursued may be less important than the way that they are implementedpic.twitter.com/Geu1J3TtwJ
Show this thread -
7/n All that being said, what's the science here Well, it's a bit...lacklustre reallypic.twitter.com/I1QmHqb1j3
Show this thread -
8/n The sample size is minuscule. 8 mrNPI and 2 lnNPI countries is far too few to make any realistic conclusions about much, especially given how wildly different these countries arepic.twitter.com/1I91oGQfBh
Show this thread -
9/n The authors claim that they used these countries because those were the only ones where they could get data for each administrative region, but if that's the case then they really can't have looked very hardpic.twitter.com/41d7yh4dy0
Show this thread -
10/n Off the top of my head, I can name at least half a dozen other countries for which case/policy data is available by day in every administrative region. Switzerland, Australia, New Zealand, China, Brazil, etc
Show this thread -
11/n Indeed, going by the references that the authors cite, this issue may have arisen because they used Statista as their primary source for case data, which is...not idealpic.twitter.com/W34W1uThYe
Show this thread -
12/n The definition of restrictions is also a bit weird. I mean, South Korea didn't forcibly close businesses, but they do have national legislation allowing practices that many countries would find very restrictive (as the references the authors cite show)pic.twitter.com/TzQWN8J2EF
Show this thread -
13/n There's also nothing in this paper about lags for any policy for implementation, how policies were associated with dates etc That's a huge issue!
Show this thread -
14/n We know that the lag between policy introduction, implementation, and outcome is not immediate, and this is likely to vary by country, so simply comparing them day-by-day as this paper appears to doesn't really give us any indication of their impact
Show this thread -
15/n Furthermore, the lrNPIs themselves are really poorly elucidated. This is FAR from a fair summation of the complex and detailed work South Korea put in to controlling COVID-19!pic.twitter.com/razBpmmN8h
Show this thread -
16/n I mean, reading the paper you might get the impression that all South Korea did was some optional social distancing, emergency declaration, and case quarantine, rather than a coordinated and multi-step approach including HUGE healthcare investment
Show this thread -
17/n There's also not much effort to disentangle the complexities of the marginal benefit of each intervention, unlike previous research. It's likely, for example, that closing schools in Sweden (that did little else) had a huge impact...
Show this thread -
18/n ...but that this was not as effective as in Italy, which had many interventions
Show this thread -
19/n The authors also use some fairly inappropriate causal language throughout. These are the potential benefits ASSOCIATED WITH the announcement of policies in each place, we certainly can't infer a causal impact herepic.twitter.com/RXGpCijYlD
Show this thread -
20/n In other words, there are innumerable confounding factors that may have made the interventions more/less effective, like the age structure of the population, how socially distanced they were pre-pandemic etc...
Show this thread -
21/n At best, this study provides us with some evidence that mrNPIs are not associated with a large marginal benefit in terms of case numbers over lnNPIs, when comparing a tiny group of dissimilar nations
Show this thread -
22/n More realistically, I think we can probably say that the paper tells us little useful except that analysing the impact of NPIs generally is extremely hard
Show this thread -
23/n This is a bit of a shame, because I actually agree wholeheartedly with the authors that there is a cost to restrictive NPIs, and the marginal benefit of (say) stay-at-home orders is likely to be quite small in many circumstances
Show this thread -
24/n That being said, this paper just doesn't tell us anything useful about these mrNPIs beyond some more very vague evidence that they may not be as beneficial on top of other interventions (maybe)
Show this thread -
25/n Ultimately, the authors may have failed to find a benefit of business closures or stay-at-home orders, but the methodology used just doesn't give us enough information to say much, if anything, conclusivelypic.twitter.com/0yrwOfkA4q
Show this thread -
26/n Some more issues with the study, which gets worse and worse the more you look at it!https://twitter.com/AndreasShrugged/status/1349464781145731073?s=20 …
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.