2/n Paper is here. It's actually very simple, basically the authors took an estimate of how many days of school kids lost due to closures during COVID-19, an estimate of how much that impacts their years of life, and multiplied outhttps://jamanetwork.com/journals/jamanetworkopen/fullarticle/2772834 …
-
-
12/n But, if we're comparing the total impact of missed education on children over a lifetime, this makes no sense. COVID-19 has many other harms, after all, some of which are disputed but many of which are easily attributable
Show this thread -
13/n Where are the years of lost life due to ICU stays? Hospitalizations? Disability, mental health issues, avoided care due to full hospitals, etc etc etc???
Show this thread -
14/n (This is also a problem for the YLL calculation for children missing school, FYI, it is FAR more complex than just missing educational attainment. So many potential things that are not included here!)
Show this thread -
15/n Ultimately, what we get in the end - the "98.1% probability" figure - makes almost no sense at all. Even if the Argentinian study was applicable to the US situation, which it clearly isn't, the counterfactual is just inappropriate
Show this thread -
16/n It may be possible to do a rigorous cost/benefit of school closures, but I really don't see how this can be considered one
Show this thread -
17/n At best, what we've got here is a very rough inference about what the impact of school closures could possibly have been (as long as US kids in 2020 are the same as Argentinians in 1977), compared to an undercount of the impact of what COVID-19 probably was
Show this thread -
18/n I should add - I am not saying that closing schools is necessarily a positive in this thread. What I AM saying, however, is that we could not possibly know whether it was or not from this paper
Show this thread -
19/n Many further criticisms of this paper that I missed in this also excellent thread. It appears that almost none of the assumptions in the paper make any sense at allhttps://twitter.com/ikashnitsky/status/1328121853970436100?s=20 …
Show this thread -
20/n Another point I've been thinking about - the authors fail to consider the impact of an ongoing epidemic on school attendance. Lots of cases in the local area probably impacts schooling as well!
Show this thread -
21/n Also, what's the impact of parents/carers getting sick and dying? Presumably this will have an outsized impact on the children it affects, which would cause (based on this model) large increases in YLL
Show this thread -
22/n Ok, there's another HUGE problem with the study The findings are entirely based on the idea that an extra year of schooling reduces your risk of death by 25%, a figure reached by aggregating studies in a meta-analysis modelpic.twitter.com/va8PWXeBc3
Show this thread -
23/n This number is the ENTIRE BASIS for the paper. If schooling reduces mortality by less than 25%, then the number of deaths caused by missed school days will drop as well And we have a problem, because this 25% figure is...wrong
Show this thread -
24/n Firstly, we have these top 2 papers. Because they're from the US, they are more heavily weighted than the others (this is arbitrary, but whatever) Except the values for the second paper are flatly incorrectpic.twitter.com/VxC2Krkj3z
Show this thread -
25/n Mazumder (2008) did not find a relative risk of 0.65 (0-1.3), it found an RR of 0.89 (0.74-1.04) The paper is here. I've read it very carefully and this appears to be the main finding from the author https://www.chicagofed.org/publications/economic-perspectives/2008/2qtr2008-part1-mazumder …pic.twitter.com/1FY15LCXFe
Show this thread -
26/n So the number is wrong, but this brings us to another issue - the two papers at the top that were double-weighted...are ON THE SAME DATASET This is a massive issuepic.twitter.com/trCaHySTYE
Show this thread -
27/n You see, Lleras-Muney compiled an estimate based on a number of factors on the reduction in risk from 1 year of extra schooling. Mazumder, three years later, took that same database and re-analyzed it considering additional factors, and estimated a lower reductionpic.twitter.com/2pWUNsP6Qb
Show this thread -
28/n However you slice it, it's inappropriate to just bung these two estimates together into a model and treat them as separate. I would argue that the Mazumder paper is probably a better estimate, but either way what they've done here is wrong
Show this thread -
29/n If I re-run the meta-analysis they've purported to use with the correct numbers, we end up with this graph instead. A relative reduction in risk of death of 5% for every year of schooling This is A QUARTER of the estimate used in the studypic.twitter.com/xQk5F1tDzT
Show this thread -
30/n So AT BEST assuming there are no other errors, the estimate of the years of life lost due to school closures should be divided by 5
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
post-pub peer-review THREAD
1/