I don’t know about this... Forecasts in general can definitely be wrong. Assigning a 99% chance to Clinton winning, for example, was a modeling error. If you don’t have the outcome in your prediction interval, from a modeling standpoint you very likely did something wrong.https://twitter.com/zeynep/status/1323649467015376896 …
-
Show this thread
-
Replying to @gelliottmorris
How is it wrong? Trump had one percent chance, that model claimed, and that happened. I don't think it was a great model, but I don't see how it's wrong. One percent events happen, obviously, and we worry about them and change our whole lives over it: see COVID CFR.
11 replies 1 retweet 17 likes -
Replying to @zeynep
If you build a model that predicts 5 million deaths from COVID over the next month, but we only have 50k, is it wrong?
5 replies 0 retweets 17 likes -
Replying to @gelliottmorris
With COVID, those models *very much* depend on behavior so they are even harder for epi models to be "wrong." It's an exponential dynamic, so yep, the range is huge. The question is were the assumptions justified? You rarely get that from outcome alone given that reflexive range.
3 replies 0 retweets 3 likes -
Replying to @zeynep
I think most modelers would not accept your exclusive definition of “wrong” as “a model that gave 100% odds to something that didn’t happen.” There are degrees of error in our modeling, and getting the outcome that far along the uncertainty interval is high on the curve.
2 replies 0 retweets 15 likes -
Replying to @gelliottmorris @zeynep
Anyway. We’re not going to make any progress here so maybe we can discuss in an academic setting or blog about it. I think your overall critique has some important truths to it, but the idea that we can’t diagnose and/or reject bad (“wrong”) models is silly.
2 replies 0 retweets 8 likes -
Replying to @gelliottmorris
My assertion is that especially for events where the forecast changes behavior, and where there is a lot of sources of uncertainty exogenous to the model, outcome by itself does not show which model is lower quality, and also that "bad" and "wrong" are not the same thing. 1/2
1 reply 0 retweets 0 likes
But agree on lack on progress here. I look forward to your post defining "wrong" for such forecasts, because it would be helpful.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.