I don’t know about this... Forecasts in general can definitely be wrong. Assigning a 99% chance to Clinton winning, for example, was a modeling error. If you don’t have the outcome in your prediction interval, from a modeling standpoint you very likely did something wrong.https://twitter.com/zeynep/status/1323649467015376896 …
-
Show this thread
-
Replying to @gelliottmorris
How is it wrong? Trump had one percent chance, that model claimed, and that happened. I don't think it was a great model, but I don't see how it's wrong. One percent events happen, obviously, and we worry about them and change our whole lives over it: see COVID CFR.
11 replies 1 retweet 17 likes -
Replying to @zeynep
If you build a model that predicts 5 million deaths from COVID over the next month, but we only have 50k, is it wrong?
5 replies 0 retweets 17 likes -
Replying to @gelliottmorris
With COVID, those models *very much* depend on behavior so they are even harder for epi models to be "wrong." It's an exponential dynamic, so yep, the range is huge. The question is were the assumptions justified? You rarely get that from outcome alone given that reflexive range.
3 replies 0 retweets 3 likes -
Replying to @zeynep
I think most modelers would not accept your exclusive definition of “wrong” as “a model that gave 100% odds to something that didn’t happen.” There are degrees of error in our modeling, and getting the outcome that far along the uncertainty interval is high on the curve.
2 replies 0 retweets 15 likes -
Replying to @gelliottmorris @zeynep
Anyway. We’re not going to make any progress here so maybe we can discuss in an academic setting or blog about it. I think your overall critique has some important truths to it, but the idea that we can’t diagnose and/or reject bad (“wrong”) models is silly.
2 replies 0 retweets 8 likes -
This Tweet is unavailable.
Yep, those are some of the good ways to evaluate model quality, rather than winner. I never said there is no way to evaluate model quality. I said it's wrong to focus on "winner" given both options are included as probable. There are lots of lots of ways to evaluate models.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.