Great question! Yes, theoretically you can use a learned parametric curve (such as a DL model) to extrapolate outside of the training zone. But will this be meaningful, i.e. will it lead to generalization? Well, there are two possible settings.https://twitter.com/Gavin_Cawley/status/1450726111957209093 …
-
-
Your model priors can't make that jump -- not even close. They can barely move you from "here's this scene" to "here's the same scene but a bit cloudy".
Show this thread -
(Note: you could achieve this type of local, mini-extrapolation either via a brightness normalization step or via data augmentation, both of which encode prior knowledge about the characteristics of the visual world.)
Show this thread -
Meanwhile, a trivial example of meaningful extrapolation would be the case of a linear dataset. If your data fits on a line, *and* you make the strong (and correct) assumption that it fits on a line, then you can use your learned model to extrapolate outside of the training zone.
Show this thread -
But in most practical settings, the answer is no, you can't extrapolate. If you want models that can extrapolate in the real world, you should move away from differentiable curves and use… discrete search over graphs of logical operators (programs). This is for another thread...
Show this thread -
Also, if these topics sound interesting to you, remember -- there's this book you can grab.https://www.manning.com/books/deep-learning-with-python-second-edition?a_aid=keras&a_bid=76564dff …
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.