The foundational tension in machine learning is between optimization and generalization. If you could define generalization as an objective, you could solve for it. But you can't! That's the point.
Show this thread
Fun point: it's important to remember that, in machine learning, the problem you're trying to solve is *always* misspecified. You can only *optimize* (fit to the data you have), but your actual goal is to *generalize* -- and generalization is fundamentally ill-defined.https://twitter.com/mariohsouto/status/992242166121431040 …
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
I like the clarity in calling this out here. In theory if we had access to infinite training data (assuming it were possible) we could do pure optimization and hope that factors of variation and features picked up are general rules as opposed to intricacies of the data?
-
I think It’ll follow the representative capacity of the model: If that had infinite capacity too, it’ll just memorise the whole set. If not, it’ll memorise some average within the limits of the model. Whether that’s general features or not depends on the model architecture.
End of conversation
New conversation -
-
-
Thanks for the reply. Coming from a math. opt. background these are really interesting statements. If you don't have a mathematical expression for generalization, can you at least measure it? Can you say model A generalizes better than model B?
-
Cross validation
- Show replies
New conversation -
-
-
Very interesting
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Morden day wisdom in geek
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Yes you are right and I think it’s because Customers looking for business insights. Generalisation is again correct because that’s what developers do and reason is it’s data should be pluggable.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Given the tools of the site, every John could make a
or airplane.Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Abstract and assume deficient?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.