The new normal.
Old: Memorizing the data is overfitting. New: Memorizing the data is essential for good generalization.
-
-
-
Old. A pseudoinverse might be ill-conditioned. New. But maybe not.
Keskustelun loppu
Uusi keskustelu -
-
-
Overfitting is a small sample size phenomena.
-
Sure, but small is relative to the complexity you're seeking to describe.
- Näytä vastaukset
Uusi keskustelu -
-
-
Hmm... if we take a step back from Stats/ML mindset and think about how humans generalize, the second statement doesn't seem as strange sounding anymore. Would be weird if one day we find out that information retrieval by the brain is just some kind of locality sensitive hashing.
-
worth reflecting again on your definition of abstraction. You could say abstraction is a compression from perception to concepts. It grows not out of memorization of individual memories, but out of connections between memories. As Borges wrote "to think is to forget differences"
- Näytä vastaukset
Uusi keskustelu -
-
-
Classical view on what is overfitting, i.e., train performance is exceeding test performance then it is memorizing. I would call this 'overtraining'. In this sense, deep learning really generalised well when in reality we are memorising.
Kiitos. Käytämme tätä aikajanasi parantamiseen. KumoaKumoa
-
-
-
Torralba has been saying for decades that, "The aim of computer vision is to overfit to our visual world." http://www.cs.cmu.edu/~efros/courses/LBMV09/ …
-
Probably a reference to his 80 million tiny images work, i.e. ppl working with huge datasets always had the above view!
Keskustelun loppu
Uusi keskustelu -
-
-
Inherently one can always view memorization as part of learning. That said I believe the difference between overfitting and generalization lies in what you memorize. Overfitting memorizes info of samples in isolation, while generalization memorizes info over samples.
-
but not over samples as the convolutional today, which rotation of input point missing unfortunate. such notion learning geometric structure as orientable manifolds, perspective inherent to humanity fibered over attention path also objective known motive.
Keskustelun loppu
Uusi keskustelu -
Lataaminen näyttää kestävän hetken.
Twitter saattaa olla ruuhkautunut tai ongelma on muuten hetkellinen. Yritä uudelleen tai käy Twitterin tilasivulla saadaksesi lisätietoja.