I will have to disagree. Many of the concepts in physics such as non-linear dynamics, phase transitions and criticality, renormalization, complex systems etc aren't in the vocabulary of deep learning research.
-
-
Replying to @IntuitMachine @Meaningness
Most papers that are submitted with physics inspired approaches are rejected from conferences. There is indeed a mono culture, but it's not due to a bias in physics. Maybe physics envy.
1 reply 0 retweets 1 like -
Replying to @IntuitMachine
That's interesting... so the monoculture may be specific to deep learning as such; something that developed just within that narrow field. (
@michael_nielsen, maybe this is the point you were making too?)2 replies 0 retweets 2 likes -
Replying to @Meaningness @IntuitMachine
Pretty much. There are useful ideas from physics in deep learning research, but I don't think any (currently) really key/dominant ideas are from physics. Maybe I'm not thinking of one?
3 replies 0 retweets 3 likes -
Replying to @michael_nielsen @IntuitMachine
I mean... the whole thing looks like physics to me? You’re shaking a ball on an energy surface.
2 replies 0 retweets 2 likes -
Replying to @Meaningness @IntuitMachine
Minimizing a cost function using SGD/backprop is like physics in much the same way as eigenvectors are part of quantum mechanics.
1 reply 0 retweets 8 likes -
It occurs to me that I don't know where the theory of mathematical optimization (& notions like objective function, gradient descent) comes from. Certainly, it was buzzing as a subject independent of physics by the 1950s. But its roots may be in part in physics, earlier.
5 replies 0 retweets 7 likes -
Replying to @michael_nielsen @IntuitMachine
One major strand in the history would be cybernetics/control theory, which was applied physics initially—the radar-controlled antiaircraft gun being the paradigm for the field
2 replies 0 retweets 6 likes -
Replying to @Meaningness @michael_nielsen
This is the long forgotten connection. What happened was that the Dartmouth AI folk didn't invite Wiener because his approach was very different from emerging computer paradigm.
2 replies 0 retweets 0 likes -
Replying to @IntuitMachine @michael_nielsen
Right. And that antipathy led to the Minsky/Papert Perceptrons book, and then the standard histrionics takes over. But the mid-80s backprop people were very big on physics analogies, as I recall it, and that shaped the texture of the field.
2 replies 0 retweets 1 like
Whoops, “history,” not “histrionics”!!
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.