Easy take: calling linear algebra “quantum physics” and “artificial intelligence” is hype. Deeper: the huge influx of physicists into AI has produced an intellectual monoculture that isn’t capable of addressing key problems in the field.https://twitter.com/WIRED/status/1181437300414275584 …
-
Minimizing a cost function using SGD/backprop is like physics in much the same way as eigenvectors are part of quantum mechanics.
It occurs to me that I don't know where the theory of mathematical optimization (& notions like objective function, gradient descent) comes from. Certainly, it was buzzing as a subject independent of physics by the 1950s. But its roots may be in part in physics, earlier.
-
-
One major strand in the history would be cybernetics/control theory, which was applied physics initially—the radar-controlled antiaircraft gun being the paradigm for the field
-
The MIT Radiation Lab was the biggest contributor I think
End of conversation
New conversation -
-
-
The divergence must be very long ago. Lagrange multiplier (i.e. regularization) is very old. Only the physicists know what a Lagrangian is! The concept of Hamiltonian or Lagrangian Dynamics is alien.
-
Ok, this is weird to me, because as I remember it the connection with Hamiltonian dynamics was super salient in the field when I was involved. That was 30 years ago and I guess everything is different.
- 3 more replies
New conversation -
-
-
Gradient descent was done by Cauchy in 1847 https://www.math.uni-bielefeld.de/documenta/vol-ismp/40_lemarechal-claude.pdf …
-
This attributes SGD to a 1951 paper by two mathematicians, both with eclectic interests:https://stats.stackexchange.com/questions/313681/who-invented-stochastic-gradient-descent …
End of conversation
New conversation -
-
-
In particular: consider the equations of gradient descent in a 2-dimensional configuration space. Then take the small learning rate limit (effectively, moving to continuous time). The resulting equations are completely unphysical, & nothing like "a ball rolling down a hill".
- 1 more reply
New conversation -
-
-
Probably arose in the context of differential equations one can't solve in closed form. Those might arise both in economic resource allocation as well as in engineering (space? nukes?)
-
Monte Carlo methods definitely came from nuclear simulation.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.