I think that the brain almost certainly approximates gradient descent. And here is why: Any learning episode only appears to change the brain a tiny bit. 1/5
-
Show this thread
-
Replying to @KordingLab
I thought "Intrinsic Gradient Networks", Jason Tyler Rolfe's PhD Thesis (2012, doi:10.7907/YCB7-7X24. https://resolver.caltech.edu/CaltechTHESIS:04202012-133210844 …) was an underappreciated approach to the problem of 𝛁∈
2 replies 9 retweets 32 likes -
Replying to @BAPearlmutter
Omg. Why didn't I know of it!
2 replies 0 retweets 5 likes -
-
Fascinating that one of the intrinsic gradient networks is... belief propagation. Rolfe’s earlier MS thesis has a dendritic + columnar model for sum-product BP. Once can do BP itself, gradient of BP is easy. Comes full circle with
@dileeplearning’s comments on the brain GD issue?1 reply 0 retweets 4 likes -
Replying to @AdamMarblestone @tyrell_turing and
Adam, do you have a link to his MS thesis?
1 reply 0 retweets 1 like -
Replying to @dileeplearning @tyrell_turing and2 replies 0 retweets 4 likes
(With these bits he basically sets the stage for 6 years later saying BP is an “intrinsic gradient“ system and then generalized that concept to other dynamics like specific kinds of RNNs...)pic.twitter.com/dpQpuQwwkP
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.