Let’s add meat to this then, though. What are good options for credit assignment that *don’t* require efficient access to an estimate of the 1st order gradient of an objective function w.r.t. a given synaptic weight deep in a network? Honest question.
This may well be right. I certainly can’t say I’m confident it will be figured out with the level of data we have now, plus thinking. But I think BP and BP are two good broad classes of possibility that are not vacuous and tend toward very different predictions when fleshed out.
-
-
From what I've seen after a decade in the field, people who like deep learning and people who like Bayes can take the same data and claim it supports their favorite type of BP. I'd like to know what the different experimental predictions would be.
-
But more broadly - and I'd have to think more carefully about this to be sure it's true - I suspect that any fixed point iteration can be reframed as an optimization problem. Which, if true, would take "the brain does optimization" to "neural dynamics converge to a fixed point".
- 6 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.