exactly!
-
-
-
I just don't buy it... Algs with good credit assignment will likely still way outperform those w/o. But look, I would be happy to be proven wrong. If someone can show good performance on hard tasks using architecture search + non-end-to-end training, cool. Ball's in your court!
- 20 more replies
New conversation -
-
-
I've read this paper and think it's great. Definitely shows the power of good architecture init. But, I don't think you can avoid cred assignment this way. On the contrary, architecture optimization plus good credit assignment is probably key to human level intelligence.
-
I think he's looking to contrast local learning rules (which implement credit assignment) with global ones. To which you might reply that local learning rules such as STDP have been and are actively being explored. Primarily by neuro-folk and those in neuromorphic computing. :)
- 5 more replies
New conversation -
-
-
Thanks for the discussion
@tyrell_turing@TonyZador ! -
ditto. loved it
End of conversation
New conversation -
-
-
I think for every NN, there is a weight-agnostic circuit that implements the same function. Searching for that circuit, is still 'learning', and likely much more expensive than backprop. The search doesn't seem to be using any local cues.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
That means the network has to learn its own structure for credit assignment. Which is probably how it works in brains. (How do I need to flex to make my model bend in the right way...)
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Without getting into the debate on credit assignment rules, I'd wonder the other way around: how well can we do architecture search/optimization to generate functional forms that are good at approximating the target function regardless of the weights in the linear layers?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
So, about that petition you signed for
@Joi...https://www.newyorker.com/news/news-desk/how-an-elite-university-research-center-concealed-its-relationship-with-jeffrey-epstein …Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.