2/ In this piece, we argue that neuroscience would benefit from adopting a framework that parallels the approach to designing intelligent systems used in deep learning.
-
-
Prikaži ovu nit
-
3/ ANN researchers do not attempt to design specific computations by-hand, instead, they design three core components: (1) architectures, (2) objective functions (cost/loss), and (3) learning rules that provide good inductive biases for learning how to do specific computations.pic.twitter.com/MK2w5pQAgb
Prikaži ovu nit -
4/ Why is this a good approach for ANNs? Because human-interpretable, hand-designed computations may not be the best solution for many complicated tasks. Instead, by optimising the networks after designing the three components above, non-intuitive solutions can be found.
Prikaži ovu nit -
5/ Similarly, we suggest that the brain's computations are a result of two optimisation processes, evolution and learning within the lifetime. And, like ANNs, we argue that there is no reason to believe that the solutions discovered by the brain are easy to describe/comprehend.
Prikaži ovu nit -
6/ Thus, we suggest neuroscience would benefit from trying to frame our investigations with respect to the architectures, objective functions, and learning rules that have shaped the brain.
Prikaži ovu nit -
7/ Importantly, this is not a claim that the brain is best approximated by a deep convent, or something facile like that. Rather, it's a call to recognise that we may gain much more traction by using these three components to frame our theories and experiments.
Prikaži ovu nit -
8/ As an analogy: when we study the phylogeny of a species in ecology, we frame the data using mechanisms that explain what guided the species' evolution, e.g. natural selection, sexual selection, niche, etc. We don't try to explain the species' phenotype without these concepts.
Prikaži ovu nit -
9/ Similarly, when we try to understand a given brain, we should, whenever we can, frame the computations using the concepts of the architectures, objective fns and learning rules that would have guided the emergence of those computations, both over evolution or in a lifetime.
Prikaži ovu nit -
10/ Another important point: we are not denying the existence of innate behaviour here. Indeed, an important point about deep learning, that is often missed, is that deep learning is not actually a "blank slate" approach.
Prikaži ovu nit -
11/ Only by crafting the right inductive biases for an ANN (via the three components), can we get good performance. Likewise, innate behaviours are intimately linked with these three components, even learning rules (since innate behaviours are still tunable).
Prikaži ovu nit -
12/ We believe that both theorists and experimentalists should start to view neural computation with these three components, using as central conceptual tools for understanding how the computations that they study actually came to be.
Prikaži ovu nit -
13/ Finally, I just want to say that this work was the result of a workshop at the Bellairs Institute of
@mcgillu organised in large part by some of the folks at@element_ai. Thanks to both of them! We couldn't have done this without their support.Prikaži ovu nit -
14/ Thanks to all my co-authors (31 of them)! Special shout out to Denis Therien from
@element_ai for being the major organiser, and to Tim Lillicrap and@KordingLab for really pushing forward on the writing with me.Prikaži ovu nit -
15/ Also, shout out to
@colleenjgillon for her wonderful figures (with contributions from@Pieters_Tweet,@somnirons and Andrew Saxe)!Prikaži ovu nit -
16/ This was truly a joint paper, and all the co-authors contributed to the ideas and writing herein. Thanks everyone for your work on this!
Prikaži ovu nit -
Fin/ CCing the other co-authors I know are on Twitter and who are not tagged yet:
@PhilBeaudoin@achristensen56@SuryaGanguli@KepecsLab@NKriegeskorte@neurograce@kendmil@NeuroNaud@YiotaPoirazi@AnnaSchapiro@dyamins@hisspikenessPrikaži ovu nit
Kraj razgovora
Novi razgovor -
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.