Visual perception relies on an initial "feedforward" sweep of information through processing stages in visual cortex, which is refined by recurrent "feedback" computations. 2/17
-
-
Prikaži ovu nit
-
Feedforward processing forms the basis for modern deep neural networks (DNNs). However, the feedforward sweep in visual cortex is likely shallower than DNNs http://bit.ly/3956nuH , and feedback is responsible for much of the flexibility and robustness of biological vision. 3/17
Prikaži ovu nit -
We want to understand the computational principles of feedback, and incorporate these principles into models for artificial vision. We begin with a circuit model of feedback, which we extend into a fully differentiable module that can be trained "end-to-end" in DNNs.pic.twitter.com/xUg54UZmbA
Prikaži ovu nit -
With this deep feedback model we ask: (1) What do the corner cases of biological vision tell us about feedback? http://bit.ly/iclrfob (2) What is the computational role of different types of feedback connections that exist in cortex? http://bit.ly/iclrdis 5/17
Prikaži ovu nit -
Visual illusions are corner cases of perception, elicited by artificial/unlikely visual stimuli. Why do we have them? In http://bit.ly/iclrfob , we ask whether illusions are simply "bugs" of biological vision, or if they are necessary byproducts of recurrent computations. 6/17
Prikaži ovu nit -
We focus on the orientation-tilt illusion, where the perceived orientation of a center grating is repulsed from the surround when the two are similar, and attracted to the surround when the two are dissimilar. 7/17pic.twitter.com/CfTiHOGNNr
Prikaži ovu nit -
Our deep feedback model exhibits the orientation-tilt illusion after being trained for contour detection. Feedforward models do not have this illusion, which we find is dependent on recurrent feedback (specifically for aficionados, spatially broad recurrent interactions) 8/17pic.twitter.com/QcLN0RJFDB
Prikaži ovu nit -
At the same time, our deep feedback model outperforms the state-of-the-art computer vision models for contour detection, particularly on small datasets. In other words, this feedback model needs less supervision to learn to detect object contours. 9/17pic.twitter.com/71GxGEQx8i
Prikaži ovu nit -
How does the orientation-tilt illusion influence contour detection? We test this by correcting the illusion in our model. The illusion-corrected model becomes far worse at object contour detection, and begins to prefer "low-level" non-object contours. 10/17pic.twitter.com/xuxpXBcHcw
Prikaži ovu nit -
http://bit.ly/iclrfob shows that at least one kind of illusion is an important constraint for building feedback models! It remains to be seen whether this holds in other visual domains (i.e., color, motion, depth), but we are working hard on these questions at the moment! 11/17
Prikaži ovu nit -
In our other paper, http://bit.ly/iclrdis , we explore a long-standing question in vision science: What are the computational roles of different forms of feedback connections in visual cortex? 12/17pic.twitter.com/2a5sxSUHgL
Prikaži ovu nit -
Feedback connections in visual cortex have broadly been split into local "horizontal" connections and long-range "top-down" connections. We created two datasets to disentangle these connections, both of which ask whether two dots are on the same or different objects. 13/17
Prikaži ovu nit -
The first dataset is solved with "Gestalt": finding a dot and tracing to the other end of the long path. The second dataset is solved with semantics: recognizing one of the two letters, ignoring the other, and counting the dots on it. 14/17pic.twitter.com/ngdN507GtG
Prikaži ovu nit -
We train deep feedback models on these tasks, and find a double-dissociation. Horizontal connections are important for leveraging Gestalt, or in this case path tracing. Top-down connections are important for leveraging semantic information, and selecting letters. 15/17pic.twitter.com/3ZXcU5xfT6
Prikaži ovu nit -
Our feedback models are also significantly better than leading feedforward DNNs at capturing human decisions on these tasks. The recurrent visual strategies learned by the models are a better fit for human decision making than visual strategies learned by typical DNNs. 16/17pic.twitter.com/J3gYaBCWro
Prikaži ovu nit -
In sum: (Paper 1) Introduce a deep feedback model exhibits an orientation-tilt illusion and outperforms state-of-the-art vision models in contour detection. (Paper 2) Test the computational role of feedback connections for solving complex visual tasks. See you at ICLR! 17/17
Prikaži ovu nit
Kraj razgovora
Novi razgovor -
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.