You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
Moreover, the sensory cue and movement related components have distinct time courses (mvmt-associated appears in first ~10 trials, sensory cue-associated over hundreds of trials) and also sum - a classic demonstration of independent inputs to a neuron.pic.twitter.com/e8VvPyLctd
Finally, once behavior has fully adapted to the task and will not change further (overtrained) - only then do we observe the canonical neural correlate of reward prediction error (RPE). In other words, after learning is complete and as a consequence of learning.pic.twitter.com/2Ye5xTbrG0
That RPE correlates are a consequence of learning is quite surprising. It has been argued that RPE signals in DA neurons are a cause of learning. A key point: we replicate observed RPE correlates _after_ learning and we replicate that DA stim is sufficient for reinforcement.
Our dataset rules out the now canonical interpretation of how a temporal difference like computations could be implemented to give rise to DA RPE correlates (e.g. Suri & Schultz 1999) providing strong evidence that RPE correlates do not arise from an error computation. But…
An implementation consistent with temporal integration of sensory cue-associated and action initiation-associated inputs + Hebbian plasticity can explain DA correlates both on the timescale of learning (hours/days) and of synaptic integration (millisec) http://dudmanlab.org/html/learnda.html …pic.twitter.com/5eNQtU4TR2
Thus, we conclude that the timing of action is a critical determinant of reward prediction correlates in the activity of DA neurons during novel associative learning. This has important implications for the learning rules that underly DA-dependent learning.
In particular, it suggests that correlation-like signals in the activity of mDA neurons may allow a novice animal to rapidly learn from its successes - namely, when an appetitive action is initiated in response to a predictive cue - akin to Hebbian conceptualizations of learning.
The independence of cue-related and reward-delivery responses + the fact that DA responses lag after learning provides support for the notion that changes in DA activity are a consequence of learning in other brain circuits. Eg, amygdala https://goo.gl/KKxT3S
Onward... how is learning in distributed brain circuits coordinated to control behavior. Two things we propose from this study: Novel learning (ie before overtraining coordinates multiple circuits) and a detailed examination of behavior are key to focus on in future work. Fin.
this is really fascinating work, Josh!
Thanks. It’s a real pleasure working with @fluketc
is the implication that you don't get RPE before because otherwise you can't actually predict the reward, or something else?
Something else. We see behav. evidence of prediction starting in session 2, quite robust by session 4-8 (Fig1c,d, Fig4). So prediction is there, it just doesn't directly modulate DA responses. Prediction has to modulate behavior in major ways before effects are seen in DA
I love this. In our DA photometry recordings in recent paper we don't see clear cue responses until a few days in, but behavior is starting to emerge in the first day or two. And only after behavior is well established does the cue-evoked signal correlate with it.
What do you think this says about incentive salience models? I've thought they would predict a very strong correlation between cue signals and the behavior that follows, but we definitely don't see that on a trial-by-trial basis, and we also see the dissociation you describe
(Not to be totally negative about everything, the context is that a number of our other observations might fit with incentive salience pretty well)
Yeah. Been thinking about this and I don't have a good answer/conclusion. I tend to think all of these ideas are partly true, & 1 issue is that new versus well learned actions engage dopamine differently - which theoretical conceptions usually ignore.
I'm also interested in redoing some of those opto conditioning studies with terminal recordings. But yeah it's clear that incentive salience attribution lags behavioral acquisition. Though also not all conditioned behaviors necessarily reflect incentive salience.
For example simple orienting to DA predictive cues emerges early, before gross locomotion and specific actions like cue-directed approach. The emergence of approach specifically correlates better w emergence of cue-evoked signal. Need to get a better handle on that relationship
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
