Hrvoje Stojic

Briefly, uncertainty plays an important role in both learning and choice in reinforcement learning. We go beyond inferring uncertainty's role from choices alone by focusing on people's eye movements during learning and choice.

By examining interaction between learning, choice and attention processes, we found signatures of uncertainty in visual fixation during both choice and learning. The value and uncertainty of options jointly influenced what subjects looked at before choosing.

Interestingly, there is also a component unrelated to value and uncertainty in fixation that strongly influenced choice. I cannot help but wonder that it might be a source of random exploration as in Softmax or eGreedy - a wild speculation for now :)

Finally, absolute prediction error, which acts as a proxy of uncertainty, determined how long participants looked at the obtained outcomes.