Gary Marcus

Can you post a pointer? I'm not sure what you are referring to. There is quite a lot of work in ML on viewing reinforcement learning as inference.

I’m referring to Prof. Karl Friston’s work on Active Inference. Among many other findings, it dispenses with reward functions and describes action as a result of inference on learned sequences of observations. It’s a rather simple principle with enormous explanatory power.

Guess I still don’t the “enormous explanatory power” you promised. Can you give a concrete example?

I believe it explains, at various levels, the phenomenons of biological life, from evolutionary origins to observed behaviors to dysfunctions. It is based on the concept that our bodies and brains are merely imperfect models of our ecological niche, 1/n

comprised of *beliefs* about useful combinations of sensory observations (e.g. concepts or symbols), beliefs about their dynamics (valid paths between them) and beliefs about preferred observations, plus inference machinery that operates on these and can add to them. 2/n

The purpose of the machinery is to infer action that puts the agent on (otherwise improbable) paths towards preferred outcomes. The primary preferred, but quite improbable outcome is continued survival of the individual (or community/species). 3/n

This process of inference must be conducted on a number of timescales under energy constraints, and is therefore hierarchically organized. In advanced agents with many layers of hierarchy to aid long-term planning, high-level concepts like symbols emerge 4/n

as a result of language emerging first to allow coordination of individuals, plus energy constraints forcing simplification, thus generalization. 5/n