Unifying perceptual and behavioral learning through the synergistic coupling of feedback and feedforward control through counterfactual errors

Paul F.M.J. Verschure

Barcelona Institute of Science and Technology
Wednesday, June 5, 2019 at 12:00pm
560 Evans

Motor control is usually seen as the result of a gradual replacement of feedback by feedforward control. The perceptual states that inform this process are considered to be defined through qualitatively different processes giving rise to the classical distinction between perceptual and behavioral learning. We have addressed this question from the perspective of an integrated architecture called the Distributed Adaptive Control (DAC) theory of mind and brain. DAC proposes that the brain is a multi-layer control system which optimizes the how of action by considering why (motivation), what (objects), where (space), when (time) and who (agents) or the H5W problem. We have shown that for DAC to realize optimal solutions in foraging problems, its decision-making renders policies that simultaneously optimize perceptual evidence, memory bias, goals, and utility. This raises the question of what the principles are that underlie the processing and adaptation of these factors. In this presentation, I will focus on a link between policy adaptation and perceptual learning we have recently advanced. The dominant model of anticipatory motor control relies on the notion of an inverse model that by learning from encountered errors acquires corrective responses that supersede feedback control. However, these models are predicated on a Markovian world assumption and thus by necessity face problems in handling exceptions, such as observed in probe trials, where fast feedback control is required. We solve this challenge by proposing that adaptive motor control can also be obtained by relying on a cascade of purely sensory predictions that drive feedback control via counterfactual errors or Hierarchical Sensory Predictive Control. Using robot experiments, we have demonstrated the robustness of this solution. We have found further supporting evidence for the relevance of counterfactual error in motor learning in the rehabilitation of stroke patients.  I will show how this unified model of perceptual and behavioral learning captures relevant properties of the anatomy and physiology of the cerebellum and its dense interactions with the forebrain.