Dynamic programming with sparse codes: investigating a new computational role for sparse representations of natural image sequences
Dynamic programming (DP) is a general algorithmic approach used in optimal control and Markov decision processes that balances desire for low present costs with undesirability of high future costs when choosing a sequence of controls to apply over time. Recent interest in this field has grown since Google Deepmind’s algorithms have beaten humans and world-champion programs in Atari games, and games such as chess, shogi, and go. But why do these algorithms work as well as they do? In many image-based tasks, early-layer weights of trained deep neural networks often resemble neural receptive field profiles found in the mammalian visual system. From modelling efforts in the neuroscience and signal processing communities we know this architecture generates efficient (low bit-rate) representations of natural images called sparse codes. In this work, I investigate the computational role of sparse codes by applying DP to solve the optimal control problem of tracking an object (dragonfly) over a sequence of natural images. By comparing speed of learning, memory capacity, interference, generalization, and fault tolerance for different codes, I will show some distinct computational advantages of sparse codes.
Of Rodents And Primates: Comparative Decision Making
In rapid sensory decision-making, the time taken to choose and the accuracy of the choice are related in three distinct ways. First, it takes more time to assess noisy signals, so decisions about weak sensory stimuli are slower, as well as less accurate. Second, for any given stimulus strength, adopting an overall policy of higher stringency will make decisions slower, but more accurate. Third, even when stimulus strength and stringency are the same, reaction time is extremely variable from trial to trial; the literature from humans an monkeys reports that reaction time is anti-correlated with accuracy: later responses are less accurate. The first two facts are easily explained by the Bounded Drift Diffusion Model. The third is not – but multiple competing models can account for it, such as collapsing decision bounds or urgency signals. In this talk, I will present data that rodents are the same as primates in the first two respects, but opposite in the third: for rodents, later responses are more accurate. I will show that at least one model can parsimoniously account for both primate and rodent behavior. Fitting both species in one model provides insight into what is conserved and what differs, when it comes to the decisions of mice and men.
Related background: Reinagel, P (2013) Speed and accuracy of visual motion discrimination by rats. PLoS-ONE 8(6):e68505 . https://doi.org/10.1371/journal.pone.0068505
From paws to hands: The evolution of the forelimb and cortical areas involved in complex hand use
Forelimb morphology and use in mammals is extraordinarily diverse. Evolution has produced wings, flippers, hooves, paws and hands which are specialized for a variety of behaviors such as flying, swimming and grasping to name a few. While there is a wealth of data in human and non-human primates on the role of motor cortex and posterior parietal cortical areas in reaching and grasping with the hand, these cortical networks did not arise de novoin primates, but likely arose early in mammalian evolution since most mammals use the forelimbs for reaching and grasping as well as other behaviors. Yet, we know relatively little about how frontoparietal networks that control the forelimb have evolved in mammals. Our laboratory has previously described the organization of somatosensory cortical areas in a variety of mammals and find that both morphology of the limb and how the limb is used are reflected in the organization of cortical fields that represent both mechanosensory receptors and proprioreceptors. In recent studies we examine the organization of movement maps using intracortical microstimulation techniques in a range of mammals to determine the extent of cortex from which movements can be evoked, and how behavioral specializations of the limb are represented in movement maps in the cortex. While there are some features of organization that are similar across species, such as gross topography, most of the details of map organization are species specific. Thus, movement maps are much more variable across species than are somatosensory maps, and are variable across individuals within a species, suggesting that these maps are, in large part, a product of experience. We propose that motor cortex co-evolved with modifications to the hand and forelimb, and is built during development based on commonly used muscle synergies.