An important goal of vision science is to understand the coding strategies underlying the representation of visual information. I will describe experiments and analyses where we have explored these coding strategies using two different approaches. In the first approach, we factor-analyzed individual differences in observers’ color judgments to reveal the representational structure of color appearance. In contrast to

Forelimb morphology and use in mammals is extraordinarily diverse.  Evolution has produced wings, flippers, hooves, paws and hands which are specialized for a variety of behaviors such as flying, swimming and grasping to name a few. While there is a wealth of data in human and non-human primates on the role of motor cortex and posterior parietal cortical areas in reaching and grasping with the hand, these cortical networks

How do neural circuits in the brain accomplish rapid learning? When foraging for food in a previously unexplored environment, animals store memories of landmarks based on as few as one single view. Also, animals remember landmarks and navigation decisions that eventually lead to food, which requires that the brain associate events with delayed outcomes. I will present evidence that a particular neural circuit structure

Artificial intelligence has a trust problem. We are relying on A.I. more and more, but it hasn’t yet earned our confidence. Despite the intense recent hype surrounding AI, no current AI system remotely approaches the flexibility of human intelligence; as I will show, even the ability to read at grade-school level eludes current approaches. Building on my recent synthesis with Ernest Davis, I argue that

In rapid sensory decision-making, the time taken to choose and the accuracy of the choice are related in three distinct ways. First, it takes more time to assess noisy signals, so decisions about weak sensory stimuli are slower, as well as less accurate. Second, for any given stimulus strength, adopting an overall policy of higher stringency will make decisions slower, but more accurate. Third, even when stimulus strength and stringency

Dynamic programming (DP) is a general algorithmic approach used in optimal control and Markov decision processes that balances desire for low present costs with undesirability of high future costs when choosing a sequence of controls to apply over time. Recent interest in this field has grown since Google Deepmind’s algorithms

We propose a hierarchical EM algorithm for simplifying a finite mixture model into a reduced mixture model with fewer mixture components. The reduced model is obtained by maximizing a variational lower bound of the expected log-likelihood of a set of virtual samples. We develop three applications for our mixture

Motor control is usually seen as the result of a gradual replacement of feedback by feedforward control. The perceptual states that inform this process are considered to be defined through qualitatively different processes giving rise to the classical distinction

It is a truth not yet universally acknowledged that a self-regulating system which is stable in one environment can become unstable when the environment changes. This truth is called homeostatic fragility.  Mathematically, the key mechanism is sign-reversal, which converts a

Over the last few years, there has been considerable interest in Ising machines, ie, analog hardware for solving difficult (NP hard/complete) computational problems effectively. We present a new way to make Ising machines using networks of coupled self-sustaining

It is well-known that the expressivity of a neural network depends on its architecture, with deeper networks expressing more complex functions. For ReLU networks, which are piecewise linear, the number of distinct linear regions is a natural measure of expressivity. It is possible to

Learning a complex skill requires traversing a potentially enormous search space. While reinforcement learning (RL) algorithms can approach human levels of performance in complex tasks, they require much more training to do so. This may be because humans constrain search

Deep neural networks have proved to be extremely effective in image recognition, machine translation, and a variety of other data centered engineering tasks. However, some other domains, such as learning to model physical systems requires a more careful examination of

If biology is the study of self-replicating entities, and we want to understand the role of information, it makes sense to see how information theory is connected to the ‘replicator equation’ — a simple model of population dynamics for self-replicating entities. The relevant concept

I’ll present an approach from mathematical logic which shows how sub-symbolic dynamics may give rise to higher-level cognitive representations of structures, systems of knowledge, and algorithmic processes. This approach posits

Perception has long been considered to be an inference operation in which internal models of the sensory environment are constructed through experience and are subsequently used in order to assign sensory stimuli to

As the nervous system’s evolved output, spinal motor neuron activity is from an evolutionary perspective a natural source of signals for a neural interface. Furthermore, the amplification of these signals by muscle fibers allows

Computer vision has advanced rapidly with deep learning, achieving super-human performance on a few recognition benchmarks.  At the core of the state-of-the-art approaches for image classification, object detection, and

Activity of neuronal populations in several cortical regions can be described by a Dynamic Neural Field (DNF) equation. A DNF is a continuous in time and in space activation function defined over

Sparse reward and long horizon tasks are among the most interesting yet challenging problems to solve in reinforcement learning. I will discuss recent work leveraging representation learning

I will describe studies of the neural bases of cognitive processes. Rodents, mostly rats, are trained to perform behaviors that lend themselves to quantitative modeling that can help identify

My lab focuses on the neural mechanism of visual object recognition and developing techniques for neuronal circuit mapping. We have established long-term two-photon imaging in awake monkeys — the first and critical

Machine learning models are vulnerable to adversarial examples: small changes to images can cause computer vision models to make mistakes such as identifying a school bus as an ostrich. However, it is still an open question whether humans

(Joint work with Gil Kur and Boaz Nadler)

The notion of projection pursuit according to Huber (1985) appeared first in work of J.B. Kruskal (1969,1972) and was implemented and popularized by Friedman and Tukey (1974). Key papers are Huber

A primary goal of this work is to give a clear example of the limits of current, deep-learning techniques and suggest how progress can be made.  The presentation will include a discussion of open questions, unpublished experiments, suggestions on

How can we build autonomous robots that operate in unstructured and dynamic environments such as homes or hospitals? This problem has been investigated under several disciplines, including planning (motion planning, task planning, etc.), and reinforcement learning. While both of these

In many situations, two or more sources have some information about a target. For instance, sensory input and context information can jointly determine the firing pattern of a neuron. Since the information from the two sources is typically not identical, one

Social interaction is fundamental to our everyday life and that of diverse animals. When two animals interact, they behave in different ways. Thus, to get a full picture of the neural activity underlying each interaction, we need to record from the brains of both animals at the same time. We do so in

It is clear that animals do not build a 3D reconstruction of their environment in the way that computer vision does in SLAM systems. I will describe two experiments from our lab (mostly in VR but one reproduced in

In the past decade, research in machine learning has been exceedingly focused on the development of algorithms and models with remarkably high predictive capabilities. Models such as convolutional neural networks (CNNs) have achieved state-of-the-art predictive performance for

Recent results in information theory show how error-correcting codes on large, sparse graphs (“expander graphs”) can leverage multiple weak constraints to produce near-optimal performance. I will demonstrate a mapping between these error-correcting codes and canonical models of neural memory (“Hopfield networks”), and

Slow feature analysis (SFA) is a biologically motivated algorithm for extracting slowly varying features from a quickly varying signal and has proven to be a powerful general-purpose preprocessing method for spatio-temporal data in brain modeling as well as technical applications.  We have applied SFA to the learning of

The brain computes with spikes, and spikes are costly to metabolism. Yet, the spiking responses of most cortical cells appear extremely noisy, to the extent that the only feature that repeats from trial to trial is the firing rate (and only for the small minority of cells that are

Obtaining a better understanding of neural networks with random parameters is relevant for deep learning practice — for instance, by informing good initializations — and is a natural first step in building a more complete base of knowledge within deep learning. I will survey some of our recent work at Google Brain which

Are there any benefits in incorporating the foveated nature of human vision into image-based metrics of perception and computer vision systems? In this talk I hope to advance our understanding of this question through

Recent technological progress has dramatically increased our access to the neural activity underlying memory-related tasks. These complex high-dimensional data call for theories that allow us to identify signatures of collective activity in the

A common task facing computational scientists and, arguably, the brains of primates more generally is to construct models for data, particularly ones that invoke latent variables. Although it is often natural to identify the

Compositionality, generalization, and learning from a few examples are among the hallmarks of human intelligence. In this talk I will describe how Vicarious combines these ideas to create approaches to CAPTCHA breaking and Atari

I will present a review of the role of the primary visual cortex V1 in the functions of looking and seeing in vision. Looking is attentional selection, to select a fraction of…

Characterizing the computations performed by high-level sensory regions of the brain remains enigmatic due to the many nonlinear signal transformations that separate the input…

In this talk I will describe a theory that sensory regions of the neocortex process two inputs. One input is the well-known sensory data arriving via thalamic relay cells. The second…

Constrained maximization of information entropy yields least biased probability distributions. In statistical physics, this powerful inference method yields classical thermodynamics under…

In many real-world scenarios, rewards extrinsic to the agent are extremely sparse, or absent altogether. In such cases, curiosity can serve as an intrinsic reward signal to enable the agent…

In this talk, we derive the calculation of two critical numbers that quantify the capabilities of artificial neural networks with gating functions, such as sign, sigmoid, or rectified…

Determining whether certain properties are related to other properties is fundamental to scientific discovery. As data collection rates accelerate, it is becoming increasingly…

I will describe the Stabilized Supralinear Network mechanism and its application to understanding sensory cortical behavior. The mechanism is based on a network of excitatory…

The embodied mammalian brain evolved to adapt to an only partially known and knowable world. The adaptive labeling of the world is critically dependent on the neocortex which in turn…

I will discuss some implications of an approach that attempts to describe the various non-linearities of neurons in the visual pathway using a geometric framework. This approach…

Two potential bottlenecks on the expressiveness of recurrent neural networks (RNNs) are their ability to store information about the task in their parameters, and to…

Traditional computing is deterministic. It is based on bits and assumes that the bits compute perfectly. However, (near)perfection at high speeds consumes large amounts of…

We introduce a novel neural architecture for navigation in novel environments that learns a cognitive map from first person viewpoints and plans a sequence of actions towards goals in…

I will talk about how naive agent can learn the notion of space ex-nihilo. I am particularly interested in what hints space gives to the agent by constraining the agent’s sensorimotor…

We propose a self-supervised approach for learning representations entirely from unlabeled videos recorded from multiple viewpoints. This is particularly relevant to robotic imitation…

Technology scaling had been carrying computer science thru the second half of the 20th century until single CPU performance started leveling off, after which multi- and…

I will present 4 topics from the theory of brain computation: Memory/Encoding, Invariance, Behavioral Rhythms, and Language. Each topic incorporates a particular mathematical model…

The notion of interpreting cortical operations as probabilistic computation has been steadily gaining ground in neuroscience, and with the emergence of the PPC-based and…

Decoding the dynamics of brain activity underlying conscious behavior is one of the key questions in systems neuroscience. Sensory neurons, such as those in the auditory…

Deep learning is now state-of-the-art across a wide variety of machine learning domains including speech, video, and text. Nervana is a startup providing deep learning as a platform…

Let’s go for the real thing: Every waking second, our mind represents for us the situation we are immersed in. How do the physical states of our brain create this mental imagery?

Decision-making has been modeled in great detail based on 2-alternative choice (2AC) tasks; however it remains unclear how these models apply to more naturalistic settings, where…

Early visual processing has been studied extensively over the last decades. From these studies a relatively standard model emerged of the first steps in visual processing. However…

Language is the most important channel for humans to communicate about what they see. To allow an intelligent system to effectively communicate with humans it is thus important to…

The ability to forecast how different objects will be affected by the applied action (i.e. intuitive physics), is likely to be very useful for executing novel manipulation tasks…

Evolutionary pressure suggests that the spike-based code in the sensory nervous system should satisfy two opposing constraints: 1) minimize signal distortion in the encoding…

We explore the following crucial question: how could brains potentially perform the kind of powerful credit assignment that allows hidden layers of a very deep network to be trained…

Evolution can be thought of as a filter for evolvable developmental architectures. Comparing neuron numbers in the retina, midbrain, thalamus and cortex for a variety of…

In songbird vocalisation, the choice and sequencing of units (syllables) are widely studied, and amenable to standard machine learning methods. However, the fine detail in…

The Group equivariant Convolutional Neural Network (G-CNN) is a new kind of neural network that obtains better sample complexity by exploiting symmetries. G-CNNs use G-convolutions…

Conscious minds are sometimes revealed upon the permanent or even temporary loss of a functioning brain hemisphere. We know this because of two medical procedures which have existed…

Enabling computers to recognize abstract visual situations remains a hard open problems in artificial intelligence. No machine vision system comes close to matching human…

As transistors shrink to nanoscale dimensions, it is becoming increasingly difficult to make the current computing paradigm work. At two-dozen nanometers wide, a transistor’s…

We introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition. Samples from the model are of high perceptual…

One fundamental class of dimensionality reduction algorithms are nonlinear embedding (or manifold learning) algorithms. Their goal is to find meaningful low-dimensional coordinates…

We introduce a parametric nonlinear transformation for jointly Gaussianizing patches of natural images. The transformation is differentiable, can be efficiently inverted…

Anatomically, the brain is deep. To understand the ramifications of depth on learning in the brain requires a clear theory of deep learning. I develop the theory of gradient…

The escalating statistical variation of device behaviors in the nano-scale era has led to a search for alternative computational paradigms where randomness and statistics…

Adaptation to an oriented stimulus changes both the gain and preferred orientation of neural responses in V1. Neurons tuned near the adapted orientation are suppressed, and their…

Medical and governmental authorities around the world have identified a host of problematic behaviors with names like internet addiction, gaming addiction, and texting addiction…

The proliferation of transistors has increased the performance of computing systems by over a factor of a million in the past 30 years, and is also dramatically increasing the…

I will present two studies – one in the barrel cortex and one in the visual cortex – that detail surprising roles for inhibitory circuits in the neocortex. The first will address…

Curiosity is very predominant in people and many animals, but its mechanisms are poorly understood. Motivated by research on neuroscience and child development we want to…

Long thought to be the brains janitorial staff, sopping up excess transmitter and turning up the blood supply when the neurons complained, Astrocytes are now known to be doing something…

Policy search methods based on reinforcement learning and optimal control can allow robots to automatically learn a wide range of tasks. However, practical applications of policy…

The Locally Competitive Algorithm (LCA) is a neurally-plausible sparse solver based on lateral inhibition between leaky integrator neurons. LCA accounts for many linear and nonlinear…

Recent advances in deep generative models involved learning complex nonlinear transformations from a simple distribution of independent factors to a more complex distribution…

Cognitive Neuromorphic systems are gaining increasing importance in an era where CMOS digital computing techniques are meeting hard physical limits. These silicon systems mimic…

Recent advances in volume electron microscopy make it possible to image neuronal tissue volumes containining hundreds of thousands of neurons at sufficient resolution to discern…

The olfactory bulb receives rich glutamatergic projections from the piriform cortex. However, the dynamics and importance of these feedback signals remain unknown. In the…

It has been long proposed that the brain should perform computation efficiently to increase the fitness of the organism. However, the validity of this prominent..

Recent advances in associative memory design through structured pattern sets and graph-based inference algorithms have allowed reliable learning and recall of an…

We present a novel method for the classical task of finding and extracting recurring spatiotemporal patterns in recorded spiking activity of neuronal populations. In contrast…

Artificial Neural Networks (ANNs) form a powerful class of models with both theoretical and practical advantages. Networks with more than one hidden layer…

Sensory neuron responses in awake cortex can be quite variable across repeated presentations of the same stimulus. In many cortical areas, only a small fraction of…

Currently some of the best deep learning methods, e.g. the ones that are beating records on ImageNet, rely purely on supervised learning. Theoretically, it is clear that…

Functional changes in visual cortex as a consequence of blindness are a major model for studying crossmodal neuroplasticity. Previous work has shown that…

This informal seminar will give an overview of the BioRC project, and a discussion of why brain emulation is so difficult. Some breakthroughs in nanotechnology…

Dr. Vivienne Ming, named one of 10 Women to Watch in Tech by Inc. Magazine, is a theoretical neuroscientist, technologist and entrepreneur. She is the co-founder…

Natural images are scale invariant. After a quick tutorial on percolation, I will talk about formulating a geometric view of scale invariance based on percolation theory…

I will present a framework and a combined empirical-computational program that explores what cortical neural representation could underlie our intelligent behavior…

The efficient coding hypothesis broadly predicts that the tuning properties of neurons should reflect the statistics of the signal being encoded. For example, a sparse…

Grid cells in rat medial entorhinal cortex discharge when the animal moves through particular regions of the external world. For each grid cell, these regions form…

While sensory environments can vary dramatically in their statistics, neurons have a limited dynamic range with which they can encode sensory information. In sensory…

What enables proficient control of a brain-computer interface (BCI)? In this talk, I will argue that it is our ability to conceptualize a physical model of the…

The human visual system consists of many areas, each of which represents different features or aspects of the visual world. Primary visual cortex represents…

Pitch is one of the fundamental attributes of auditory perception. Melodies are created by sequential changes in pitch, and harmonies are formed by simultaneous…

Two well-established literatures have provided elegant models of sensorimotor adaptation and decision making, with relative little connection between the two. I will discuss…

The DSM5, reveals serious concerns about psychiatric diagnosis. The most serious concern is the failure of DSM to provide a valid ethiopathological psychiatric…

The birdsong system has become a paradigmatic example of biological learning. In this talk we will discuss how detailed biological responses can help this system to implement…

The “sensorimotor” theory of perceptual experience suggests that experience of the world necessarily involves understanding the relation between possible actions…

Captain Ahab, following the loss of his leg in a skirmish with the whale Moby Dick perceives phantom pain, and Ludwig von Beethoven, after losing his hearing, perceives…

Human beings have the unique ability to extract the meaning, or semantic content, from spoken language. Yet little is known about how the semantic content of everyday…

Clustering involves placing entities into mutually exclusive categories. We wish to relax the requirement of mutual exclusivity, allowing objects to belong simultaneously…

Top-down modulation of sensory processing is a prominent cognitive process that allows the animal to select sensory information most relevant to the current task, but…

The cortex processes information through distributed networks of functionally heterogeneous brain areas. This processing requires spiking activity to be transferred between…

While the term “Big Data” has only recently come into vogue, it can be argued that many of the practical advances in fields like computer vision have been driven…

Self-organized criticality is considered a common phenomenon in nature and became a fascinating research subject for neuroscience, when critical avalanches were predicted…

The human neocortex participates in a wide range of tasks, yet superficially appears to adhere to a relatively uniform six-layered architecture throughout its extent. For that…

I will talk about the active, unsupervised, curious, creative systems we have developed since 1990. They not only learn to solve externally posed tasks, but also…

Autonomy is a puzzling phenomenon in nature and a major challenge in the world of artifacts. A key feature of autonomy in both natural and artificial systems is seen in…

Two radically different forms of electrical activity can be observed in the rat hippocampus: spikes and local field potentials (LFPs). Hippocampal pyramidal neurons…

I’ll be talking about a joint effort between the Feldman, Carmena and Costa labs to study abstract task learning by small neuronal assemblies in intact networks. Brain-machine…

The two-dimensional Gabor function is adapted to natural image statistics by learning the joint distribution of the Gabor function parameters. This joint distribution…

Do mental states derive entirely from brain neuronal membrane activities? Neuronal interiors are organized by microtubules (‘MTs’), protein polymers proposed to…

I propose an experimental method to test various hypotheses on consciousness. Inspired by Sperry’s observation that split-brain patients possess two independent…

Visual perception has all the hallmarks of an ongoing, cooperative-competitive process: probabilistic outcome, self-organization, order-disorder transitions…

Data from neuroscience is fiendishly complex. Neurons exhibit correlations on very long timescales and across large populations, and the activity of individual neurons…

The retina utilizes a variety of signal processing mechanisms to compute direction from image motion. The computation is accomplished by a circuit that includes starburst…

The brain’s ability to seamlessly assimilate and process spatial and temporal information is critical to most behaviors, from understanding speech to playing the piano…

How does the human brain use neural activity to create and represent meanings of words, sentences and stories? One way to study this question is to give people text to…

Subtle alterations in how cortical network dynamics are modulated by different behavioral states could disrupt normal brain function and underlie symptoms of neuropsychiatric…

Structural inhomogeneities have a strong impact on population response dynamics of cortical networks and are believed to play an important role in their functioning. However…

The brain sitting inside its bony cavity sends and receives myriads of sensory inputs and outputs. A problem that must be solved either in ontogeny or phylogeny is how to extract…

Dictionaries and phrase tables are the basis of modern statistical machine translation systems. I will present a method that can automate the process of generating and extending dictionaries…

Our computing devices are outfitted with more sensors and dedicated sensor processors than ever before, spanning audio, image, motion, touch, climate, and more. Meanwhile, the technology industry…

Sparse Coding has been a very successful concept since many natural signals have the property of being sparse in some dictionary (basis). Some natural signals are even sparse in an orthogonal…

Short-term memory is a cognitive faculty central for the apprehension of music and speech. Only little is known, however, about memory for musical timbre despite its “sisterhood” with speech…

We study optimal image encoding based on a generative approach with non-linear feature combinations and explicit position encoding. By far most approaches to unsupervised learning of…

Cortical connections consist of feedforward, feedback and lateral pathways. Infragranular layers project down the cortical hierarchy to both supra- and infragranular layers…

In this talk I will briefly describe several techniques I explored in my thesis that improve how to efficiently model signal representations and learn useful information from…

Recently it has become possible to directly measure simultaneous collective states of many biological components, such as neural activities, genetic sequences, or gene expression…

Animal behaviour arises from computations in neuronal circuits, but our understanding of these computations has been frustrated by the lack of detailed synaptic connection…

Multiscale modeling/simulations attracts an increasing number of neuroscientists to study how different levels of organization (networks of neurons, cellular/subcellular levels) interact…

This talk is about computing with discrete compositional data structures in analog computers. A core issue for both computer science and cognitive neuroscience is the degree…

Sensory motor integration involves transforming a pattern of sensory input to a motor output and is a core neural operation carried out by all nervous systems. Analyses of…

The motor system successfully plans and executes sophisticated movements despite sensory feedback delays and effector dynamics that change over time. Behavioral studies suggest…

Since the beginning of the year, the European Union and United States have separately announced major initiatives in brain science. The latter is called the Brain Activity…

The auditory system exploits disparities in the sounds arriving at the left and right ear to extract information about the spatial configuration of sound sources. According to…

The perceptual organization of sounds in the environment into coherent objects is a feat constantly facing the auditory system. It manifests itself in the everyday challenge…

Information about the physical world is excluded from visual stimuli by the nature of biological vision (the inverse optics problem). Nonetheless, humans and other visual…

The inner ear constitutes a remarkable biological sensor that exhibits nanometer-scale sensitivity of mechanical detection. The first step in auditory processing is…

The talk will show the results of an empirical statistical analysis of images processed by complex Gabor-like filters. The analysis intends to be a compilation of statistical…

Stereo “3D” vision refers to the depth perception we have by virtue of viewing the world through two slightly offset eyes. This ability is receiving attention at the moment…

Autonomously learning about one’s own body and its interaction with the environment is a formidable challenge, yet it is ubiquitous in biology: every animal’s pup and…

Gradual accumulation of evidence is thought to be a fundamental component of decision-making. Over the last 16 years, research in non-human primates has revealed neural…

The paradox of Maxwell’s demon has stimulated numerous thought experiments, leading to discussions about the thermodynamic implications of information processing…

Cognitive decision processes are generally seen as classical Bayesian probabilities, but better suited to quantum mathematics. For example: 1) Psychological conflict…

We address the question of how to build a ‘strong’ probabilistic model of object shapes (binary silhouettes). We define a strong model as one which meets two requirements…

The traditional deep Boltzmann machine training algorithm requires a greedy layerwise pretraining phase. Existing techniques for avoiding greedy pretraining do not perform…

We present a dynamic nonlinear generative model for visual motion based on a latent representation of binary-gated Gaussian variables connected in a network. Trained on…

Efficient coding provides a powerful principle for explaining early sensory processing. Among the successful applications of this theory are models that provide functional…

People are remarkably good at acquiring complex knowledge from limited data, as is required in learning causal relationships, categories, or aspects of language…

General anesthesia and natural sleep share some commonalities and some differences. Quite a lot is known about the chemical and neuronal effects of general anesthetic…

Neural networks can integrate sensory information and generate continuously varying outputs, even though individual neurons communicate only with spikes—all-or-none…

The development of cerebral white matter involves both myelination and pruning of axons, and the balance between these two processes may differ between individuals. Cross-sectional measures of…

Probabilistic models of natural images have been used to solve a variety of computer vision tasks as well as a means to better understand the computations…

The famous Neural Binding Problem (NBP) comprises at least four distinct problems with different computational and neural requirements. This talk will review the…

The efficient coding hypothesis states that the front end properties of sensory systems, such as visual, can be understood from the statistics…

Computations in neural circuits are inherently flexible, allowing humans and animals to respond to sensory stimuli with actions that are appropriate in a given context. Fundamental to…

Learning good image priors is of utmost importance for the study of vision, computer vision and image processing applications. Learning priors…

Brain imaging is based on measuring not neural activity but rather, brain hemodynamics – local changes in blood volume, blood flow and oxygenation. These hemodynamic…

The concept of the receptive field in visual science has been transformative. It fueled great discoveries of the second half of the 20th C, providing the dominant understanding of how the visual system works…