Practically everything computers do is better, faster, and more power-efficient than the brain. For example, a calculator performs numerical computations more energy-efficiently than any human. Yet modern AI models are a thousand times less efficient than the brain. These models have increasingly larger and larger dimension to maximize their representational capacity, requiring GPUs to

Standard deep learning models generalize via interpolation, relying on an implicit bias toward smooth functions to fit training data. While effective in-distribution, this approach often fails out-of-distribution (OOD) because it fails to capture the underlying generative structure of the data. To achieve robust extrapolation, a model cannot merely approximate the training

The Fitzhugh-Nagumo model with diffusive coupling is known to admit a variational (energy-based) formulation, which is a result of its underlying skew-gradient structure. We show that since stationary solutions of the diffusive Fitzhugh-Nagumo model are described by self-adjoint operators, the methods of equilibrium propagation for performing credit assignment can be applied. Further,

Optic flow refers to the motion pattern formed on the retina as an observer moves through the environment. Although optic flow conveys rigid 3D structure information associated with self-motion, prior studies have often conflated it with non-rigid “unreal” motion patterns that lacks coherent 3D structure, leading to ambiguity in interpretations of functional specificity across motion-sensitive visual brain regions. Here, we combined

We do not know how the brain assigns credit to synapses during learning, but we do know that it almost certainly does not use backpropagation. Existing evidence instead points to low-dimensional, indirect error signals and to a diversity of synaptic plasticity rules that are often incompatible with gradient descent. In this talk, I will show that taking these constraints seriously yields new insights into how biological

Although contemporary theories of consciousness offer rich conceptual insights, the field remains fragmented, largely because few proposals provide a mechanistic testable account of how neural dynamics give rise to conscious experience.  The talk will present a unifying perspective grounded in critical brain dynamics, the hypothesis that

This talk will start with a presentation of two distinct theoretical approaches to characterize synaptic plasticity. The first approach consists in building a minimal biophysical calcium-based model to describe plasticity at glutamatergic synapses onto pyramidal cells. This model can reproduce successfully a large amount of experimental data in various preparations, and

How can we move beyond hand-crafted assumptions in building intelligent systems? Rigid priors hold back embodied AI: forcing perception to handle only predefined transformations, treating locomotion and navigation as isolated silos, and reducing perception to a single, context-free snapshot. I’ll present recent work on data-driven, context-adaptive approaches that move beyond these constraints, pointing toward

The appearance at the anatomical level of a canonical laminar microcircuit suggests that each six-layer column of granular cortex may mediate a canonical computation, but it remains unknown what that computation is and which aspects of neuronal activity mediate it. Predictive processing theorists have suggested that

I will introduce generalized empowerment maximization—an information-theoretic framework for modeling intrinsic motivation. I’ll show how to compute empowerment under practical approximations. I will then show how agents driven by this objective can solve standard control problems with no external rewards. Extending this to multi-agent settings,

What does it mean to perceive something? What does it mean when you understand? A long-standing hypothesis about these phenomena stems from the brain’s ability to perform “analysis-by-synthesis”. Rather than just computing a complex function from inputs to outputs, the analysis-by-synthesis approach suggests that the brain incorporates a generative model of the

How can we answer impossible questions in vision like “What would vision look like if evolution started over?” or “What if evolution had taken different paths?”.  Traditional approaches are unable to systematically answer these questions: we can’t re-run large-scale evolution experimentally, evolutionary

Grid cells in the medial entorhinal cortex construct an intriguing multiperiodic representation of space whose properties have been the subject of much theoretical speculation. Here we combine modeling and analyses of entorhinal cortex population recordings to show that the grid cell representation is ideally set up to decorrelate and assign easily distinguishable

What does a cortical column do? What do the different laminae of the cortex compute? What is the role of feedback connections in the cortex? What computations happen in the thalamus? — all these are important questions in our pursuit to understand human-like intelligence. In this talk I will present a detailed theory of canonical cortical circuits for vision

The ability to record responses of single neurons in awake humans allows us to understand the neurocomputational foundations of language. We recorded responses of neural populations in hippocampus and anterior cingulate cortex during speech listening and during interactive conversations. We find that single neurons in both regions use a dense code with

While all fundamental physical interactions obey the law of action-equals-reaction, the dynamics we effectively observe in complex nonequilibrium systems ubiquitously break reciprocity at different scales. In this talk, I will discuss the intricate connection between nonreciprocity and time-reversal symmetry breaking, and how irreversibility

A substantial body of work over the last decade plus has converged on a tractable computational model for how the brain represents the time of past events in the ongoing firing of neurons. Populations of neurons in a variety of brain regions code for time, but with two distinct forms of receptive fields. So-called temporal context cells, observed in a variety of brain regions, respond to

On-device learning has emerged as a prevailing trend that enables intelligence on edge devices for IoT applications. However, there remain multiple gaps before large-scale deployments in the real world, including dynamic and changeable environments, limited hardware resources, heterogeneous sensor modalities, etc. Hyperdimensional Computing (HDC) is an

We now have powerful computer vision algorithms that can segment scenes, label objects, and recognize actions. It is tempting to use these algorithms as models of visual processing in biological brains. I will provide an overview of some of the successes in using neural network models to partially describe visual behavior

I present the Central-peripheral Dichotomy (CPD) theory (Zhaoping 2017, 2019) together with visual psychophysics tests of its predictions. CPD states that central vision is specialized for seeing (recognizing), and peripheral vision for looking (shifting gaze/attention). Given an information

Building upon the efficient coding and predictive information theories, we present a novel perspective that neurons not only predict but may also actively influence their future inputs through their outputs. We model neurons as feedback controllers of their environments, a role traditionally considered computationally demanding, particularly

Changes in an animal’s behavioral state, such as arousal and movements, induce complex modulations of the baseline input currents to sensory areas, eliciting sensory modality-specific effects. A simple computational principle explaining the effects of baseline modulations to recurrent cortical circuits is lacking. We investigate the benefits of baseline modulations, referred to as biases in machine learning, using a reservoir computing

Every day we coordinate eye, head, and body movements seamlessly to go about our daily activities. Similarly, flying insects coordinate eye and body movements to orient in space. A challenge in studying vision is that moving eyes are coupled to a moving body. Another challenge is that movement is inherently closed-loop: e.g., information flows

Hyperdimensional Computing (HDC) is an emerging computational paradigm for representing compositional information as high-dimensional vectors, and has a promising potential in applications ranging from machine learning to neuromorphic computing. One of the longstanding challenges in HDC is the decomposition of compositional information to its

In this talk, we contend that a natural objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a low-dimensional Gaussian mixture supported on incoherent subspaces. The goodness of such a representation can be evaluated by a principled measure, called sparse rate reduction, that

“What is consciousness?” is a highly general question, a much more quantifiable sub-question is “what are the common mathematical properties of learning-capable systems?”. In this talk we will examine a number of architectures implementing learning systems, the features determining

Reinforcement learning (RL) provides a general suite of algorithms for the approximate solution of many important and interesting optimal control tasks and sequential decision-making problems. At the heart of these problems is a complex tradeoff between exploration of new states

An important difference between brains and deep neural networks is the way they learn. Nervous systems learn online where a stream of noisy data points are presented in a non-independent, identically distributed (non-i.i.d.) way. Further, synaptic plasticity in the brain depends only on information local to synapses. Deep networks, on the other hand,

As the demand for big data increases and the speed of traditional CPUs cannot keep pace, new computing paradigms and architectures are needed to meet the demands for our data hungry world. To keep pace with this, Ising Computing and probabilistic computing have emerged as a method to solve NP-Hard

Visual attention selects only a tiny fraction of visual input information for further processing and recognition. This makes human blind to most of visual inputs. Attentional selection starts at the primary visual cortex (V1), which creates a bottom-up saliency map to guide

Virtually all cortical systems feature “functional organization”: the arrangement of neurons with specific functional properties into characteristic spatial arrangements. Functional organization is ubiquitous across systems and species, highly reproducible, and central to our understanding of

All organisms make temporal predictions, and their evolutionary fitness level generally scales with the accuracy of these predictions. In the context of visual perception, observer motion and continuous deformations of objects and textures structure the dynamics of visual signals, which allows for partial prediction of future inputs from past ones. Here, we propose

Hyperdimensional computing (HDC) is a biologically-inspired framework which represents symbols with high-dimensional vectors, and uses vector operations to manipulate them. The ensemble of a particular vector space and a prescribed set of vector operations (including one addition-like for “bundling” and one outer-product-like for “binding”) form a vector symbolic architecture (VSA). While VSAs have been

MARS is a popular method for nonparametric regression introduced by Friedman in 1991. MARS fits simple nonlinear and non-additive functions to regression data. We propose and study a natural LASSO variant of the MARS method. Our method is based on least squares estimation over a convex class of functions

Many networks in the brain are sparsely connected, and the brain eliminates connections during development and learning. How could the brain decide which synapses to prune? In a recurrent network, determining the importance of a connection between two neurons is a difficult computational problem, depending on

Across thermodynamics, systems exchange quantities that are conserved globally, such as energy, particles, and electric charges. These quantities are represented by Hermitian operators, which have traditionally been assumed to commute with each other. However, noncommutation is a key feature of quantum theory. A major challenge in studying how charges’ noncommutation influences thermodynamic phenomena is isolating

Subjective ratings given by observers are a critical part of research in image and video quality assessment. In this talk, I first review different approaches to subjective ratings and identify potential pitfalls that are often overlooked. Using existing and newly collected data, I statistically demonstrate the non-linear use of the scale, changes in ratings throughout the experiment, individual observer differences in rating features, and biases from allowing observers to decide how many ratings

The cell assembly hypothesis postulates that information processing in the brain entails the repetitive co-activation of groups of neurons. The activation of such assemblies would lead to spatio-temporal spike patterns (STPs) at the resolution of a few milliseconds. In order to test the cell assembly hypothesis, we searched for significant STPs in parallel spike trains, using the SPADE method. We analyzed experimental data from the motor cortex (M1/PMd) of macaque monkeys

High-dimensional brain activity is in many cases organized into lower-dimensional neural manifolds. Feedback from V4 to V1 is known to mediate visual attention and computational work has shown that it can also rotate neural manifolds in a context-dependent manner. However, whether feedback signals can modulate neural manifolds in vivo remains to be ascertained. Here, we studied the neural manifolds in macaque (Macaca mulatta) visual cortex during resting state

Vector Symbolic Architectures (VSAs) combine a high-dimensional vector space with a set of carefully designed operators in order to perform symbolic computations with large numerical vectors. Major goals are the exploitation of their representational power and ability to deal with fuzziness and ambiguity. The basis

In the Drosophila brain, head direction neurons form a network whose activity tracks the angular position of the fly using both self-movement and visual inputs. Our recent work seeks to understand the circuit and synaptic mechanisms that allow these internal and external signals to be seamlessly combined into a coherent sense of direction. Using in vivo electrophysiology and calcium imaging in combination with virtual reality we show that

Place fields in the hippocampus show a variety of remapping phenomena in response to environmental changes. These remapping phenomena get characterized in terms of different types of coding of spatial information — object vector cells, landmark vector cells, distance coding, etc. But what if these phenomena are side effects of

(Slides) What is a reasonable architecture for an algorithmic view of the mind? Is it akin to a single giant deep network or is it more like several small modules connected by some graph? How is memory captured — is it some lookup table? Take a simple event like meeting someone over coffee — how would your mind remember who the person was,

Mounting evidence suggests that during conscious states, the electrodynamics of the cortex are poised near a critical point or phase transition and that this near-critical behavior supports the vast flow of information through cortical networks during conscious states. Here, we empirically identify a mathematically specific critical point near which waking cortical oscillatory dynamics operate, which is known as the edge-of-chaos critical point, or the boundary between stability and chaos. We do so by applying

Decision trees, one of the oldest statistical and machine learning models, are widely used as interpretable models and as the basis of random and boosted forests. However, their true power remains underexploited.

Recent work suggests that feature constraints in the training datasets of deep neural networks (DNNs) drive robustness to adversarial noise (Ilyas et al., 2019). The representations learned by such adversarially robust networks have also been shown to be

The visual system must constantly generate meaning by combining noisy sensory information with efficient internal representations. However, as essential as these hidden representations are in shaping many aspects of cognition and behavior, they can be difficult to measure directly. In my work, I apply adaptive sampling techniques

This work proposes a new computational framework for learning an explicit generative model for real-world datasets. More specifically, we propose to learn a closed-loop transcription between a multi-class multi-dimensional data distribution and a linear discriminative representation (LDR) in the feature space that consists of multiple

What is the most important unsolved problem in Artificial Intelligence? It may well be the problem of creating a unified model of the universe, to which all observations of the system can be related to. This requires representations that are universal and dynamic, combine tacit and propositional knowledge, and

Analog compute-in-memory (CIM) architectures for low-power neural networks have recently been shown to achieve excellent compute efficiency and high accuracy, comparable to software-based deep neural networks. However, two primary limitations prevent them from reaching their potential: 1) resistive crossbars have difficulty scaling to large, sparse networks; and

The neocortex exhibits a detailed architecture that is largely preserved across different functional areas and between species. This has led to the idea of a common cortical algorithm that underlies all aspects of perception, language, and thought. Whether such a common algorithm exists and what it could be has been debated for decades. Our team has proposed an answer

Why is visual information transmitted through many parallel channels in the optic nerve, with each channel encoding a different feature-map of the visual scene? Why do neurons in the retina prefer disk-shaped light dots and in the brain oriented lines? In this talk we will see how these simple questions can be investigated using artificial neural networks and a bit of maths.

Perception and cognition are teeming with signals and concepts that interact in a multiplicative way, and it is largely this pattern of combination that generates the awesome variability and complexity of our physical and mental worlds. Human brains are prodigious in contending with such complexity, in part due to their ability to factor concepts into more

Most of the core computational concepts in visual neuroscience come from studies using anesthetized or fixating subjects. Although this approach is designed to maximize experimental control – by stabilizing the subject’s gaze – the net result is that the most commonly used visual stimulus is a fixation point and little is known about how active visual behavior shapes the encoding process

Building artificial agents that can mimic human learning and reasoning has been a longstanding objective in artificial intelligence. I will discuss some of the empirical data and computational models from neuroscience and cognitive science that could help us advance towards this goal. Specifically, I will talk about the importance of structured representations of knowledge, particularly about mental or cognitive maps for time, space, and

Connectionism is a movement in Psychology that began in the 1980ies to understand cognition using neural network models operating on distributed representations. Connectionism not only greatly advanced traditional neural networks, i.e. working out error back propagation, it also identified their limitations. Importantly, traditional neural networks lack a full set of operations required for symbolic reasoning, in particular,

Self-organization is frequently observed in active collectives, from ant rafts to molecular motor assemblies. General principles describing self-organization away from equilibrium have been challenging to identify. We offer a unifying framework that models the behavior of complex systems as largely random, while capturing their driven response properties. Such a “low-rattling principle” enables prediction and control of fine-tuned emergent properties in

When engaging in natural tasks, the human visual system processes a highly dynamic visual data stream. The retina, performing the very first steps in this processing, is thought to be adapted to take advantage of low-level signal regularities, such as the autocorrelation function or power spectrum, to produce a more efficient encoding of the data (Atick & Redlich, 1992). Previous work examined the joint spatio-temporal power spectrum of handheld camera videos and Hollywood movies, showing that power falls as an inverse power-law function of spatial and temporal frequency, with an inseparable relationship (Dong & Attick, 1995). However…

(Note: this was a MCB/neuro seminar, not a Redwood Seminar).  Human design is constrained by natural selection to maximize performance for a given energy cost. The brain predicts what will be needed and controls metabolism, physiology, and behavior to deliver just enough, just in time. By preventing errors (allostasis), rather than correcting them (homeostasis), energy is saved. Predictive control is a core function that requires rapid computations by the whole brain to guide its tiny effector (the hypothalamus).

A central goal of vision science is to understand the principles underlying the perception and cortical encoding of the complex visual environment of our everyday experience. In the visual cortex, foundational work with artificial stimuli, and more recent work combining natural images and deep convolutional neural networks, have revealed much about the tuning of cortical neurons to specific image features. However, a major limitation of this existing work is

The standard model of computing treats variables very differently from their values. Variables are addresses to memory, and values are the contents of the addressed memory locations. This model of computing has proved to be extremely successful but also poor for modeling the brain’s computing. Neither does it look like what we see happening in brains. Rather, information is widely distributed over massive circuits, and

Building models to efficiently represent images is a central problem in the machine learning community. The brain and especially the visual cortex, has long find economical and robust solutions to solve such a problem. At the local scale, Sparse Coding is one of the most successful framework to model neural computation in the visual cortex. It directly derives from the efficient coding hypothesis, and could be thought as a competitive mechanism that

Image understanding is often conceived as a hierarchical process with many levels where complexity and invariance of object selectivity gradually increase with level in the hierarchy. In contrast, neurophysiological studies have shown that figure-ground organization and border ownership coding, which imply understanding of the object structure of an image, occur at levels as low as V1 and V2 of the

In this talk, I will first present a signal representation framework called the Sparse Manifold Transform that combines key ideas from sparse coding, manifold learning, and slow feature analysis. It turns non-linear transformations in the primary sensory signal space into linear interpolations in a representational embedding space while maintaining approximate invertibility. The sparse manifold transform is an

An important goal of vision science is to understand the coding strategies underlying the representation of visual information. I will describe experiments and analyses where we have explored these coding strategies using two different approaches. In the first approach, we factor-analyzed individual differences in observers’ color judgments to reveal the representational structure of color appearance. In contrast to

Forelimb morphology and use in mammals is extraordinarily diverse.  Evolution has produced wings, flippers, hooves, paws and hands which are specialized for a variety of behaviors such as flying, swimming and grasping to name a few. While there is a wealth of data in human and non-human primates on the role of motor cortex and posterior parietal cortical areas in reaching and grasping with the hand, these cortical networks

How do neural circuits in the brain accomplish rapid learning? When foraging for food in a previously unexplored environment, animals store memories of landmarks based on as few as one single view. Also, animals remember landmarks and navigation decisions that eventually lead to food, which requires that the brain associate events with delayed outcomes. I will present evidence that a particular neural circuit structure

Artificial intelligence has a trust problem. We are relying on A.I. more and more, but it hasn’t yet earned our confidence. Despite the intense recent hype surrounding AI, no current AI system remotely approaches the flexibility of human intelligence; as I will show, even the ability to read at grade-school level eludes current approaches. Building on my recent synthesis with Ernest Davis, I argue that

In rapid sensory decision-making, the time taken to choose and the accuracy of the choice are related in three distinct ways. First, it takes more time to assess noisy signals, so decisions about weak sensory stimuli are slower, as well as less accurate. Second, for any given stimulus strength, adopting an overall policy of higher stringency will make decisions slower, but more accurate. Third, even when stimulus strength and stringency

Dynamic programming (DP) is a general algorithmic approach used in optimal control and Markov decision processes that balances desire for low present costs with undesirability of high future costs when choosing a sequence of controls to apply over time. Recent interest in this field has grown since Google Deepmind’s algorithms

We propose a hierarchical EM algorithm for simplifying a finite mixture model into a reduced mixture model with fewer mixture components. The reduced model is obtained by maximizing a variational lower bound of the expected log-likelihood of a set of virtual samples. We develop three applications for our mixture

Motor control is usually seen as the result of a gradual replacement of feedback by feedforward control. The perceptual states that inform this process are considered to be defined through qualitatively different processes giving rise to the classical distinction

It is a truth not yet universally acknowledged that a self-regulating system which is stable in one environment can become unstable when the environment changes. This truth is called homeostatic fragility.  Mathematically, the key mechanism is sign-reversal, which converts a

Over the last few years, there has been considerable interest in Ising machines, ie, analog hardware for solving difficult (NP hard/complete) computational problems effectively. We present a new way to make Ising machines using networks of coupled self-sustaining

It is well-known that the expressivity of a neural network depends on its architecture, with deeper networks expressing more complex functions. For ReLU networks, which are piecewise linear, the number of distinct linear regions is a natural measure of expressivity. It is possible to

Learning a complex skill requires traversing a potentially enormous search space. While reinforcement learning (RL) algorithms can approach human levels of performance in complex tasks, they require much more training to do so. This may be because humans constrain search

Deep neural networks have proved to be extremely effective in image recognition, machine translation, and a variety of other data centered engineering tasks. However, some other domains, such as learning to model physical systems requires a more careful examination of

If biology is the study of self-replicating entities, and we want to understand the role of information, it makes sense to see how information theory is connected to the ‘replicator equation’ — a simple model of population dynamics for self-replicating entities. The relevant concept

I’ll present an approach from mathematical logic which shows how sub-symbolic dynamics may give rise to higher-level cognitive representations of structures, systems of knowledge, and algorithmic processes. This approach posits

Perception has long been considered to be an inference operation in which internal models of the sensory environment are constructed through experience and are subsequently used in order to assign sensory stimuli to

As the nervous system’s evolved output, spinal motor neuron activity is from an evolutionary perspective a natural source of signals for a neural interface. Furthermore, the amplification of these signals by muscle fibers allows

Computer vision has advanced rapidly with deep learning, achieving super-human performance on a few recognition benchmarks.  At the core of the state-of-the-art approaches for image classification, object detection, and

Activity of neuronal populations in several cortical regions can be described by a Dynamic Neural Field (DNF) equation. A DNF is a continuous in time and in space activation function defined over

Sparse reward and long horizon tasks are among the most interesting yet challenging problems to solve in reinforcement learning. I will discuss recent work leveraging representation learning

I will describe studies of the neural bases of cognitive processes. Rodents, mostly rats, are trained to perform behaviors that lend themselves to quantitative modeling that can help identify

My lab focuses on the neural mechanism of visual object recognition and developing techniques for neuronal circuit mapping. We have established long-term two-photon imaging in awake monkeys — the first and critical

Machine learning models are vulnerable to adversarial examples: small changes to images can cause computer vision models to make mistakes such as identifying a school bus as an ostrich. However, it is still an open question whether humans

(Joint work with Gil Kur and Boaz Nadler)

The notion of projection pursuit according to Huber (1985) appeared first in work of J.B. Kruskal (1969,1972) and was implemented and popularized by Friedman and Tukey (1974). Key papers are Huber

A primary goal of this work is to give a clear example of the limits of current, deep-learning techniques and suggest how progress can be made.  The presentation will include a discussion of open questions, unpublished experiments, suggestions on

How can we build autonomous robots that operate in unstructured and dynamic environments such as homes or hospitals? This problem has been investigated under several disciplines, including planning (motion planning, task planning, etc.), and reinforcement learning. While both of these

In many situations, two or more sources have some information about a target. For instance, sensory input and context information can jointly determine the firing pattern of a neuron. Since the information from the two sources is typically not identical, one

Social interaction is fundamental to our everyday life and that of diverse animals. When two animals interact, they behave in different ways. Thus, to get a full picture of the neural activity underlying each interaction, we need to record from the brains of both animals at the same time. We do so in

It is clear that animals do not build a 3D reconstruction of their environment in the way that computer vision does in SLAM systems. I will describe two experiments from our lab (mostly in VR but one reproduced in

In the past decade, research in machine learning has been exceedingly focused on the development of algorithms and models with remarkably high predictive capabilities. Models such as convolutional neural networks (CNNs) have achieved state-of-the-art predictive performance for

Recent results in information theory show how error-correcting codes on large, sparse graphs (“expander graphs”) can leverage multiple weak constraints to produce near-optimal performance. I will demonstrate a mapping between these error-correcting codes and canonical models of neural memory (“Hopfield networks”), and

Slow feature analysis (SFA) is a biologically motivated algorithm for extracting slowly varying features from a quickly varying signal and has proven to be a powerful general-purpose preprocessing method for spatio-temporal data in brain modeling as well as technical applications.  We have applied SFA to the learning of

The brain computes with spikes, and spikes are costly to metabolism. Yet, the spiking responses of most cortical cells appear extremely noisy, to the extent that the only feature that repeats from trial to trial is the firing rate (and only for the small minority of cells that are

Obtaining a better understanding of neural networks with random parameters is relevant for deep learning practice — for instance, by informing good initializations — and is a natural first step in building a more complete base of knowledge within deep learning. I will survey some of our recent work at Google Brain which

Are there any benefits in incorporating the foveated nature of human vision into image-based metrics of perception and computer vision systems? In this talk I hope to advance our understanding of this question through

Recent technological progress has dramatically increased our access to the neural activity underlying memory-related tasks. These complex high-dimensional data call for theories that allow us to identify signatures of collective activity in the

A common task facing computational scientists and, arguably, the brains of primates more generally is to construct models for data, particularly ones that invoke latent variables. Although it is often natural to identify the

Compositionality, generalization, and learning from a few examples are among the hallmarks of human intelligence. In this talk I will describe how Vicarious combines these ideas to create approaches to CAPTCHA breaking and Atari

I will present a review of the role of the primary visual cortex V1 in the functions of looking and seeing in vision. Looking is attentional selection, to select a fraction of…

Characterizing the computations performed by high-level sensory regions of the brain remains enigmatic due to the many nonlinear signal transformations that separate the input…

In this talk I will describe a theory that sensory regions of the neocortex process two inputs. One input is the well-known sensory data arriving via thalamic relay cells. The second…

Constrained maximization of information entropy yields least biased probability distributions. In statistical physics, this powerful inference method yields classical thermodynamics under…

In many real-world scenarios, rewards extrinsic to the agent are extremely sparse, or absent altogether. In such cases, curiosity can serve as an intrinsic reward signal to enable the agent…

In this talk, we derive the calculation of two critical numbers that quantify the capabilities of artificial neural networks with gating functions, such as sign, sigmoid, or rectified…

Determining whether certain properties are related to other properties is fundamental to scientific discovery. As data collection rates accelerate, it is becoming increasingly…

I will describe the Stabilized Supralinear Network mechanism and its application to understanding sensory cortical behavior. The mechanism is based on a network of excitatory…

The embodied mammalian brain evolved to adapt to an only partially known and knowable world. The adaptive labeling of the world is critically dependent on the neocortex which in turn…

I will discuss some implications of an approach that attempts to describe the various non-linearities of neurons in the visual pathway using a geometric framework. This approach…

Two potential bottlenecks on the expressiveness of recurrent neural networks (RNNs) are their ability to store information about the task in their parameters, and to…

Traditional computing is deterministic. It is based on bits and assumes that the bits compute perfectly. However, (near)perfection at high speeds consumes large amounts of…

We introduce a novel neural architecture for navigation in novel environments that learns a cognitive map from first person viewpoints and plans a sequence of actions towards goals in…

I will talk about how naive agent can learn the notion of space ex-nihilo. I am particularly interested in what hints space gives to the agent by constraining the agent’s sensorimotor…

We propose a self-supervised approach for learning representations entirely from unlabeled videos recorded from multiple viewpoints. This is particularly relevant to robotic imitation…

Technology scaling had been carrying computer science thru the second half of the 20th century until single CPU performance started leveling off, after which multi- and…

I will present 4 topics from the theory of brain computation: Memory/Encoding, Invariance, Behavioral Rhythms, and Language. Each topic incorporates a particular mathematical model…

The notion of interpreting cortical operations as probabilistic computation has been steadily gaining ground in neuroscience, and with the emergence of the PPC-based and…

Decoding the dynamics of brain activity underlying conscious behavior is one of the key questions in systems neuroscience. Sensory neurons, such as those in the auditory…

Deep learning is now state-of-the-art across a wide variety of machine learning domains including speech, video, and text. Nervana is a startup providing deep learning as a platform…

Let’s go for the real thing: Every waking second, our mind represents for us the situation we are immersed in. How do the physical states of our brain create this mental imagery?

Decision-making has been modeled in great detail based on 2-alternative choice (2AC) tasks; however it remains unclear how these models apply to more naturalistic settings, where…

Early visual processing has been studied extensively over the last decades. From these studies a relatively standard model emerged of the first steps in visual processing. However…

Language is the most important channel for humans to communicate about what they see. To allow an intelligent system to effectively communicate with humans it is thus important to…

The ability to forecast how different objects will be affected by the applied action (i.e. intuitive physics), is likely to be very useful for executing novel manipulation tasks…

Evolutionary pressure suggests that the spike-based code in the sensory nervous system should satisfy two opposing constraints: 1) minimize signal distortion in the encoding…

We explore the following crucial question: how could brains potentially perform the kind of powerful credit assignment that allows hidden layers of a very deep network to be trained…

Evolution can be thought of as a filter for evolvable developmental architectures. Comparing neuron numbers in the retina, midbrain, thalamus and cortex for a variety of…

In songbird vocalisation, the choice and sequencing of units (syllables) are widely studied, and amenable to standard machine learning methods. However, the fine detail in…

The Group equivariant Convolutional Neural Network (G-CNN) is a new kind of neural network that obtains better sample complexity by exploiting symmetries. G-CNNs use G-convolutions…

Conscious minds are sometimes revealed upon the permanent or even temporary loss of a functioning brain hemisphere. We know this because of two medical procedures which have existed…

Enabling computers to recognize abstract visual situations remains a hard open problems in artificial intelligence. No machine vision system comes close to matching human…

As transistors shrink to nanoscale dimensions, it is becoming increasingly difficult to make the current computing paradigm work. At two-dozen nanometers wide, a transistor’s…

We introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition. Samples from the model are of high perceptual…

One fundamental class of dimensionality reduction algorithms are nonlinear embedding (or manifold learning) algorithms. Their goal is to find meaningful low-dimensional coordinates…

We introduce a parametric nonlinear transformation for jointly Gaussianizing patches of natural images. The transformation is differentiable, can be efficiently inverted…

Anatomically, the brain is deep. To understand the ramifications of depth on learning in the brain requires a clear theory of deep learning. I develop the theory of gradient…

The escalating statistical variation of device behaviors in the nano-scale era has led to a search for alternative computational paradigms where randomness and statistics…

Adaptation to an oriented stimulus changes both the gain and preferred orientation of neural responses in V1. Neurons tuned near the adapted orientation are suppressed, and their…

Medical and governmental authorities around the world have identified a host of problematic behaviors with names like internet addiction, gaming addiction, and texting addiction…

The proliferation of transistors has increased the performance of computing systems by over a factor of a million in the past 30 years, and is also dramatically increasing the…

I will present two studies – one in the barrel cortex and one in the visual cortex – that detail surprising roles for inhibitory circuits in the neocortex. The first will address…

Curiosity is very predominant in people and many animals, but its mechanisms are poorly understood. Motivated by research on neuroscience and child development we want to…

Long thought to be the brains janitorial staff, sopping up excess transmitter and turning up the blood supply when the neurons complained, Astrocytes are now known to be doing something…

Policy search methods based on reinforcement learning and optimal control can allow robots to automatically learn a wide range of tasks. However, practical applications of policy…

The Locally Competitive Algorithm (LCA) is a neurally-plausible sparse solver based on lateral inhibition between leaky integrator neurons. LCA accounts for many linear and nonlinear…

Recent advances in deep generative models involved learning complex nonlinear transformations from a simple distribution of independent factors to a more complex distribution…

Cognitive Neuromorphic systems are gaining increasing importance in an era where CMOS digital computing techniques are meeting hard physical limits. These silicon systems mimic…

Recent advances in volume electron microscopy make it possible to image neuronal tissue volumes containining hundreds of thousands of neurons at sufficient resolution to discern…

The olfactory bulb receives rich glutamatergic projections from the piriform cortex. However, the dynamics and importance of these feedback signals remain unknown. In the…

It has been long proposed that the brain should perform computation efficiently to increase the fitness of the organism. However, the validity of this prominent..

Recent advances in associative memory design through structured pattern sets and graph-based inference algorithms have allowed reliable learning and recall of an…

We present a novel method for the classical task of finding and extracting recurring spatiotemporal patterns in recorded spiking activity of neuronal populations. In contrast…

Artificial Neural Networks (ANNs) form a powerful class of models with both theoretical and practical advantages. Networks with more than one hidden layer…

Sensory neuron responses in awake cortex can be quite variable across repeated presentations of the same stimulus. In many cortical areas, only a small fraction of…

Currently some of the best deep learning methods, e.g. the ones that are beating records on ImageNet, rely purely on supervised learning. Theoretically, it is clear that…

Functional changes in visual cortex as a consequence of blindness are a major model for studying crossmodal neuroplasticity. Previous work has shown that…

This informal seminar will give an overview of the BioRC project, and a discussion of why brain emulation is so difficult. Some breakthroughs in nanotechnology…

Dr. Vivienne Ming, named one of 10 Women to Watch in Tech by Inc. Magazine, is a theoretical neuroscientist, technologist and entrepreneur. She is the co-founder…

Natural images are scale invariant. After a quick tutorial on percolation, I will talk about formulating a geometric view of scale invariance based on percolation theory…

I will present a framework and a combined empirical-computational program that explores what cortical neural representation could underlie our intelligent behavior…

The efficient coding hypothesis broadly predicts that the tuning properties of neurons should reflect the statistics of the signal being encoded. For example, a sparse…

Grid cells in rat medial entorhinal cortex discharge when the animal moves through particular regions of the external world. For each grid cell, these regions form…

While sensory environments can vary dramatically in their statistics, neurons have a limited dynamic range with which they can encode sensory information. In sensory…

What enables proficient control of a brain-computer interface (BCI)? In this talk, I will argue that it is our ability to conceptualize a physical model of the…

The human visual system consists of many areas, each of which represents different features or aspects of the visual world. Primary visual cortex represents…

Pitch is one of the fundamental attributes of auditory perception. Melodies are created by sequential changes in pitch, and harmonies are formed by simultaneous…

Two well-established literatures have provided elegant models of sensorimotor adaptation and decision making, with relative little connection between the two. I will discuss…

The DSM5, reveals serious concerns about psychiatric diagnosis. The most serious concern is the failure of DSM to provide a valid ethiopathological psychiatric…

The birdsong system has become a paradigmatic example of biological learning. In this talk we will discuss how detailed biological responses can help this system to implement…

The “sensorimotor” theory of perceptual experience suggests that experience of the world necessarily involves understanding the relation between possible actions…

Captain Ahab, following the loss of his leg in a skirmish with the whale Moby Dick perceives phantom pain, and Ludwig von Beethoven, after losing his hearing, perceives…

Human beings have the unique ability to extract the meaning, or semantic content, from spoken language. Yet little is known about how the semantic content of everyday…

Clustering involves placing entities into mutually exclusive categories. We wish to relax the requirement of mutual exclusivity, allowing objects to belong simultaneously…

Top-down modulation of sensory processing is a prominent cognitive process that allows the animal to select sensory information most relevant to the current task, but…

The cortex processes information through distributed networks of functionally heterogeneous brain areas. This processing requires spiking activity to be transferred between…

While the term “Big Data” has only recently come into vogue, it can be argued that many of the practical advances in fields like computer vision have been driven…

Self-organized criticality is considered a common phenomenon in nature and became a fascinating research subject for neuroscience, when critical avalanches were predicted…

The human neocortex participates in a wide range of tasks, yet superficially appears to adhere to a relatively uniform six-layered architecture throughout its extent. For that…

I will talk about the active, unsupervised, curious, creative systems we have developed since 1990. They not only learn to solve externally posed tasks, but also…

Autonomy is a puzzling phenomenon in nature and a major challenge in the world of artifacts. A key feature of autonomy in both natural and artificial systems is seen in…

Two radically different forms of electrical activity can be observed in the rat hippocampus: spikes and local field potentials (LFPs). Hippocampal pyramidal neurons…

I’ll be talking about a joint effort between the Feldman, Carmena and Costa labs to study abstract task learning by small neuronal assemblies in intact networks. Brain-machine…

The two-dimensional Gabor function is adapted to natural image statistics by learning the joint distribution of the Gabor function parameters. This joint distribution…

Do mental states derive entirely from brain neuronal membrane activities? Neuronal interiors are organized by microtubules (‘MTs’), protein polymers proposed to…

I propose an experimental method to test various hypotheses on consciousness. Inspired by Sperry’s observation that split-brain patients possess two independent…

Visual perception has all the hallmarks of an ongoing, cooperative-competitive process: probabilistic outcome, self-organization, order-disorder transitions…

Data from neuroscience is fiendishly complex. Neurons exhibit correlations on very long timescales and across large populations, and the activity of individual neurons…

The retina utilizes a variety of signal processing mechanisms to compute direction from image motion. The computation is accomplished by a circuit that includes starburst…

The brain’s ability to seamlessly assimilate and process spatial and temporal information is critical to most behaviors, from understanding speech to playing the piano…

How does the human brain use neural activity to create and represent meanings of words, sentences and stories? One way to study this question is to give people text to…

Subtle alterations in how cortical network dynamics are modulated by different behavioral states could disrupt normal brain function and underlie symptoms of neuropsychiatric…

Structural inhomogeneities have a strong impact on population response dynamics of cortical networks and are believed to play an important role in their functioning. However…

The brain sitting inside its bony cavity sends and receives myriads of sensory inputs and outputs. A problem that must be solved either in ontogeny or phylogeny is how to extract…

Dictionaries and phrase tables are the basis of modern statistical machine translation systems. I will present a method that can automate the process of generating and extending dictionaries…

Our computing devices are outfitted with more sensors and dedicated sensor processors than ever before, spanning audio, image, motion, touch, climate, and more. Meanwhile, the technology industry…

Sparse Coding has been a very successful concept since many natural signals have the property of being sparse in some dictionary (basis). Some natural signals are even sparse in an orthogonal…

Short-term memory is a cognitive faculty central for the apprehension of music and speech. Only little is known, however, about memory for musical timbre despite its “sisterhood” with speech…

We study optimal image encoding based on a generative approach with non-linear feature combinations and explicit position encoding. By far most approaches to unsupervised learning of…

Cortical connections consist of feedforward, feedback and lateral pathways. Infragranular layers project down the cortical hierarchy to both supra- and infragranular layers…

In this talk I will briefly describe several techniques I explored in my thesis that improve how to efficiently model signal representations and learn useful information from…

Recently it has become possible to directly measure simultaneous collective states of many biological components, such as neural activities, genetic sequences, or gene expression…

Animal behaviour arises from computations in neuronal circuits, but our understanding of these computations has been frustrated by the lack of detailed synaptic connection…

Multiscale modeling/simulations attracts an increasing number of neuroscientists to study how different levels of organization (networks of neurons, cellular/subcellular levels) interact…

This talk is about computing with discrete compositional data structures in analog computers. A core issue for both computer science and cognitive neuroscience is the degree…

Sensory motor integration involves transforming a pattern of sensory input to a motor output and is a core neural operation carried out by all nervous systems. Analyses of…

The motor system successfully plans and executes sophisticated movements despite sensory feedback delays and effector dynamics that change over time. Behavioral studies suggest…

Since the beginning of the year, the European Union and United States have separately announced major initiatives in brain science. The latter is called the Brain Activity…

The auditory system exploits disparities in the sounds arriving at the left and right ear to extract information about the spatial configuration of sound sources. According to…

The perceptual organization of sounds in the environment into coherent objects is a feat constantly facing the auditory system. It manifests itself in the everyday challenge…

Information about the physical world is excluded from visual stimuli by the nature of biological vision (the inverse optics problem). Nonetheless, humans and other visual…

The inner ear constitutes a remarkable biological sensor that exhibits nanometer-scale sensitivity of mechanical detection. The first step in auditory processing is…

The talk will show the results of an empirical statistical analysis of images processed by complex Gabor-like filters. The analysis intends to be a compilation of statistical…

Stereo “3D” vision refers to the depth perception we have by virtue of viewing the world through two slightly offset eyes. This ability is receiving attention at the moment…

Autonomously learning about one’s own body and its interaction with the environment is a formidable challenge, yet it is ubiquitous in biology: every animal’s pup and…

Gradual accumulation of evidence is thought to be a fundamental component of decision-making. Over the last 16 years, research in non-human primates has revealed neural…

The paradox of Maxwell’s demon has stimulated numerous thought experiments, leading to discussions about the thermodynamic implications of information processing…

Cognitive decision processes are generally seen as classical Bayesian probabilities, but better suited to quantum mathematics. For example: 1) Psychological conflict…

We address the question of how to build a ‘strong’ probabilistic model of object shapes (binary silhouettes). We define a strong model as one which meets two requirements…

The traditional deep Boltzmann machine training algorithm requires a greedy layerwise pretraining phase. Existing techniques for avoiding greedy pretraining do not perform…

We present a dynamic nonlinear generative model for visual motion based on a latent representation of binary-gated Gaussian variables connected in a network. Trained on…

Efficient coding provides a powerful principle for explaining early sensory processing. Among the successful applications of this theory are models that provide functional…

People are remarkably good at acquiring complex knowledge from limited data, as is required in learning causal relationships, categories, or aspects of language…

General anesthesia and natural sleep share some commonalities and some differences. Quite a lot is known about the chemical and neuronal effects of general anesthetic…

Neural networks can integrate sensory information and generate continuously varying outputs, even though individual neurons communicate only with spikes—all-or-none…

The development of cerebral white matter involves both myelination and pruning of axons, and the balance between these two processes may differ between individuals. Cross-sectional measures of…

Probabilistic models of natural images have been used to solve a variety of computer vision tasks as well as a means to better understand the computations…

The famous Neural Binding Problem (NBP) comprises at least four distinct problems with different computational and neural requirements. This talk will review the…

The efficient coding hypothesis states that the front end properties of sensory systems, such as visual, can be understood from the statistics…

Computations in neural circuits are inherently flexible, allowing humans and animals to respond to sensory stimuli with actions that are appropriate in a given context. Fundamental to…

Learning good image priors is of utmost importance for the study of vision, computer vision and image processing applications. Learning priors…

Brain imaging is based on measuring not neural activity but rather, brain hemodynamics – local changes in blood volume, blood flow and oxygenation. These hemodynamic…

The concept of the receptive field in visual science has been transformative. It fueled great discoveries of the second half of the 20th C, providing the dominant understanding of how the visual system works…

How is it that the eye can have an exquisite sense of motion even while the retinal image of the stable world during fixation is in constant motion? Several hypotheses have arisen: The “efference-copy” hypothesis holds that efferent signals derived

Quantitative modeling of human brain activity can provide crucial insights about cortical representations and can form the basis for brain decoding devices. Recent functional magnetic resonance imaging (fMRI) studies have…

Data sharing between labs, and indeed between disciplines, reduces duplication of effort, facilitates new discoveries, and leads to the development of more flexible, reliable, and reproducible analysis techniques. Initially, a data…

Understanding the physical processes that underly the functioning of biological computing machinery often requires describing processes that occur far from thermodynamic equilibrium. In recent years…

Any physical device we know, including computers, when comparing A to B must send the information to point C. I have done experiments in three modalities, somato-sensory, auditory, and visual, where…

Even simple experiences activate large numbers of neurons in the central nervous system. It is not at all clear how many neurons are needed to generate a sensory percept or how activity among these neurons leads…

The design of complete imaging systems using a joint design framework have led to significant achievements in terms of reduced system size and cost or enriched imaging features. At Ricoh Innovations we are designing…

One of the central tenets of systems neuroscience is that tuning curves are a byproduct of the interactions between neurons. Using multi-electrode recordings and recently developed inference techniques we can begin…