Computing in Superposition in the Classical Domain

Pentti Kanerva

Redwood Center for Theoretical Neuroscience, UC Berkeley
Friday, February 7, 2020 at 11:00pm
Evans 560

The standard model of computing treats variables very differently from their values. Variables are addresses to memory, and values are the contents of the addressed memory locations. This model of computing has proved to be extremely successful but also poor for modeling the brain’s computing. Neither does it look like what we see happening in brains. Rather, information is widely distributed over massive circuits, and a particular circuit element has little meaning by itself. That kind of computing is better modeled with high-dimensional vectors. Yet we want to hold on to the notions of a variable and a value, because they make it possible to represent and compute with structured information, such as language and its grammar. Now the variables and the values are vectors of a common mathematical space, and composing them in superposition yields vectors that lie in the same space and thus can in turn become elements in further composition. High dimensionality (e.g., N = 10,000), randomness, and the operations on the vectors are the key to this model of computing, first proposed by Tony Plate in his thesis on Holographic Reduced Representation (1994).