Memory-prediction framework
This article needs additional citations for verification. (March 2011) |
The memory-prediction framework is a theory of brain function created by Jeff Hawkins and described in his 2004 book On Intelligence. This theory concerns the role of the mammalian neocortex and its associations with the hippocampi and the thalamus in matching sensory inputs to stored memory patterns and how this process leads to predictions of what will happen in the future.
Overview
The theory is motivated by the observed similarities between the brain structures (especially neocortical tissue) that are used for a wide range of behaviours available to mammals. The theory posits that the remarkably uniform physical arrangement of cortical tissue reflects a single principle or algorithm which underlies all cortical information processing. The basic processing principle is hypothesized to be a feedback/recall loop which involves both cortical and extra-cortical participation (the latter from the thalamus and the hippocampi in particular).
The basic theory: recognition and prediction in bi-directional hierarchies
The central concept of the memory-prediction framework is that bottom-up inputs are matched in a hierarchy of recognition, and evoke a series of top-down expectations encoded as potentiations. These expectations interact with the bottom-up signals to both analyse those inputs and generate predictions of subsequent expected inputs. Each hierarchy level remembers frequently observed temporal sequences of input patterns and generates labels or 'names' for these sequences. When an input sequence matches a memorized sequence at a given level of the hierarchy, a label or 'name' is propagated up the hierarchy – thus eliminating details at higher levels and enabling them to learn higher-order sequences. This process produces increased invariance at higher levels. Higher levels predict future input by matching partial sequences and projecting their expectations to the lower levels. However, when a mismatch between input and memorized/predicted sequences occurs, a more complete representation propagates upwards. This causes alternative 'interpretations' to be activated at higher levels, which in turn generates other predictions at lower levels.
Consider, for example, the process of
The sensory hierarchy induces a number of differences between the various levels. As one moves up the hierarchy,
- Extent – for example, larger areas of the visual field, or more extensive tactile regions.
- Temporal stability – lower-level entities change quickly, whereas, higher-level percepts tend to be more stable.
- Abstraction – through the process of successive extraction of invariant features, increasingly abstract entities are recognized.
The relationship between sensory and motor processing is an important aspect of the basic theory. It is proposed that the motor areas of the cortex consist of a behavioural hierarchy similar to the sensory hierarchy, with the lowest levels consisting of explicit motor commands to musculature and the highest levels corresponding to abstract prescriptions (e.g. 'resize the browser'). The sensory and motor hierarchies are tightly coupled, with behaviour giving rise to sensory expectations and sensory perceptions driving motor processes.
Finally, it is important to note that all the memories in the cortical hierarchy have to be learnt – this information is not pre-wired in the brain. Hence, the process of extracting this
Other terms
Hawkins has extensive training as an electrical engineer. Another way to describe the theory (hinted at in his book) is as a
That is, the evolutionary purpose of the brain is to predict the future, in admittedly limited ways, so as to change it.
Neurophysiological implementation
The hierarchies described above are theorized to occur primarily in mammalian neocortex. In particular, neocortex is assumed to consist of a large number of columns (as surmised also by Vernon Benjamin Mountcastle from anatomical and theoretical considerations). Each column is attuned to a particular feature at a given level in a hierarchy. It receives bottom-up inputs from lower levels, and top-down inputs from higher levels. (Other columns at the same level also feed into a given column, and serve mostly to inhibit the activation exclusive representations.) When an input is recognized – that is, acceptable agreement is obtained between the bottom-up and top-down sources – a column generates outputs which in turn propagate to both lower and higher levels.
Cortex
These processes map well to specific layers within mammalian cortex. (The cortical layers should not be confused with different levels of the processing hierarchy: all the layers in a single column participate as one element in a single hierarchical level). Bottom-up input arrives at layer 4 (L4), whence it propagates to L2 and L3 for recognition of the invariant content. Top-down activation arrives to L2 and L3 via L1 (the mostly axonal layer that distributes activation locally across columns). L2 and L3 compare bottom up and top-down information, and generate either the invariant 'names' when sufficient match is achieved, or the more variable signals that occur when this fails. These signals are propagated up the hierarchy (via L5) and also down the hierarchy (via L6 and L1).
Thalamus
To account for storage and recognition of sequences of patterns, a combination of two processes is suggested. The nonspecific
Hippocampus
Another anatomically diverse brain structure which is hypothesized to play an important role in hierarchical cognition is the hippocampus. It is well known that damage to both hippocampi impairs the formation of long-term declarative memory; individuals with such damage are unable to form new memories of episodic nature, although they can recall earlier memories without difficulties and can also learn new skills. In the current theory, the hippocampi are thought of as the top level of the cortical hierarchy; they are specialized to retain memories of events that propagate all the way to the top. As such events fit into predictable patterns, they become memorizable at lower levels in the hierarchy. (Such movement of memories down the hierarchy is, incidentally, a general prediction of the theory.) Thus, the hippocampi continually memorize 'unexpected' events (that is, those not predicted at lower levels); if they are damaged, the entire process of memorization through the hierarchy is compromised.
In 2016 Hawkins hypothesized that cortical columns did not just capture a sensation, but also the relative location of that sensation, in three dimensions rather than two (situated capture), in relation to what was around it.[1] "When the brain builds a model of the world, everything has a location relative to everything else" [1] —Jeff Hawkins.
Explanatory successes and predictions
The memory-prediction framework explains a number of psychologically salient aspects of cognition. For example, the ability of experts in any field to effortlessly analyze and remember complex problems within their field is a natural consequence of their formation of increasingly refined conceptual hierarchies. Also, the procession from 'perception' to 'understanding' is readily understandable as a result of the matching of top-down and bottom-up expectations. Mismatches, in contrast, generate the exquisite ability of biological cognition to detect unexpected perceptions and situations. (Deficiencies in this regard are a common characteristic of current approaches to artificial intelligence.)
Besides these subjectively satisfying explanations, the framework also makes a number of testable
Although these predictions are not highly specific to the proposed theory, they are sufficiently unambiguous to make verification or rejection of its central tenets possible. See On Intelligence for details on the predictions and findings.
Contribution and limitations
By design, the current theory builds on the work of numerous neurobiologists, and it may be argued that most of these ideas have already been proposed by researchers such as
The most significant limitation of this theory is its current[
IBM is implementing Hawkins' model.[2]
![]() | This section needs to be updated.(April 2023) |
Machine learning models
![]() | This section's use of external links may not follow Wikipedia's policies or guidelines. (May 2023) |
The memory-prediction theory claims a common algorithm is employed by all regions in the neocortex. The theory has given rise to a number of software models aiming to simulate this common algorithm using a hierarchical memory structure. The year in the list below indicates when the model was last updated.
Models based on Bayesian networks
The following models use belief propagation or belief revision in singly connected Bayesian networks.
- Hierarchical Temporal Memory(HTM), a model, a related development platform and source code by Numenta, Inc. (2008).
- HtmLib[dead link], an alternative implementation of HTM algorithms by Greg Kochaniak with a number of modifications for improving the recognition accuracy and speed (2008).
- Project Neocortex, an open source project for modeling memory-prediction framework (2008).
- Saulius Garalevicius' research page, research papers and programs presenting experimental results with a model of the memory-prediction framework, a basis for the Neocortex project (2007).
- George, Dileep (2005). "A Hierarchical Bayesian Model of Invariant Pattern Recognition in the Visual Cortex" (Document). IEEE. pp. 1812–1817. a paper describing earlier pre-HTM Bayesian model by the co-founder of Numenta. This is the first model of memory-prediction framework that uses Bayesian networks and all the above models are based on these initial ideas. Matlab source code of this model had been freely available for download for a number of years.
Other models
- Implementation of MPF, a paper by Saulius Garalevicius describing a method of classification and prediction in a model that stores temporal sequences and employs unsupervised learning (2005).
- M5, a pattern machine for Palm OS that stores pattern sequences and recalls the patterns relevant to its present environment (2007).
- BrainGame, open source predictor class which learns patterns and can be linked to other predictors (2005).
See also
- Adaptive resonance theory, a neural network architecture developed by Stephen Grossberg.
- Computational neuroscience
- Neural Darwinism
- Predictive coding
- Predictive learning
- Sparse distributed memory
References
- ^ a b Metz, Cade (October 15, 2018). "A new view of how we think". The New York Times. pp. B1, B4. See: 'Clarity Over a Coffee Cup'
- ^ Simonite, Tom (April 8, 2015). "IBM tests mobile computing pioneer's controversial Brain Algorithms". MIT Technology Review. Retrieved 2015-04-08.
Further reading
- Jeff Hawkins (2004), ISBN 0-8050-7456-2
External links
- Hierarchical vision algorithm source code & data
– similar to the Memory-Prediction Framework (from MIT Center for Biological & Computational Learning) - Group of articles about neuroscience and AI
– Group of articles and papers supporting Jeff's MPF theory. - MIT Technology Review Monday, February 12, 2007: Building the Cortex in Silicon