A control architecture and algorithms to implement the architecture have been conceived to enable a robot to learn from its experiences and to combine knowledge gained from prior experiences in such a way as to be able to solve new problems. The architecture is an abstraction of an interacting system of relatively simple components that, when properly interconnected, should enable the spontaneous emergence of behaviors from the complete system that would not necessarily be expected from the individual components. These emergent behaviors should enable a robot to interact robustly and intelligently with a complex, dynamic environment.

A control architecture and algorithms to implement the architecture have been conceived to enable a robot to learn from its experiences and to combine knowledge gained from prior experiences in such a way as to be able to solve new problems. The architecture is an abstraction of an interacting system of relatively simple components that, when properly interconnected, should enable the spontaneous emergence of behaviors from the complete system that would not necessarily be expected from the individual components. These emergent behaviors should enable a robot to interact robustly and intelligently with a complex, dynamic environment.

The architecture represents a set of parallel distributed computational modules (software objects) that communicate with each other through message passing. The principal objects loosely correspond to the (presumed) common computational modules of mammalian brains. A set of sensory processing modules continually updates a spatio-temporally indexed short-term memory structure denoted the sensory ego-sphere (SES). Depending on the task context, an attentional mechanism determines the saliency of incoming sensory information. In response to changes in the state of motion of the robot, the time series of sensory information is partitioned into episodes. Episodes are encoded in vector form and stored in a database, denoted the database associative memory (DBAM), that comprises the long-term memory of the robot.

The DBAM has the implicit mathematical structure of a linear vector space that is constructed as the direct sum of three subspaces. One subspace encodes a change in motor state (a motor event). Another subspace encodes the state of the sensors immediately prior to the motor event — in other words, the sensory preconditions of the motor event. The third subspace encodes the sensory state subsequent to the motor event (sensory post-conditions). The vectors in the DBAM and the external events from which they are formed are called sensory-motor coordination (SMC) events.

Several research projects have demonstrated that over time, as a robot senses and acts, clusters form within a vector space of SMC descriptors. Categories emerge. Thus, it becomes possible to select an exemplar that describes an equivalence class of SMC events. The encoding of SMC events as motor events with sensory preconditions and sensory post-conditions makes it possible to use simple vectorspace distances as measures of dissimilarity or similarity of complete SMC events. Vector-space distances can also serve as measures of dissimilarity or similarity between the sensory post-conditions of one event and the sensory preconditions of another.

The architecture utilizes a control strategy, called topological action mapping, that appears to be employed by mammals to achieve specific goals. In this architecture, action maps are implemented by a spreading activation network (SAN). (A SAN is one of a particular class of graph traversal algorithms that selects optimal paths.) For each goal to be reached by the robot, there corresponds an SMC event. Moreover, the current state of the robot is indicated by an SMC event. Given the goal event, the SAN searches through the DBAM, matching its sensory preconditions to the sensory post-conditions of other events. The search is constrained by a number of global state variables and previously formed linkages. Upon finding a sufficiently close match, the SAN iterates the search until the current state is reached. The current motor state (hence, the current behavior) is maintained until the sensory post-conditions trigger a change of state. Thus, the robot proceeds through a sequence of behaviors until the goal is reached.

One key aspect of this architecture is that the robot cannot be programmed through conventional means: it must be taught to perform tasks, either through direct control or by example. The robot must practice each task several times to enable the formation of clusters of SMC events and to learn the sequences of these events. Such repetition enables the robot to perform the task later, without supervision. However, this does not, in itself, enable creative problem solving.

To be able to solve problems creatively, the robot must engage in a process analogous to dreaming. After intervals of continuous motor activity, the robot performs computations in which it plays back data on recently performed sequences of tasks. SMC events are compared quasi-randomly to other events in the DBAM. If the sensory post-conditions of a given event are sufficiently similar to the sensory precondition of another event, links are formed between them. Such a link is formed, even if the two behaviors have never occurred in sequence before. This link gives the robot a new possible behavior transition that it could make, given the appropriate sensory trigger. Thus, the robot learns to anticipate possible sequences of events and learns strategies for the solving of problems that it has not encountered but that could occur.

The architecture is still undergoing development. An SES has been implemented in the NASA Robonaut (a developmental anthropomorphic robot intended to serve as an astronaut's assistant). Production-quality software to implement the architecture has not yet been written. A major problem in writing this software is that of efficient representation of sensory data in vector spaces.

This work was done by Alan Peters of Vanderbilt University for Johnson Space Center. In accordance with Public Law 96-517, the contractor has elected to retain title to this invention. Inquiries concerning rights for its commercial use should be addressed to:

Christopher D. McKinney, Director
Office of Technology Transfer
and Enterprise Development
Vanderbilt University
1207 17th Avenue South, Suite 105
Nashville, TN 37212
Phone: (615) 343-2430
E-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Refer to MSC-23489, volume and number of this NASA Tech Briefs issue, and the page number.