Mixed-initiative teams of robots and humans working side-by-side in close proximity to each other need to be capable of reasoning about the actions and activities of each other within a mission context. This capability requires the robotic agents to have a sense of self, and their roles and responsibilities.

PACIFIC (Process Algebra Capture of Intent From Information Content) is an activity reasoning system that takes image sequences as input and produces an interpretation of the events that have occurred. It uses a formal process algebra framework to build the interpretation as the sequence unfolds. This software provides a formal basis for the extraction of actions and activities of humans from an image sequence that is generated by fixed or moving cameras. It matches these extracted actions/activities to a mission profile in order to enable a mixed-initiative team of robots and humans to cooperate on a common task. This is the first software system to map image content to a formal mathematical process algebra basis that can be used for reasoning and matching. The software is efficient and utilizes an optimized set of inference mechanisms that scale linearly with image complexity.

This software would enable mixed human/robot teams on the surface of the Moon or Mars to perform tasks cooperatively without the need for excessive communication. The robots would be able to watch the humans and follow the mission flow in such a way that they are active members of the team. In the reimbursable realm, PACIFIC would be capable of long-term surveillance from aerial platforms that would lessen the load on the human operators for scene interpretation.

This software is applicable to any type of surveillance application where something more than a simple change detection approach is needed. The current fielded systems have limited reasoning capabilities of stringing sequences of actions together to derive activities for extraction of intent.

This work was done by Terrance L. Huntsberger of Caltech for NASA’s Jet Propulsion Laboratory. For more information, contact This email address is being protected from spambots. You need JavaScript enabled to view it..

This software is available for commercial licensing. Please contact Dan Broderick at This email address is being protected from spambots. You need JavaScript enabled to view it.. Refer to NPO-47935.