The Ground Systems Development and Operations (GSDO) Smart Firing Room Project aims to create a firing room using cutting-edge technologies of today that are expected to be the state-of-the-art for the 2020s. One aspect of this project is providing a seamless Interactive Collaborative Environment (ICE) across a diverse array of user-facing devices — numerous screens of varying sizes, personal mobile devices, and natural user interface (NUI) sensors for multi-touch, gesture, and voice inputs. Applications accessible through the ICE are expected to provide Distributed User Interfaces (DUIs) that support collaborative features such as sharing applications with remote users, multi-user interaction for collaborative editing, and modular User Interfaces (UIs) to support customized workspaces spread across multiple devices. Using current technologies, developing an application with a DUI supporting such a wide variety of platforms is extremely costly due to the tight coupling between UIs, host platforms, and the application logic.

The glass wall prototype provides a desktop-like interface for launching applications on the large-format, touch-enabled display. These applications include the 3D Space Launch System Viewer.

The Glass Wall ICE consists of a large-format gesture and touchscreen interface, a tabletop touchscreen interface, a touchscreen console, and any number of smartphones, tablets, and laptops. Users will be able to collaborate via any combination of these devices to visualize the state of the Launch Complex, receive anomalous event alerts, access detailed sensor and video feeds, pull documentation and schematics for various systems, and conference with responsible personnel, both local and remote, to collaboratively troubleshoot issues. The integration of such a wide variety of user interface platforms and modalities with an equally diverse set of applications in an ICE poses significant challenges using current user interface development practices.

Numerous model-based UI development tools for DUIs and traditional UIs have been proposed, but generate platform-specific UI implementations that must be maintained and compiled with platform-specific application logic, a costly proposition considering the number of applications and potential platforms that must be supported. The same limitations plague other techniques traditionally used for reducing the complexity of user interface development, such as cross-platform user interface frameworks, and design patterns such as Model View Control. These challenges have driven the rise of Web applications that leverage the Web browser as the lowest common denominator for rendering application user interfaces across various platforms. Unfortunately, the Web browser was never intended for this purpose, and significant effort must be made to work around features imposed by standardization and backwards compatibility, such as statelessness and strict security policies.

This project introduces the concept of the Distributed User Interface Management System (DUIMS) to manage UI data structures and provide a query language for defining, observing, and manipulating these data structures. The DUIMS provides a general-purpose reusable solution for managing the complexity of DUIs for multi-application collaborative environments. The DUIMS manages the state of a logical UI rendered through reusable front-end clients for a variety of platforms and interface modalities. Applications utilize a DUIMS client library to define, manipulate, and react to changes in the state of the logical UI using a query language while users interact with DUIs for various applications spread across a number of devices running front-end clients. The DUIMS facilitates complete decoupling of the application logic from the rendered DUI, significantly reducing system complexity and costs. The goal of this project was the initial development of a DUIMS research prototype for studying various aspects relating to a DUIMS, such as query languages and data models for DUIs, performance issues, DUI development process, privacy and security, and novel use cases.

This work was done by Kyle Martin, a PhD student at the University of Central Florida, as part of an internship at Kennedy Space Center. KSC-13949

NASA Tech Briefs Magazine

This article first appeared in the February, 2016 issue of NASA Tech Briefs Magazine.

Read more articles from this issue here.

Read more articles from the archives here.