An assessment of a system’s safety requirements is required early in the design process when cost and time impacts are minimal. These requirements include fault detection, failure isolation, and failure recovery. Systems engineering groups at NASA have been exploring model-based products to better support verification of these safety requirements. One such product involves qualitative failure propagation modeling as a means to achieve early assessment of the system prior to the availability of higher fidelity information. These early development products lack the advanced reporting and analysis capabilities required to address the needs of system designers and domain experts. Therefore, post-processing tools need to be developed to support the interrogation of the model, and the structured reporting required by system engineers.

Current commercial off-the-shelf (COTS) development tools offer a model development interface, but have limited assessment and reporting capabilities. They intentionally do not enforce any modeling conventions or standard, in order to support a wide array of consumers. The Extended Testability Analysis (ETA) Tool addresses the insufficient reporting capability by utilizing a NASA-developed modeling convention to effectively communicate to system domain experts with little or no understanding of the internal diagnostic model, the results and conclusions from the associated diagnostic assessment (i.e. testability analysis). With a graphical user interface, the ETA Tool enables the user to perform advanced analyses and create various reports from a single testability analysis conducted by the COTS software. In addition, system engineering verification can require combined results from multiple testability analyses from the COTS program, which would be time-intensive and require tedious manual effort. These types of repeated processes can be automated through post-processing tools to ensure quick, systematic results. The ETA Tool V8.0 supports fault management (FM) by performing testability analyses on the fault propagation model of a given system. The ETA Tool processes a diagnostic model developed from a commercial software package called TEAMS Designer using a defined set of modeling conventions and practices. ETA Tool can provide an assessment of the modeled system based on diagnostic metrics of detectability and failure isolation. The program can provide assessment of a system’s detection coverage, test utilization, and failure isolation at the root cause or at a specified component level; generate a FMEA Report; perform sensor sensitivity studies; and provide a trace-mapping of failure modes to higher level system effects. ETA Tool generates reports from each selected analysis option in both HTML and XML format to enable user access in either standard Web browsers or loadable into a Microsoft Excel workbook for review/editing.

The ETA Tool is a command line process with several user-selectable report output options. The software package includes a graphical user interface that guides the user through the report and analysis options. The ETA Tool also extends the COTS testability analysis, enabling variation studies with sensor sensitivity impacts on system diagnostics, and assessing failure mode isolation at the component level employing various sets of component candidates using a single testability output. The testability results and the diagnostic model are accessed by ETA Tool so that further processing can provide detailed failure mode and system design information reports in a format that is relatively easy to read and understand. The software takes advantage of NASA-developed functional fault modeling standards to facilitate the reporting detail, and to provide specific analyses to be conducted.

The following analysis reports are available to the user: Detectability Report (provides a breakdown of how each tested failure mode was detected), Test Utilization Report (identifies all the failure modes that each test detects), Failure Mode Isolation Report (demonstrates the system’s ability to discriminate between failure modes), Component Isolation Report (demonstrates the system’s ability to discriminate between failure modes relative to the components containing the failure modes), Sensor Sensitivity Analysis Report (shows the diagnostic impact due to loss of sensor information), Effect Mapping Report (identifies failure modes that result in specified system-level effects), and Failure Mode and Effects Analysis (FMEA) Report (generates a snapshot of the system diagnostic information contained within the diagnostic model, and presents it in a historical FMEA reporting format).

This work was done by William Maul of Vantage Partners, LLC; Rachael Bis of N&R Engineering and Management; and Kevin Melcher of Glenn Research Center. This software is available for use. To request a copy, please visit https://software.nasa.gov/software/LEW-19241-1