The complexity of military and aerospace systems is growing — more components, interfaces, power, bandwidth, processing, features, and data — and these systems are being networked to form even more complex "systems of systems." Modern networkc-entric systems can contain hundreds, even thousands of electronic modules.
The reliability, or Mean Time Between Failure (MTBF), of an electronic system is inversely related to the number of components in the system. Each component has a statistical failure rate, and the summation of all component failure rates determines the system failure rate. Large complex systems will typically have some, and possibly many, component failures over time. System engineers should embrace a new mindset where frequent component failures in large networked systems is the "normal" operating condition, rather than the "fault" condition.
Furthermore, a large system usually dissipates more heat and power. Module reliability is also inversely related to operating temperature. Simply put, modules operating at higher temperatures tend to fail more often. A rule of thumb is that the failure rate of an electronic module approximately doubles for every 10 to 20°C rise in temperature. There fore cooling is critical to system reliability. Other factors that contribute to system reliability include ESD protection, redundancy, fault localization, and fault isolation.
Availability is typically defined as the ratio MTBF/(MTBF + MTTR) where MTTR is the Mean Time to Repair. In simpler terms, availability is the percentage of time that a system is available for normal operation, or conversely the percentage of time that the system is not broken or under repair. For example, a system that is down for one day of the year is 99.7% available, assuming 24x7 operation. A system that is down for only one hour of the year is 99.98% available.
Until recently, most high-availability systems were "hand-rolled" with proprietary architectures. The Advanced Tele - com Computing Architecture (ATCA) and MicroTCA specifications from the PICMG organization, collectively known as xTCA, have changed this regime. The xTCA platforms provide a systematic approach to building standard, reliable, modular, network-centric systems with multi-vendor support. xTCA features include redundancy, cooling, fault isolation, ESD protection, rapid MTTR, and advanced system management.
The xTCA specifications were developed and fine-tuned to provide availability as high as 99.999%. This is equivalent to an average system downtime of less than 5.26 minutes each year, using standard commercial off-the-shelf enclosures and boards! For the first time in embedded system engineering, there is a costeffective, standard commercial approach to building high-availability systems.
The Network Rules
Not only does xTCA provide superb availability, it also provides an excellent open network-centric platform for modern systems. All xTCA blade slots have network connectivity with standard 1000BASE-TX dual-redundant Ethernet links to each card slot for a "base" dual-star network. In addition to the base network, the system engineer can select a platform with 1 and/or 10 Gbps high-speed "fabrics" network in dual-star and full mesh topologies. A single rackmount xTCA enclosure can support Terabits/sec of total data transport across the platform. In addition to Ethernet protocols, the xTCA platform can support protocols such as Infiniband, Fibre Channel, RapidIO, PCIExpress, and Serial ATA. Finally, front-panel ports on the blades are well suited to emerging link speeds such as 40 Gbps and 100 Gbps for state-of-the-art inter-platform networking.
Attention to Details
To meet 99.999% availability goals, the xTCA specifications provide close attention to details. For example, cards can be hot-swapped. Ground pins connect before power pins during blade insertion. ESD strips on the blade connect discharges directly to the chassis. Ejector handle switches power down units upon ejection. Rogue cards that indicate a fault (e.g., high power or temperature) are powered down and isolated from other units. All blade power supplies are electrically isolated from the backplane power bus, and protected against over-voltage, under-voltage, and over-current. Hold-up capacitors guard against transient bus power dips such as those that might be caused by a power fault on another module on the redundant power bus. Power distribution is a modern point-of-load-architecture.
Enclosure airflows are engineered to optimize xTCA module cooling and power dissipation. The platforms support up to 200-300 Watts per ATCA blade and up to 40-80 Watts per MicroTCA blade, permitting a single rack-mount 14-slot ATCA chassis to operate nearly 3000 Watts of electronic gear in a 55°C environment. Commercial xTCA platforms generally sport redundant fans and cooling units with temperature monitoring and fan speed control. For those engineers who shun fans as a rule, low-power ATCA systems have been fielded without fans, and conduction-cooled Micro TCA systems have been demonstrated.
The xTCA specifications also raise the bar in system configuration, monitoring, and management. All commercial modules are typically heavily instrumented, with temperature, power, timing, and network health monitors. Intelligent Platform Management Interface (IPMI) defines the management protocols and architecture, with an I2C interface to each blade for communication to the shelf managers and out-of-band channels to each platform for remote management of the systems, while still supporting common in-band management protocol standards such as SNMP. Configuration reports, electronic keying, multi-tenant support, and remote configuration updates (software, firm - ware) provide leading-edge embedded system management and control.
ATCA in the Military
More than 100 organizations currently support xTCA systems and products. Sales of xTCA systems and products are estimated to be more than $1B annually and have been projected to reach levels as high as $15B in the next five years. Although much of this rapid growth is attributed to commercial telecom, wireless, and computing systems, a number of military and aerospace programs are also using xTCA platforms for high-availability network-centric systems.
In 2005, the U.S. Navy began using ATCA in large, networked, undersea sensor systems for the purpose of tests and measurements on submarines. Since that time more than 35 ATCAbased systems have been built and delivered into programs for similar applications in harsh undersea environments. An example of a 14-slot ATCA undersea enclosure with conductively cooled power units is shown in Figure 2.
Boeing selected the ATCA platform for the P-8A Poseidon Multi-Mission Maritime Aircraft program. This new Navy aircraft provides long-range support for a wide variety of operations including anti-submarine warfare, anti-surface warfare, intelligence, surveillance, battle damage assessments, and reconnaissance. The electronics suite on the Poseidon must be configurable, open, scalable, reliable, and high performance to handle a wide variety of mission requirements including electrooptic sensors, infrared sensors, signal intelligence, magnetic sensors, acoustic sensors, radar, satellite comms, surface-to-air and air-to-air comms, countermeasures, and weapons control. The aircraft essentially operates as a complex, mobile airborne sensor network and data communications hub — a great application for the versatile network-centric xTCA platform. More than 100 P8-A aircraft are now planned for production. Future network-centric aerospace platforms such as AWACS, ACS, and BAMS would do well to consider xTCA for their electronic payloads.
Other military ATCA developments include high-speed adaptive radar and electronic warfare processing systems that are 1000 times faster than legacy systems; wideband data subsystems that can switch and process multi-gigabit/sec data flows at processing rates on the order of 10 million packets per second; and antenna array digitization and beam-forming systems that can process signals at compute rates up to 15 Teraops. ATCA is clearly enabling the next-generation of reliable high-performance embedded systems.
MicroTCA Establishes a Beachhead
The ATCA standard was released in 2003. MicroTCA, the small form factor relative of ATCA, was released later in 2006, but it is quickly catching on in military applications. The smaller size, lower power, and lower cost of MicroTCA make it well suited for applications where ATCA is a bit too much, while retaining the five-nines availability, superb system management, and high-performance networking of ATCA.
In October 2007, at the industry military communications conference MILCOM, vendors showcased a ruggedized Micro TCA chassis that has reportedly been selected by BAE Systems for the WIN-T JC4ISR network radio, a key component of the Army's future wireless combat system network. Information on these impressive ruggedized MicroTCA radios was also presented recently at the ATCA Summit in Santa Clara, CA.
Two commercial manufacturers have announced the development and testing of Air Transport Racks (ATRs) for MicroTCA blades. One of the vendors has announced a shock-mounted Micro - TCA enclosure with military-standard shock, vibration, thermal, EMI, and EMC qualifications. The other manufacturer, Schroff, has announced a prototype ATR enclosure for conduction cooled, ruggedized, MicroTCA. These rugged enclosures will help enable new network and computing applications in rough environments.
The success of early military ATCA systems, the emerging availability of commercial rugged xTCA products, the sophisticated network-centric xTCA architecture, and the significant cost and performance advantages of the xTCA platform should help military and aerospace system engineers develop better, more reliable systems for the future.