AdvancedTCA (ATCA) continues to evolve to meet not just the market demands for the telecom central office, but networking, data center, medical, and military communications applications. The two main goals that ATCA suppliers have been trying to solve often conflict with each other: offering more performance in less space, while meeting the specific market and compliance requirements for each industry.
In the early days of ATCA, the architecture had a viable ecosystem for central office applications using vertical 12U-13U shelves in 14 slots. For the central office, the shelves required full redundancy and FRU (field replaceable units), 48V DC power feeds, and 200W per slot front-torear cooling performance. Thus many 12U sizes were later abandoned as cooling demands increased.
Not long after, horizontal mount chassis entered the market in 4U-6U heights for telecom and networking applications with backplanes in 5 or 6 slots. AC power options were often a key requirement for these applications (and therefore units like the 4U, where space is limited for AC power, shelf managers, etc., also largely faded away). But, these early solutions had side-to-side cooling and were limited for telecom applications that require NEBS-compliance, and virtually ignored other applications such as military and ruggedized applications. Further, as chip advances continue, providers need to find practical solutions to cooling, signal integrity, and EMI. Plus, cooling demands have changed the focus from shelves per rack to overall performance density.
There have been two main pushes for performance density. One of these efforts is to increase the signal speeds. The move to four lanes of 3.125 Gbps into a single lane (port) for 10GBASEKR and KX4 and bundling four 10 Gbps lanes into one 40 Gbps channel (40GBase-KR4) has been addressed by the IEEE 802.3b/ba subcommittees. At 40 Gbps speeds, there is a significant performance gain across the shelf. Additionally, simply going from Dual Star to Full Mesh designs can add significant bandwidth. Before we go to 100G efforts, there is certainly opportunity to incorporate Full Mesh across 40G for even more performance gains.
The second approach is to offer more performance using less chassis space by increasing the computing density at the blade level. Carrier grade solutions have gone from 12U to 13U and up to 15U for the additional cooling required for 350W+ per slot of today’s ATCA boards. Plus, the RTM section can add another 40-70W per slot of heat to dissipate. But, with 40G options, the bandwidth/performance density is improved off the bat. With the push to 40G, conducted and radiated EMI immunity needed to be changed, posing further challenges to the increased cooling requirements. Many 5U shelves have moved to 6U to increase cooling, but also offer the fully redundant AC power modules, shelf managers, etc. AC power is a key requirement for the flexibility of use in a wide range of applications outside of the central office. It is more conducive to medical, data center, networking, and some mil/aero applications (where a combined AC/DC solution is attractive).
Aside from the signal speeds, there is a way to increase ATCA computing density by 50% in a horizontal 6-slot chassis. By integrating the switch fabric functionality into redundant shelf managers, two slots that are usually dedicated switch fabric slots in the ATCA shelf are saved. The two extra slots are utilized as standard payload (node) slots, which greatly increases the performance density. Thus, instead of having only 4 payload slots in the 6U shelf, there are a full 6 slots available.
The ATCA backplane has to be specifically configured to properly route the switch/shelf signals and be integrated mechanically into the enclosure. The shelf switch slot can be a 1GBe base and 10/40GBe fabric layer two or layer three managed switch on the same module. For many data centers and other applications, front-to-rear cooling is not a requirement. Thus, the cooling configuration can be side-to-side. The 5U-6U size is very attractive to reduce providers’ overhead costs. A vertical ATCA chassis with 14 slots is often not fully subscribed, wasting valuable rack space and increasing costs. The mid-range size utilizes all of the slots, bringing down the overhead costs. If a 6U chassis now offers 6 payload slots, then two units equals the computing density on one 14-slot vertical chassis (two Dual Star configuration slots for switches and 12 payload slots). Thus, including the benefit of less overhead, you have 12 payload slots in two 6U units (12U) versus 12 payload slots in one 13U-15U high Dual Star horizontal chassis.
Figure 1 shows a 6U-high ATCA enclosure with combined switch/shelf manager slots. This type of ATCA shelf can be configured for full High Availability (HA) redundancy across all FRUs — including power modules, shelf managers/ telco alarm, cooling units, and switches. For applications that do not require AC power, this type of chassis configuration can be performed in a 5U height. When developing a horizontalmount enclosure, it is important to place units such as the shelf managers where they can be easily accessible in the front and placed in the optimal location for both signal performance (re: length of signal trace, etc.) and thermal management.
While aiming for the goal of increased performance density, it is important that you do not impede other critical functions. For example, rear IO access should not be impeded in your attempts to offer AC power, cooling, or shelf managers. Issues such as removable air filters and shelf managers that can conveniently be extracted, cable management hooks/tabs, and even an aesthetic design cannot be overlooked.
Performance density is not limited to horizontal-mount chassis, as signal performance increases along with more advanced chips such as using dual Intel Sandybridge chips on a board, and the heat buildup is intense. It’s not uncommon to see demands of 325W/front slot or more as we approach the barriers of physics in forced-air cooling, balancing airlow, acoustic noise limits, static pressure, etc. Plus, adding up to 80W/slot in the RTM area exacerbates the issue. In many cases, the only feasible solution is to increase the chassis height. Figure 2 shows the CFD thermal modeling of a 15U AdvancedTCA (Figure 3) geared to cool 400W/slot. The image shows the air velocity (yellow and red are faster airflow). From examining the effects on each slot in the enclosure, the designers can incorporate air plenums, adjust fans, and more, to optimize the cooling.
There is more to the answer than incorporating powerful blowers — the chassis solution must also prevent the significant static pressure buildup and cool for defined periods of time with some empty slots. Further, the increased air velocity will add to the acoustic noise. If you make the holes too large, you will violate shielding requirements such as FCC class B. This is a Catch-22. One way around this problem is to utilize an air “mat” right below the card cage that dissipates emissions, but has enough openings in its metallic design to allow ample airflow. So, the EMC containment is within the card cage, not the whole shelf. The larger aperture air intake holes provide enough air (without whistling) but are sized to fight the rest of the EMC battle of radiated emissions.
Reaching More Markets/Applications
AdvancedTCA is becoming more widely adopted in ground-based and less stringent rugged applications in the mil/aero markets, including computing systems in mobile communications vehicles, etc. ATCA systems can be deployed with shock-mounted racks. One convenient way for military and other mobile applications to provide both the shock/vibration protection as well as ease of transportation is using rugged transport racks. These rugged, molded shells are designed to meet with MILSTD 810F and IP65 and are stable in temperatures from –40°F to +158°F.
A key issue for telecom applications is the front-to-rear cooling requirements for NEBS-compliance. As the boards are 8U high, it is not practical to adequately cool a vertical-mount ATCA shelf in under 12U. (Again, most ATCA shelves are now 13U-15U). However, 14-slot and 13U-15U of space is not always ideal since many systems do not require more than 6 payload blades. So, to provide front-to-rear cooling in a horizontal configuration, a unique approach is feasible. By employing a rear heat extractor with the air intake at the front, air travels down the sides of the shelf, then over the blades and is forced out of the rear while impellars pull the heat to the back and out of the rear of the enclosure. The heat extractor needs to be carefully placed to optimize performance.
It is essential that a front-to-back cooling, 6-slot, telco-grade chassis also cool 325Watts+ per slot so that equipment providers can port their solutions from 14 slots to 6 slots. The combined shelf manager/switch card solution can also be incorporated into this shelf. Thus, it is possible to maintain and even increase performance density in an 8U NEBScompliant shelf with a front-to-rear cooling solution. The increase in chassis size (5U/6U to 8U) is offset by gaining the two payload slots, as described earlier. In addition, a full mesh 40G backplane offers even more performance density.
AdvancedTCA is expected to see another growth spurt as the architecture enhances its value proposition in new markets. Already proven, widely available, and reliable, ATCA is getting faster and offering more bang for the buck. The increased silicon performance, speeds across the backplane, and creative performance density techniques are requiring a shift in the electronics packaging solutions for the AdvancedTCA shelves. Solutions that offer ever more density and capitalize on the performance trends should see quite a bit of success. As powerful processors increase the thermal and power demands, it will take creative design solutions to meet these challenges.