The first row of cabinets for the new Trinity supercomputer at the Lab. Credit: Los Alamos National Laboratory A moment of inspiration during a wiring diagram review has saved more than $2 million in material and labor costs for the Trinity supercomputer at Los Alamos National Laboratory.
The Laboratory's High Performance Computing (HPC) facilities team, led by Data Center Manager Ron Velarde of HPC-Design (HPC-DES). discovered the potential to re-engineer Cray's initial wiring diagram for the power feed to Trinity's computing racks. As a result, the facilities team was able to perform a redesign, approved by Cray and the Laboratory, that ultimately saved the Lab an estimated $2.6 M in material and labor costs. Sandia National Laboratories adopted this redesign for their Trinity-like systems and Lawrence Berkeley National Laboratory is considering it as well.
"My team and I are always looking to improve engineering means and methods for supercomputer installations in our data centers," said Velarde.
Trinity, Los Alamos's latest major high-performance computer, is designed to provide increased computational capability for the NNSA Nuclear Security Enterprise in order to improve geometric and physics accuracy in calculations that can be completed in weeks—not years.
The facilities team also used prefabricated copper tray cable rather than fabricating the 22,000 feet of cables on site, which was the plan originally proposed for the project. For high-performance computing systems in the Laboratory Data and Communications Center (LDCC), where power cables are being routed over the computers for the first time, the facilities team worked with the Lab's electrical standards personnel to approve new aluminum power cables, saving 20 percent in materials cost and a factor of three in weight. This pioneering use of aluminum is also expected to generate cost and weight savings in other areas of the Laboratory in addition to the high-performance computing data centers.
The Trinity Phase 1 system, based on the Haswell processor, is currently performing simulations. The Trinity Phase 2 system, based on the Knight's Landing processor, is being delivered this summer. As an example of the scales involved, Phase 2 of the Trinity system, which began to arrive in June, weighs just under 100 tons. Trinity in its completed form will weigh about 175 tons and contain twelve miles of copper cable and 44 miles of optical cable. More 16,000 gallons of water will circulate in the inner process loop to cool Trinity at a flow rate of around 10,000 gallons per minute.
The team and the funding
The team that performed the work includes Velarde, Andres Borrego, Michael Ferguson, Alynna Montoya-Wiuff, Eloy Romero, and Loren Serna, all of the Los Alamos HPC-DES group. The NNSA and Advanced Simulation and Computing (ASC) Program funded the work, which supports the Lab's Nuclear Deterrence mission area and the Integrating Information, Science and Technology for Prediction science pillar.
Explore further: LLNL, Intel, Cray produce big data machine
