On May 26, 2005, a new supercomputer, a pioneering giant of its time, was unveiled at BNL at a dedication ceremony attended by physicists from around the world. That supercomputer was called QCDOC, for quantum chromodynamics (QCD) on a chip, capable of handling the complex calculations of QCD, the theory that describes the nature and interactions of the basic building blocks of the universe. Now, after a career of state-of-the-art physics calculations, QCDOC has been retired — and will soon be replaced by a new “next generation” machine.
In front of the disassembled parts of the now retired giant supercomputer QCDOC are: (front, from left, holding the motherboard) Norman Christ, Columbia University (CU); Robert Mawhinney, CU; and Taku Izubuchi, RIKEN BNL Research Center (RBRC) and BNL Physics Department High Energy Theory group (HET); with (from left) Heng-Tong Ding, BNL Physics Lattice Gauge Theory group (LGT); Hiroshi Ohno, LGT; Frithjof Karsch, LGT; Taichi Kawanai, RBRC and HET; Nicholas Samios, RBRC; Masakiyo Kitazawa, Osaka University; Michael Creutz, HET/LGT; Swagato Mukherjee, LGT; Christoph Lehner, RBRC; Amarjit Soni, HET; Chulwoo Jung; HET; Tomomi Ishikawa, RHIC and RBRC; Qi Liu, CU; Eigo Shintani, RBRC.
QCDOC was a “special purpose” machine dedicated to fundamental physics. Among its prime users were the scientists of the RIKEN BNL Research Center (RBRC) at BNL, for their studies of the physics taking place at Brookhaven’s Relativistic Heavy Ion Collider (RHIC) and other forefront physics. Funded by RIKEN, the Institute of Physical & Chemical Research in Japan, with infrastructure support from DOE’s Office of Science, QCDOC was built by BNL, Columbia University, IBM, the RBRC, and the University of Edinburgh.
Most specifically, QCDOC was designed to use lattice gauge theory — a method of approximating space-time in a lattice and then computing the dynamics of quarks and gluons, the fundamental constituents of matter, by their theoretical positions on the lattice. One of the founders of this approach is BNL’s Michael Creutz.
The $5-million QCDOC, which took three years to design and build, had 10 teraflops (Tflops) of peak computing power and could perform 10 trillion arithmetic calculations per second. When installed, it took pride of place over BNL’s 1998 supercomputer called QCDSP — for quantum chromodynamics on digital signal processors — which had 0.6 Tflops at peak speed. Together, these two supercomputers provided comparable or greater computing resources for lattice QCD than were available anywhere else in the world.
“We expected great things from QCDOC, and it surpassed all expectations,” said Nicholas Samios, RBRC director. “It has been the workhorse of three groups at BNL — the High Energy Physics Theory Group, the Lattice Gauge Theory Group, and the RBRC Computing Group that together have established BNL as a world leader in the QCD field. With QCDOC installed, we produced a series of ‘firsts’ in calculations that shed light on much of the breakthrough physics at RHIC as it explores the fundamental properties of matter and conditions of the early universe.”
Taku Izubuchi, Physics Department, a leader of the RBRC Computing Group using QCDOC, explained that until 2008 the entire lattice physics program of RBRC was carried out on the QCDOC and QCDSP supercomputers.
“Through the capabilities of these machines, we developed successful techniques to study kaon physics, nucleon physics, hadron electromagnetic effects, and other important problems in low energy QCD,” he said. “Scientists from Japan, the UK, and many U.S. institutions — we have all benefited from the unique combination of QCDOC and QCDSP.
“We also benefited greatly from the excellence of the Brookhaven support staff from the machine’s installation through technical upkeep, as well as in collaborations with the Lab’s Computational Science Center and New York Center for Computational Sciences,” Izubuchi added.
Among the many highlights of the QCDOC computations were the first lattice QCD computation with chiral quarks (quarks treated with a proper separation of mirror-image spin states, important for the light quarks in nature) and the first and only lattice QCD computation including both dynamical chiral quark effects and electromagnetism. Another was the first and only direct computation for a long-standing problem in certain decays of kaon particles that were key to understanding important aspects of the Standard Model of particle physics. For more, see Physical Review D78: 034503 (2009), arXiv:1106.2714.
The success of these supercomputers at Brookhaven preceded the much larger commercial machines that exploited the same technology but had greater economy of scale. One example is the IBM Blue Gene/P established in 2008 at Argonne National Laboratory. The U.S. QCD group, of which RBRC is a part, used nearly one-quarter of this huge supercomputer for its work.
Now, a new project is under way: RBRC, Columbia University, and BNL, together with the University of Edinburgh, a collaborator in the United Kingdom KQCD group, are developing an even more powerful supercomputer, QCDCQ (QCD with Chiral Quarks). This machine will have 75 to 150 Tflops of sustained speed — key to RBRC’s continuing international leadership in lattice QCD. The researchers will also work with colleagues at the T.J. Watson Laboratory of IBM — the same colleagues who transformed the technology of QCDSP and QCDOC into IBM’s Blue Gene product. So QCDCQ will benefit from advantages of IBM’s next-generation supercomputer, which will be available in prototype form at RBRC and Edinburgh between one and two years before larger installation of commercial versions are available.
When installed, these QCDCQ machines will provide the RBRC and the larger RBRC/UKQCD collaboration with spectacular research opportunities for understanding more of what RHIC shows us about the early universe and exploring new physics at present not accessible to lattice QCD, Izubuchi said.
“There’s another great value of QCDCQ,” said Izubuchi. “Just as QCDOC provided the basis of innovations for QCDCQ, the advances we’ll make using QCDCQ will inspire the next generation of supercomputers even further into the future.”