Data Transfer to RCF
by Tom Throwe
300 Megabytes per second, or the contents of an audio CD in two seconds, or the contents of a DVD in 20 seconds--that is the rate at which data is being transferred from the IC experiments to the RHIC Computing Facility (RCF). Data from the PHENIX and STAR experiments start out as signals in their detectors. These signals are collected by the Front End Electronics (FEE) attached to the detectors and the readout of these FEE modules are assembled by the Data Acquisition systems of the two experiments. The resulting events are stored in files on disks arrays in the PHENIX and STAR Counting Houses until they are ready to be transferred to the Mass Storage System of the RCF.
The event files are transferred to the RCF using the pftp (for parallel ftp) client of HPSS (High Performance Storage System). The transfers are done over pairs of single mode fiber optic cables stretching from the counting houses to the RCF. The transfers used to be done over bonded (electronically combined) one Gigabit per second Ethernet channels, where individual Gigabit per second interfaces were aggregated together to provide multiple Gigabit per second throughput. Each Gigabit connection has a maximum throughput of 125 Megabytes per second. With the latest demands from the PHENIX experiment, the bonding method proved problematic and it was replaced with a single ten Gigabit per second interface over a single fiber pair. A ten Gigabit per second connection has a maximum throughput of 1250 Megabytes per second.
The pftp transfers occur between the disk buffer machines in the counting houses and the "mover" machines in HPSS. The transfers place the event files in the disk cache of HPSS connected to the mover machines and then the files are migrated from the disk cache to tape storage by the movers. The tape drives used in HPSS are third generation LTO (Linear Tape - Open) drives. These drives are capable of storing 400 GigaBytes of uncompressed data per cartridge and are capable of sustained write speeds of 50 Megabytes per second. Multiple tape drives are used in parallel to achieve that necessary aggregate migration speed to keep up with the incoming data.
The LTO tape drives are housed in two Sun Storage Tek SL8500 Modular Library Systems or robots. Each robot can accommodate up to 64 tape drives and, when expanded to its full five modules, over 10,000 tape cartridges. With LTO-3 cartridges, the robot can hold 3.8 PetaBytes of data or the contents of approximately 800,000 DVDs. The fourth generation of LTO drive will become available later this year and it will double both the capacity of the tape cartridge and the write speed of the drive.
The future holds detector and Data Acquisition system upgrades for the RHIC experiments and luminosity upgrades for the collider. Each of these will lead to increases in the transfer rate of data to the RCF, and the RCF will continue to upgrade its technology and bandwidth to keep up with the data flowing from RHIC.