Data Management

Scientific trends are toward experiments that collect greater volumes of data, and CFN facilities are no exception. For example, the CFN Environmental Transmission Electron Microscope is capable of high-frame-rate imaging, which captures 4 TB of image data in 15 minutes. The CFN is working to implement solutions for its Users and staff in order to manage these large quantities of information and easily transport data offsite to home institutions. While we are still developing a Center-wide, integrated solution, some noteworthy practices include:

• For Users collecting substantive amounts of data, the CFN has access to approximately 500 PB of temporary storage onsite at the RHIC/ATLAS Computing Facility. Users with this level of storage needs are encouraged to consider cloud-based data management services, such as Globus Online, for data retrieval and transfer.
• The CFN is working to establish a cloud-based data management offering, which will unify and facilitate data transfer for Users of all CFN facilities.
• Users of the CFN Computational Cluster gain access to their data content through remote login via the secure BNL Virtual Private Network (VPN). This allows Users to generate and manage their data remotely for the duration of their User project.
• Although an unsophisticated approach, many Users bring portable, high-capacity storage devices to the CFN and transport data by hand to their home institutions. Data transfer from CFN instruments to portable storage devices is conducted through secondary, data-retrieval computers to safeguard against viruses and malware.

Data Preservation

Unfortunately, the CFN cannot keep or maintain User data indefinitely. We expect Users to retrieve their data when concluding their work in the facility. Because the CFN cannot guarantee preservation of data after completion of a single facility session at the CFN, we encourage Users to transfer their data to self-managed resources as soon as possible.