Hierarchical storage

The High Performance Storage System (HPSS) service is a system capable of storing very large quantities of data with a relatively good performance for write and read operations. The particularity of this service is the use magnetic tapes as a storage medium.

Important

  • HPSS is not an archival system, it only stores a unique replica of the files.

  • HPSS is particularly unsuitable for storing small files ( < 100 MB ),

  • Read access times can vary from a few minutes to a few hours,

  • Prefer the use of HPSS for long-term data storage. For temporary or short-lived data (a few months), it is better to use SPS.

File access and management

File management on HPSS is based on RFIO. To source the RFIO environment, please load “shift”:

% module load shift

Here are the most common commands to manage files with RFIO:

rfcp

remote file copy

rfchmod

change file access permission

rfdir

remote directory list

rfmkdir

make remote directories

rfrename

rename remote file

rfrm

remove remote files and directories

rfstat

get information about a file or directory

These commands should be used with the RFIO server cchpss0, in the following way:

% rfcp /scratch/monfichier cchpss0:/hpss/in2p3.fr/group/<group>/  # Write a file

% rfdir cchpss0:/hpss/in2p3.fr/group/<group>/                     # List a directory

% rfcp chpss0:/hpss/in2p3.fr/group/<group>/<my file> /scratch/   # Read a file

Most groups have dedicated RFIO servers. In this case the name of the RFIO server is cchpss<group>, for example: cchpssatlas, cchpssantares, …

Note

More examples of using the RFIO commands can be found in the CC-IN2P3 trainings, and more specifically in those dedicated to the use of the storage.

Finding a file in the HPSS tree can be slow. In case of frequent calls to find a file in HPSS, we recommand the use of a dedicated database or catalogue.

File permissions

File permissions on HPSS work in the same way as standard Unix permissions. Keep in mind that rfcp propagates the original file permissions to the new copy.

Access from jobs

To access HPSS from a job running on the computing platform, please refer to the syntax described in the paragraph about the storage resources declaration.

File size

HPSS supports a maximum file size of 64 GB. In order to benefit from better I/O performance, users are invited to store large files (> 1GB) in HPSS, the optimum being 4 GB.

I case you need to store a large number of small files, please contact the user support.

File retrival

If the requested files are not present in the HPSS disk cache, they must be retrieved from the tapes, leading to significant delays (up to several hours).

Whether from jobs or from interactive machines, it is strongly recommended to use XRootD to read data. Indeed, the XRootD service has substantial disk caches and implements optimization mechanisms to facilitate data reading from HPSS.

% module load xrootd

% xrdcp root://ccxroot.in2p3.fr:1999//hpss/in2p3.fr/<path> / <my file>