The Computing Centre provides storage and data management systems answering the users' different needs. In order to optimize the storage systems usage it is important to take contact with Computing Centre staff beforehand when planning a heavy load to the system. A constructive information exchange will make possible to adopt the best strategy and to prepare necessary storage system. The Computing Centre aims to optimize the access performances at the best cost and to minimize the latency time to access your data.

Data storage

The distributed file systems:

  • AFS at Computing Centre.
  • SPS at Computing Centre (GPFS).

The mass storage systems (disk & tapes):

  • HPSS at Computing Centre.
  • dCache at Computing Centre.
  • Using RFIO to access to the data stored in HPSS.

The use of specific applications to access the data:


Best practices

Please find below a quick schema for best practice usage of our infrastructure:

Where are the Data before the job? File type/format Access Files/dataset types shared concurrently by jobs Working dataset size
dCache Any / ROOT (data) read AND write read n/a (>10MiB)
non posix (dCap, ROOT, local copy)
HPSS Any (data) read OR write read n/a (>100MiB)
non posix (local copy)
iRODS Any (data) read OR write read n/a
non posix (local copy)
SPS Any + binaries + logs read AND write read AND write ≤ 8GiB : direct access
posix OR local copy > 8GiB : local copy
Xrootd ROOT (any) read (non posix) read n/a
write (ALICE) (ROOT, local copy)

PLEASE NOTE:

  • “Working dataset size” corresponds to the data concurrently accessed or in a single pass/round of a job
  • When working with datasets bigger than 8GiB, it is recommended to work from a local copy on $SCRATCH. Be aware however that on the workers the $SCRATCH is emptied at the end of the job: do not forget to copy your output elsewhere.


Data Management Plan (DMP)

Life cycle of research data is important for the proper use of these data from creation to reuse and also publication. Good data management requires the development of a Data Management Plan (DMP). This document defines the use of project data by researchers during and after the project. The DMP is regularly reviewed during the project and can therefore evolve.

The Computing Center has defined a DMP model for user experiments: DMP (French version only)

Backup Service

CC-IN2P3 provides a backup service that aims to secure on a distant site (the Computing Center), IT data present in the laboratories. The service allows data restoration from the remote copy to the original site whenever data is found modified or destroyed. This service is based on IBM Tivoli Storage Manager (TSM).

For more information, visit the backup documentation

Data transfer

There are several tools available for copying or transferring your data and files from the Computing centre to another place (or inversely). The choice will be made according your needs in terms of performances, data volume to transfer, facility to use.

To have more information, please visit the page Data transfer .


In case of problems, please submit a ticket at CC-IN2P3 User Support.

  • en/stockage-et-transfert.txt
  • Last modified: 2018/09/12 14:13
  • by Gino MARCHETTI