Storage areas

To have a general view of the storage services, you may refer to the documents used during the trainings provided by the CC-IN2P3.

Data Management Plan

A Data Management Plan (DMP) describes data lifecycle. It is not an administrative form to fill out, but a planning process to be managed by the collaboration contacts with the contribution of any user data owner.

A DMP should include:

  • general description (purpose of the data)

  • information on data processing during and after the end of the project (responsibility)

  • data usage scenarios

  • allocation of resources (costs)

  • legal and ethical aspects

  • how data will be curated and stored (including after the end of the project)

To help compile the document, CC-IN2P3 offers the RDMO web interface. To obtain the necessary accreditation, please contact user support.

Personal storage (HOME)

The user HOME directory is dedicated to the storage of personal data, and is is backed-up every day (see Backup policy). This space is associated to the Unix environment variable HOME:

% echo $HOME
/pbs/home/<u>/<user>

By default, HOME directories have by default a quota of 20 GiB and are not accessible by other users, but access rights may be modified (see Manage ACL).

Important

Although “personal” by default, this directory remains subject to professional use; the data can therefore be transmitted to the collaboration contact upon request, in particular when the account is closed.

Attention

A full HOME directory can cause errors, e.g. if jobs are trying to write on it.

Group storage

Group storage areas are shared by the users belonging to the same Unix group. Such users are expected to be working on common topics and likely to need share data.

THRONG directory

The THRONG directory is a common area where users may share with eachother important data that need to be backed-up regularly. CC-IN2P3 allocates the THRONG directory upon request to the user support. The default quota is 100 GiB.

The THRONG directory is backed-up every day (see Backup policy). It is associated with the Unix environment variable THRONG_DIR:

% echo $THRONG_DIR
/pbs/throng/<group>

When created, this space can be managed indifferently by all the users belonging to the Unix group. Users can also organize and delegate authorizations to other users (see Manage ACL).

GROUP directory

Unlike THRONG, the GROUP directory is not backed-up. It is designed to host larger data volumes, like non-critical experimental data that have to be accessed regularly by several colleagues.

CC-IN2P3 allocates the GROUP directory upon request to the user support. The default quota is 5 TiB.

This space is located on the SPS service, under the path /sps/<group>. For more information, please check the dedicated page.

When created, this space can be managed indifferently by all the users belonging to the Unix group. Users can also organize and delegate authorizations to other users.

Working areas

In addition to the HOME, THRONG et GROUP directories, you can take advantage of the /scratch directory from any interactive server to store transient data.

The /scratch directory is shared between all the users able to access the server. To use it, you can create it if it doesn’t exist yet:

% mkdir /scratch/$USER

and to store your files there.

Attention

this working area is not permanent, the data is automatically and regularly cleaned-up.

From a compute server, your job runs inside a directory in the /scratch space, which path is defined by the bact system. You can get this path value through the environment variable TMPDIR.