iRODS

The Integrated Rule-Oriented Data System (iRODS) is a tool that provides transparent access to data distributed on different sites and on heterogeneous backends (disk filesystems, databases, tape systems etc.)

Important

CC-IN2P3 offers a user-friendly Python script that provides a graphical interface for interacting with iRODS. You may find the utility in its GitLab projet:

Available for Windows, macOS and Linux.

Get an account

You need an iRODS account to be able to use it at CC-IN2P3, which you can ask by contacting the user support. Once your request is approved, you will receive your connection parameters.

Use the iCommands

The iCommands are the Unix and Windows command-line utilities that interface to the iRODS system. You may install the client on your workstation or, if you have a computing account, you can connect from a interactive server and configure your environment. Please find below the recipes for both methods.

You may download the client from version 4.0 on the project site or use the package manager on your workstation.

To download version 3.x clients, select the archive corresponding to your operating system from the list below. Then extract the archive and adjust your PATH environment variable accordingly.

Important

Remember that users of 3.3.1 servers must have a 3.3.1 client, just as users of 4.x servers must have a 4.x client.

Information on the version of the server instance on which the user will be working (irodsZone) will be sent as an answer to the account request.

It is also possible to use iRODS through Java APIs using the jargon package.

For Windows, there’s a graphical client that lets you navigate the iRODS tree in a user-friendly way.

Get started with iCommands

Run the iinit command

% iinit

which will ask you for the following parameters (which you will have received as an answer to your account request):

irodsHost:

The name of the iRODS server hosting machine.

irodsPort:

The server connection port.

irodsUserName:

The user’s “iRODS account”.

irodsZone:

The iRODS instance where your data will be stored.

Finally, it will ask for your account password.

Once you have completed your activities on the iRODS environment, run the iexit command.

Attention

File names must NOT contain any accents or special characters like #?! * etc.

Useful iCommands

Please find below the list of the mostly used iCommands. Use the -h option with the command of your choice to learn more.

icd

Changes the working directory

ichmod

Gives access/grants permissions to another user

iexit

Logs off/disconnects from iRODS

iget

Downloads the file/directory from iRODS to local device

init

Initializes and starts the connection to iRODS

ils

Lists the contents of the current working directory

imeta

Manages user-defined metadata

imkdir

Creates a new directory

iput

Uploads the file/directory from the local device to iRODS

ipwd

Shows the name and path of the current remote folder

irm

Moves a file to the trash

irsync

Synchronize a folder from a local directory to a directory in iRODs

You can find more information about the iCommands on the iRODS documentation Web page.

Manage metadata

Metadata are data about your data. In iRODS metadata include system and user-defined attributes associated with a Data-Object, Collection, Resource etc. stored in the iCAT database.

To add, remove, list or query user-defined metadata, use the command imeta:

% imeta
Usage: imeta [-vVh] [command] Commands are:

add -d|C|R|u Name AttName AttValue [AttUnits] (Add new AVU triplet)
rm  -d|C|R|u Name AttName AttValue [AttUnits] (Remove AVU)
rmw -d|C|R|u Name AttName AttValue [AttUnits] (Remove AVU, use Wildcards)
ls  -d|C|R|u Name [AttName] (List existing AVUs for item Name)
lsw -d|C|R|u Name [AttName] (List existing AVUs, use Wildcards)
qu -d|C|R|u AttName Op AttVal  (Query objects with matching AVUs)
cp -d|C|R|u -d|C|R|u Name1 Name2 (Copy AVUs from item Name1 to Name2)

With imeta you can add metadata to: Data-Object, Collection, User.

A metadata is defined as a triplet containing the:

  • Attribute Name (AttName)

  • Attribute Value (AttVal)

  • Attribute Unit (AttUnits)

Add metadata to a collection /tempZone/home/<username>/images:

% imeta add -C /tempZone/home/<username>/images type tiff

% imeta add -C /tempZone/home/<username>/images zone isere

List metadata:

% imeta ls -C /tempZone/home/<username>/images
AVUs defined for collection /tempZone/home/<username>/images:
attribute: type
value: tiff
unit:
----
attribute: zone
value: isere
unit:

Remove metadata:

% imeta rm -C /tempZone/home/<username>/images/montagne type tiff

Search data with metadata:

% imeta qu -C zone = isere
collection: /tempZone/home/<username>/images

Migration to Tape system

Upon request, we can trigger the migration of your incoming iRODS data to our hierarchical tape system HPSS. Please contact the user support to activate this mechanism.