iRODS¶
The Integrated Rule-Oriented Data System (iRODS) is a tool that provides transparent access to data distributed on different sites and on heterogeneous backends (disk filesystems, databases, tape systems etc.)
Get an account¶
You need an iRODS account to be able to use it at CC-IN2P3, which you can ask by contacting the user support. Once your request is approved, you will receive your connection parameters.
Use the iCommands¶
The iCommands are the Unix and Windows command-line utilities that interface to the iRODS system.
From an interactive server¶
If you have an account at CC-IN2P3, you may connect to the interactive machines and use the iCommands:
% ssh cca.in2p3.fr
% module load irods
% which iinit
/pbs/software/centos-7-x86_64/irods/3.3.1/bin/iinit
The iCommands version corresponding to your profile will be automatically selected.
In the vast majority of use cases, you will not need to choose a particular version of the iCommands. Nonetheless, if you wish to use a particular version of the iCommands, you can use the following command to display the available versions:
% module view irods
/pbs/software/centos-7-x86_64/modules/modulefiles
DataManagement/irods :
- 3.3.1
- 4.1.8
- 4.2.1
- 4.2.10
- 4.2.11
- 4.2.2
- 4.2.3
- 4.2.6
- 4.2.9
- default (alias: irods)
And then:
% module load DataManagement/irods/<version>
For example, if you want to use version 4.2.10:
% module load DataManagement/irods/4.2.10
% which iinit
/pbs/software/centos-7-x86_64/irods/4.2.10/bin/iinit
From an external machine¶
Please refer to the paragraph Get the iRODS client to download and install the useful binaries.
Create the connection parameters¶
Create a directory ${HOME}/.irods
and create a file ${HOME}/.irods/.irodsEnv
that contains the necessary information:
irodsHost <irods-hostname>
irodsPort <irods-port-number>
irodsHome <irods-home-dir>
irodsCwd <irods-home-dir>
irodsUserName <irods-username>
irodsZone <irods-zone-name>
Get started with iCommands¶
First of all, execute the iinit
command, which asks for the password of your iRODS account defined
in the file .irodsEnv
.
iinit
encrypts the password and stores it into a file named .irodsA
.
Once done, you should not need to provide your password again for further iCommands
.
To clean the encrypted password please run the command iexit
.
Warning
File names must NOT contain any accents or special characters like #?! *
etc.
Please find below the list of the mostly used iCommands. Use the -h
option with the command of your choice to learn more.
iCommand | Description |
---|---|
icd |
Changes the working directory |
ichmod |
Gives access/grants permissions to another user |
iexit |
Logs off/disconnects from iRODS |
iget |
Downloads the file/directory from iRODS to local device |
init |
Initializes and starts the connection to iRODS |
ils |
Lists the contents of the current working directory |
imeta |
Manages user-defined metadata |
imkdir |
Creates a new directory |
iput |
Uploads the file/directory from the local device to iRODS |
ipwd |
Shows the name and path of the current remote folder |
irm |
Moves a file to the trash |
irsync |
Synchronize a folder from a local directory to a directory in iRODs |
You can find more information about the iCommands on the iRODS documentation Web page.
Manage metadata¶
Metadata are data about your data. In iRODS metadata include system and user-defined attributes associated with a Data-Object, Collection, Resource etc. stored in the iCAT database.
To add, remove, list or query user-defined metadata, use the command imeta
:
% imeta
Usage: imeta [-vVh] [command] Commands are:
add -d|C|R|u Name AttName AttValue [AttUnits] (Add new AVU triplet)
rm -d|C|R|u Name AttName AttValue [AttUnits] (Remove AVU)
rmw -d|C|R|u Name AttName AttValue [AttUnits] (Remove AVU, use Wildcards)
ls -d|C|R|u Name [AttName] (List existing AVUs for item Name)
lsw -d|C|R|u Name [AttName] (List existing AVUs, use Wildcards)
qu -d|C|R|u AttName Op AttVal (Query objects with matching AVUs)
cp -d|C|R|u -d|C|R|u Name1 Name2 (Copy AVUs from item Name1 to Name2)
With imeta
you can add metadata to: Data-Object, Collection, User.
A metadata is defined as a triplet containing the:
- Attribute Name (AttName)
- Attribute Value (AttVal)
- Attribute Unit (AttUnits)
Add metadata to a collection /tempZone/home/calvat/images
:
% imeta add -C /tempZone/home/calvat/images type tiff
% imeta add -C /tempZone/home/calvat/images zone isere
List metadata:
% imeta ls -C /tempZone/home/calvat/images
AVUs defined for collection /tempZone/home/calvat/images:
attribute: type
value: tiff
unit:
----
attribute: zone
value: isere
unit:
Remove metadata:
% imeta rm -C /tempZone/home/calvat/images/montagne type tiff
Search data with metadata:
% imeta qu -C zone = isere
collection: /tempZone/home/calvat/images
Migration to Tape system¶
Upon request, we can trigger the migration of your incoming iRODS data to our hierarchical tape system HPSS. Please contact the user support to activate this mechanism.
Get the iRODS client¶
To connect to an iRODS server from your machine, you may download the version 4.x client from the project web site or use the package manager of your machine.
To download the version 3.x clients, select the archive corresponding to your operating system from the list below. Next, unzip the archive and adjust your environment variable PATH
accordingly.
- RHEL/CentOS 7:
v3.3.1
- Ubuntu (64 bits) :
v3.3.1
- Ubuntu (32 bits) :
v3.3.1
- Windows :
v3.0
- MacOSX :
v3.3.1
Keep in mind that to connect to 3.3.1 servers a 3.3.1 client is needed in the same way that for a 4.x server a 4.x client is needed.
Note
As of version 3.0, it may be necessary to install the Microsoft Visual Studio libraries.
It is also possible to use iRODS through Java APIs using the jargon package
.
For Windows, there is a graphical client
which allows to navigate in the iRODS tree in a user-friendly way.