Jupyter Notebooks Platform

JupyterLab is a web-based interactive development environment. You can develop code, write documentation, execute your code, visualise results including graphs in a same document. It is thus a convenient tool to explore data and share your work with collaborators. To find more information please refer to the Jupyter project web site.

The CC-IN2P3 Jupyter Notebooks Platform is provided to any user having a computing account at CC-IN2P3. For a general understanding of the tool, please refer to the official JupyterLab Documentation. If you experience any problem, please open a ticket to user support.

https://notebook-beta.cc.in2p3.fr

Access is granted through the eduGAIN identity federation or by entering your login and password from your computing account.

Important

JupyterLab does not run on a dedicated server: each session is launched as an interactive job on the Slurm computing platform. You must define the resources to be allocated before starting it:

• select your group,

• select the partition (required to access GPUs),

• number of CPU cores (1 core by default) or the number of GPUs (if GPU partition chosen),

• amount of memory (3 GB by default),

• maximum execution time, or walltime (6 hours by default, 7 days maximum).

The resources required for the Slurm job execution are deducted from the allocation of the selected group as decided during the annual resource request.

• Gestion des noyaux Jupyter
  • Provided kernels
  • Add a custom kernel
  • Dependencies
  • Creation of the kernel.json file
  • Usage of the jupyter-helper.sh
  • GPU kernels
  • Troubleshooting

• Dask usage
  • Prerequisites
  • Features
  • Limitations

Notebooks Examples

A collection of ready-to-use examples categorized by usage

https://gitlab.in2p3.fr/ccin2p3-support/utilities-examples/examples/notebooks

Running the service

The eduGAIN connection to the platform requires not only to be known by your identity provider, but also to own an active computing account and to be able to map both: the email associated to your eduGAIN identity should be the same as your primary email of your CC-IN2P3 account. Please check your account information as explained in the account management page.

The STDOUT and STDERR from your notebook server are available in the Slurm job log file. These files are stored in your HOME directory, under ~/.jupyterhub/notebook_server_<job_number>.log, where <job_number> is the Slurm job ID assigned to the server.

Attention

A JupyterLab session may stop automatically when the defined limits are reached, in particular the walltime. If your session stops, check the Slurm job logs and adjust the requested resources if necessary.

If you are a member of several groups, the group selected when filling the form will be associated to your notebook server, which means that every newly created file will belong to that group.

Storage resources access

Once logged in, the notebook server will give you access to the following storage areas:

• HOME directory: /pbs/home/<u>/<username>

• THRONG directory: /pbs/throng

• SOFTWARE directory: /pbs/software

• SPS directory: /sps/<group> depending on your primary and secondary groups

Note

SPS directories must be granted to be available through the notebook server. Ask our user support if you don’t find them.

The landing directory is set on the user’s HOME directory. To browse the files through the interface on the other directories listed above, create the appropriate symbolic links on your HOME. Example:

% ln -s /pbs/throng/<group> $HOME/throng_<group>

A default environment is provided for your convenience. If it does not suit your needs, you may add your own following the kernel procedure.

Practical tips

Launching

If you can’t open a notebooks server anymore, please do one of the following in order of priority:

• make sure your HOME is not full with the command fs4 listquota (once you are in your HOME),

• clear your browser’s cache,

• delete the directory $HOME/.local/share/jupyter/runtime,

• delete the directory $HOME/.jupyter/lab/workspaces,

• delete the directory $HOME/.local/etc/jupyter (or rename it $HOME/.local/etc/jupyter.bak),

• delete the directory $HOME/.jupyter (or rename it $HOME/.jupyter.bak). If not saved, the file $HOME/.jupyter/jupyter_notebook_config.py can be re-generated by the command jupyter notebook --generate-config only from a terminal of the Notebooks server.

Attention

The two last suggetions will also delete your personal configuration (especially if the jupyter_notebook_config.py has been deleted). They must be used with caution.

Logout

To stop your notebook server, use File > Log Out (closing the tab in your browser will not stop your notebook server from running).

• In any case: - The notebook server will be terminated once the time limit (walltime) specified in the login form is reached, - The maximum runtime (as configured by Slurm) for a notebook server is 7 days.

Available JupyterLab extensions

To obtain the list of available extensions, use the following command in your notebook terminal:

List of extensions that extend or modify the Jupyter server backend

% jupyter server extension list

Note

If the jupyter command is not available by default, it can be installed via pip.

There is also a panel in the JupyterLab interface to view installed extensions: simply click on the puzzle-shaped icon in the left vertical toolbar of the window.