Differences

This shows you the differences between two versions of the page.

Link to this comparison view

en:soumetteur_de_jobs_par_un_script_automatique [2017/10/25 16:19] (current)
Gino MARCHETTI created
Line 1: Line 1:
 +
 +====== Submit a job with an automatic script ======
 +
 +\\
 +\\
 +
 +=====  Introduction =====
 +
 +Production submission is a complex task requiring a good knowledge of the computing farm to be optimized. The idea is to provide a command allowing to effectively submit the jobs on the farm. The submission takes place in two stages:
 +   * Create the job list by using the create_job_list.py command and a template
 +   * Submission of all jobs using the jobs_submit.py command and the list of jobs in a demon job
 +
 +
 +=====  Create the job list =====
 +
 +You will firstly need a template file (check an example in /​afs/​in2p3.fr/​home/​c/​calvat/​public/​jobsubmit/​template.txt). This file contains a single line namely the qsub command used for the set of jobs.
 +
 +Example for template.txt :
 +
 +template.txt
 +
 +<​code>​
 +qsub -l  sps=   ​1 ​ -o stdout_ ​ ##​_index_##​.txt -e stderr_##​_index_##​.txt /​afs/​in2p3.fr/​home/​c/​calvat/ ​ test  .sh   "​my_name##​round((_index_+1)*2+23.,​0)##​_##​round(cos(1.),​2)*_index_##"​
 +</​code>​
 +
 +
 +
 +
 +Code sample to prepare a production :
 +
 +Prepare a 10 jobs production\\
 +
 +
 +<​code>​
 +/​afs/​in2p3.fr/​home/​c/​calvat/​public/​jobsubmit/​create_job_list.py -i  10  -t /​afs/​in2p3.fr/​home/​c/​calvat/​public/​jobsubmit/​template.txt -o /​afs/​in2p3.fr/​home/​c/​calvat/​task.txt
 +</​code>​
 +
 +
 +
 +
 +
 +=====  Job submission =====
 +
 +
 +====  Interactive ====
 +
 +\\
 +
 +Running the jobs interactively
 +
 +<​code>​
 +/​afs/​in2p3.fr/​home/​c/​calvat/​public/​jobsubmit/​jobs_submit.py -p /​afs/​in2p3.fr/​home/​c/​calvat/​task.txt -i 10MiB
 +</​code>​
 +
 +
 +
 +
 +
 +====  In a job demon ====
 +
 +Running the jobs with a job demon
 +
 +<​code>​
 +qsub -l  demon= ​  ​1 ​ -q demon /​afs/​in2p3.fr/​home/​c/​calvat/​public/​jobsubmit/​jobs_submit.py -p /​afs/​in2p3.fr/​home/​c/​calvat/​task.txt -i 10MiB
 +</​code>​
  
  • en/soumetteur_de_jobs_par_un_script_automatique.txt
  • Last modified: 2017/10/25 16:19
  • by Gino MARCHETTI