Différences

Ci-dessous, les différences entre deux révisions de la page.

Lien vers cette vue comparative

mc-production [2016/12/16 10:15] (Version actuelle)
Ligne 1: Ligne 1:
 +Modifié par Kurca, le 15 Jun 2006\\
 +\\
 +
 +====== MC-production ======
 +
 +\\
 +\\
 +
 +====== Chapitre 1 : Introduction ======
 +
 +Login as a user **d0prod** on the **ccd0** .in2p3.fr node! 
 +====== Chapitre 2 : Get a new request ======
 +
 +Check the current MC- requests from the Dzero web page\\
 +[[#​url|http:​%%//​%%www-d0.fnal.gov/​computing/​mcprod/​mcc.html]] .\\
 +To get it (make a reservation) for production at CCIN2P3 do:
 +<​code>​
 +cd Production
 +Queue.new.py ​ or
 +Queue.new.py --requestid=<​requestid>​
 +Do you want to run this request (answer yes):yes
 +</​code>​
 +Note the number of events to be produced (add 10% more ...) 
 +====== Chapitre 3 : Prepare new request locally ======
 +
 +<​code>​
 +cd ~/req
 +</​code>​
 +Select an existing requestID with a process and number of events close to the new requirement.
 +<​code>​
 +newReq_no_queue.sh <old requestid>​ <​requestid>​
 +</​code>​
 +The new macro and the differences with the <old requestid>​ are shown. You should edit/change correspondigly all macros.\\
 +\\
 +Attention! Required CardfileVersion (e.g. v00-09-33) should be in the $THRONG_DIR/​dist/​packages/​cardfiles\\
 +If it is missing, install it before finishing the procedure newReq_no_queue.sh\\
 +\\
 +Example:
 +<​code>​
 +setup d0cvs
 +cvs co -r v00-09-33 cardfiles
 +Enter passphrase for RSA key '/​afs/​in2p3.fr/​home/​l/​lebrun/​.ssh/​identity':​
 +mv cardfiles ​ v00-09-33 ????? 
 +</​code>​
 +Generally, you have to modify pythia-gen.macro (cfg pythia)\\
 +Don't modify :\\
 +cfg pythia define int RunNumber <​runnumber>​\\
 +cfg pythia define int NumRecords <​numrecords>​\\
 +cfg pythia define int UseMaxopt 1\\
 +\\
 +Remove: .... really ?????????\\
 +cfg pythia define int RunNumber 0\\
 +cfg pythia define int UseMaxopt 1\\
 +cfg pythia define string SamDeclareOutput on\\
 +\\
 +Modify d0simreco.macro:​\\
 +Pay attention to :\\
 +attach d0sim\\
 +cfg d0sim define string MinBiDataset\\
 +/​afs/​in2p3.fr/​throng/​d0/​info_data/​mcp17/​Zerobias_p17MC_avelumi_1_rename2.py\\
 +\\
 +Modify local_request_<​requestid>​.py\\
 +\\
 +Ask for about 10% events more than requested in order to have a safety margin\\
 +for crashing jobs.\\
 +"​numevents/​nfirstjobs"​ should be close to the integer number 13000 and they have to be\\
 +a multiple of 250.\\
 +Example for 200000 events:
 +<​code>​
 +recphys={
 +        '​type':'​mcprod',​
 +        '​nfirstjobs':'​14',​
 +        '​numevents':'​210000'​
 +</​code>​
 +(For 150000 events --> 12 and 153000...)\\
 +\\
 +Check:
 +<​code>​
 +adminphysid=[
 +         ​['​lebrun','​no','​ok'​],​
 +         ​['​kurca','​no','​no'​],​
 +         ​['​d0prod','​ok','​ok'​]
 +         ]
 +..... and the options for job type:
 +
 +     ​saminfos={
 +        '​gen':'​locate',​
 +        '​d0gstar':'​locate',​
 +        '​d0sim':'​declare',​
 +        '​thumb':'​declare',​
 +        '​mrg':'​store'​
 +        }
 +
 +</​code>​
 +At the end of this procedure do:
 +<​code>​
 +neter setNumberOfJobs --requestid=<​requestid>​ --jobtype=d0gstar --number=0
 +</​code>​
 +This commad will set all **d0gstar** jobs into //standby// state, i.e. jobs will not run until\\
 +they are freed.
 +=====  Check Random Number Seed  =====
 +
 +Before liberating those jobs you should check if Pythia generator didn't used the same\\
 +random number. In order to do it you should wait until all jobs have finished and the output files are\\
 +declared.\\
 +Then you can do:
 +<​code>​
 +request=<​requestid>​
 +     neter qjob --requestid=$request
 +     ​for ​ jobname in `d0select.py --sep="​ " ​ "​select jobname from 
 +     ​d0_admin.samjobs where requestid in $request and type in 
 +     ​('​gen','​alpgenpythia'​)"​`
 +     do
 +     ​cmd="​grep RanSeed $GROUP_DIR/​prodOracle/​$jobname/​td_gen-*.conf"​
 +     echo $cmd
 +     eval $cmd
 +     done | grep RanSeed | sort
 +</​code>​
 +If, e.g. 2 files were produced with the same random number //​(RanSeed2)//​ ,\\
 +then take one of those files and find out the corresponding job name (with the same command as\\
 +above, but without a grep pipe).\\
 +Change the status of the found job in the Oracle DB :
 +<​code>​
 +oraclea
 +<​password>​
 +update jobs set job_status='​error'​ where jobname='<​jobname>';​
 +commit;
 +exit
 +
 +</​code>​
 +
  
  • mc-production.txt
  • Dernière modification: 2016/12/16 10:15
  • (modification externe)