Last modified: Jan 24, 2014 by Kurca

CRAB - user



Chapitre 1 : Jobs Status

"Done"  on the Dashboard
  --> job should not be "Running" in CRAB
"Unknown" on the Dashboard
  --> Dashboard probably didn't receive job exit code from the worker node
  --> CRAB might know --> do :   crab -status 

Chapitre 2 : Stage output to your SE

Without Using CMS Storage Name Convention

…. one has to specify both the

user_remote_dir = /test
   storage_path=/srm/managerv2?SFN=/pnfs/in2p3.fr/data/cms/t2data/store/user/kurca
  
crab.cfg
 [USER]
return_data=0
email=kurca@in2p3.fr

copy_data = 1
storage_element = ccsrmt2.in2p3.fr 
user_remote_dir = /test
storage_path = /srm/managerv2?SFN=/pnfs/in2p3.fr/data/cms/t2data/store/user/kurca

publish_data = 0
#publish_data_name = My_1st_test_TauloTTbar 
#dbs_url_for_publication = https://cmsdbsprod.cern.ch:8443/cms_dbs_prod_local_09_writer/servlet/DBSServlet


  

Using CMS Storage Name Convention

…. it is enough to specify SE resp. user_remote_dir

user_remote_dir = /test
storage_element = T2_CH_CAF

and the output will be written to the directory:

/castor/cern.ch/cms/store/user/kurca   

or if with user_remote_dir to

/castor/cern.ch/cms/store/user/kurca/test
crab.cfg
 [USER]
return_data=0
email=kurca@in2p3.fr

copy_data = 1 
storage_element = T2_CH_CAF
user_remote_dir = /test

Without Using CMS Storage Name Convention

…. one has to specify both the

user_remote_dir = /test
   storage_path=/srm/managerv2?SFN=/pnfs/in2p3.fr/data/cms/t2data/store/user/kurca
  
crab.cfg
 [USER]
return_data=0
email=kurca@in2p3.fr

copy_data = 1
storage_element = ccsrmt2.in2p3.fr 
user_remote_dir = /test
storage_path = /srm/managerv2?SFN=/pnfs/in2p3.fr/data/cms/t2data/store/user/kurca

publish_data = 0
#publish_data_name = My_1st_test_TauloTTbar 
#dbs_url_for_publication = https://cmsdbsprod.cern.ch:8443/cms_dbs_prod_local_09_writer/servlet/DBSServlet


  

Using CMS Storage Name Convention

…. it is enough to specify SE resp. user_remote_dir

user_remote_dir = /test
storage_element = T2_FR_CCIN2P3

and the output will be written to the directory:

/pnfs/in2p3.fr/data/cms/t2data/store/user/kurca   

or if with user_remote_dir to

/pnfs/in2p3.fr/data/cms/t2data/store/user/kurca/test
crab.cfg
 [USER]
return_data=0
email=kurca@in2p3.fr

copy_data = 1 
storage_element = T2_FR_CCIN2P3
user_remote_dir = /test

Chapitre 3 : Test if gridftp is working correctly

from any machine with UI

voms-proxy-init --voms cms
edg-gridftp-ls -v gsiftp://ccgridli03.in2p3.fr/

CRAB servers in Legnaro resp. Bari

edg-gridftp-ls -v gsiftp://crabas.lnl.infn.it/data/
edg-gridftp-ls -v gsiftp://dot1-prod-2.ba.infn.it/data

Chapitre 4 : Using CRAB Server

1) Grid UI
    lcg_env
2) CMSSW environment
   source $VO_CMS_SW_DIR/cmsset_default.(c)sh  (alias " cms_def ")  
  cd into the CMSSW release directory
  (created before with: "scramv1 project CMSSW CMSSW_3_3_1 " )
  eval `scramv1 runtime -(c)sh`             (alias " cms_sw ")
3) CRAB environment 
  source $VO_CMS_SW_DIR/CRAB/crab.(c)sh       (alias " crabS ")
   .....
  (at CERN: source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.csh)
  ...
4)  crab -create
5)  crab -submit

Chapitre 5 : Using CRAB Client - Standalone

If  you are in your CMSSW_x_y_z working directory do simply:
      cms_env
This alias is doing for you the following setups: 
1) lcg_env
2) CMSSW environment  ( * needed also for CMSSW installation, i.e. to create your working area  CMSSW_x_y_z )
   source $VO_CMS_SW_DIR/cmsset_default.(c)sh   (alias " cms_def ")
    (* see alias cmsrel)
  eval `scramv1 runtime -(c)sh`    (alias " cms_sw " )
3) CRAB environment : 
   source $VO_CMS_SW_DIR/CRAB/crabX.(c)sh   (alias " crabX ")
 
......
4) crab -create
5) crab -submit

   Comment: "crabX  "  is in fact "crabS " +  
setenv PYTHONPATH ${PYTHONPATH}:${GLITE_LOCATION}/lib:${GLITE_LOCATION}/lib/python
  
!!! pour bash shell:
   export PYTHONPATH=${PYTHONPATH}:${GLITE_LOCATION}/lib:${GLITE_LOCATION}/lib/python
  

Chapitre 6 : SiteDB Check - HN-extraction

https://twiki.cern.ch/twiki/bin/view/CMS/SiteDBForCRAB#Check_HN_name_extraction

Check HN name extraction

* In order to check if your HN name can be extracted from SiteDB please download and run this script:
o check_HN_Name.py
* usage:
o “python check_HN_Name.py” will check the extraction in a standalone way
o “python check_HN_Name.py crab” will check the extraction using the crab utilities. NOTE: this require the crab environment properly configured.

Chapitre 7 : Getting the results small datasets by "hand" to local UI

If you still have the crab working directory, this can be done automatically by doing:

crab -copyData

or, you can have the srm path by doing

crab -report

and then do manually srmcp

Chapitre 8 : Erase "wrong" data from SE

edg-gridftp-ls   (edg-gridftp-rm )
gsiftp://polgrid4.in2p3.fr:2811/dpm/in2p3.fr/home/cms/trivcat/store/user/kurca/ZZ_4l_10TeV_GEN/ZZ_4l_10TeV_GEN_HLT/3ff4a4f43d11ae9733d067a20ed87255

Chapitre 9 : Inspect your data in DBS

InspectDBS2.py --DBSURL=https://vocms31.cern.ch:8443/cms_dbs_ph_analysis_01_writer/servlet/DBSServlet --datasetPath=/ZZ_4l_10TeV_GEN/kurca-ZZ_4l_10TeV_GEN_HLT-3ff4a4f43d11ae9733d067a20ed87255/USER/
https://cmsweb.cern.ch/dbs_discovery/aSearch?caseSensitive=on&userMode=expert&sortOrder=desc&sortName=&grid=0&method=dbsapi&dbsInst=cms_dbs_ph_analysis_01&userInput=find+dataset+where+dataset+like+%2F*ZZ_4l_10TeV_GEN%2F*HLT*%2FUSER+
 
  • en/crab_-_user.txt
  • Last modified: 2016/12/16 10:16
  • (external edit)