Differences

This shows you the differences between two versions of the page.

Link to this comparison view

en:crab_-_user [2016/12/16 10:16] (current)
Line 1: Line 1:
 +Last modified: Jan 24, 2014 by Kurca\\
 +\\
 +
 +====== CRAB - user ======
 +
 +\\
 +\\
 +
 +====== Chapitre 1 : Jobs Status ======
 +
 +<​code>​
 +"​Done" ​ on the Dashboard
 +  --> job should not be "​Running"​ in CRAB
 +</​code>​
 +<​code>​
 +"​Unknown"​ on the Dashboard
 +  --> Dashboard probably didn't receive job exit code from the worker node
 +  --> CRAB might know --> do :   crab -status ​
 +</​code>​
 +
 +====== Chapitre 2 : Stage output to your SE ======
 +
 +=====  CERN  =====
 +
 +
 +====  Without Using CMS Storage Name Convention ​ ====
 +
 +.... one has to specify both the
 +<​code>​
 +user_remote_dir = /test
 +   ​storage_path=/​srm/​managerv2?​SFN=/​pnfs/​in2p3.fr/​data/​cms/​t2data/​store/​user/​kurca
 +  ​
 +</​code>​
 +<​code>​
 +crab.cfg
 + ​[USER]
 +return_data=0
 +email=kurca@in2p3.fr
 +
 +copy_data = 1
 +storage_element = ccsrmt2.in2p3.fr ​
 +user_remote_dir = /test
 +storage_path = /​srm/​managerv2?​SFN=/​pnfs/​in2p3.fr/​data/​cms/​t2data/​store/​user/​kurca
 +
 +publish_data = 0
 +#​publish_data_name = My_1st_test_TauloTTbar ​
 +#​dbs_url_for_publication = https://​cmsdbsprod.cern.ch:​8443/​cms_dbs_prod_local_09_writer/​servlet/​DBSServlet
 +
 +
 +  ​
 +</​code>​
 +
 +====  Using CMS Storage Name Convention ​ ====
 +
 +.... it is enough to specify SE resp. user_remote_dir
 +<​code>​
 +user_remote_dir = /test
 +storage_element = T2_CH_CAF
 +</​code>​
 +and the output will be written to the directory:
 +<​code>​
 +/​castor/​cern.ch/​cms/​store/​user/​kurca ​  
 +</​code>​
 +or if with user_remote_dir to
 +<​code>​
 +/​castor/​cern.ch/​cms/​store/​user/​kurca/​test
 +</​code>​
 +<​code>​
 +crab.cfg
 + ​[USER]
 +return_data=0
 +email=kurca@in2p3.fr
 +
 +copy_data = 1 
 +storage_element = T2_CH_CAF
 +user_remote_dir = /test
 +</​code>​
 +=====  CCIN2P3 ​ =====
 +
 +
 +====  Without Using CMS Storage Name Convention ​ ====
 +
 +.... one has to specify both the
 +<​code>​
 +user_remote_dir = /test
 +   ​storage_path=/​srm/​managerv2?​SFN=/​pnfs/​in2p3.fr/​data/​cms/​t2data/​store/​user/​kurca
 +  ​
 +</​code>​
 +<​code>​
 +crab.cfg
 + ​[USER]
 +return_data=0
 +email=kurca@in2p3.fr
 +
 +copy_data = 1
 +storage_element = ccsrmt2.in2p3.fr ​
 +user_remote_dir = /test
 +storage_path = /​srm/​managerv2?​SFN=/​pnfs/​in2p3.fr/​data/​cms/​t2data/​store/​user/​kurca
 +
 +publish_data = 0
 +#​publish_data_name = My_1st_test_TauloTTbar ​
 +#​dbs_url_for_publication = https://​cmsdbsprod.cern.ch:​8443/​cms_dbs_prod_local_09_writer/​servlet/​DBSServlet
 +
 +
 +  ​
 +</​code>​
 +
 +====  Using CMS Storage Name Convention ​ ====
 +
 +.... it is enough to specify SE resp. user_remote_dir
 +<​code>​
 +user_remote_dir = /test
 +storage_element = T2_FR_CCIN2P3
 +</​code>​
 +and the output will be written to the directory:
 +<​code>​
 +/​pnfs/​in2p3.fr/​data/​cms/​t2data/​store/​user/​kurca ​  
 +</​code>​
 +or if with user_remote_dir to
 +<​code>​
 +/​pnfs/​in2p3.fr/​data/​cms/​t2data/​store/​user/​kurca/​test
 +</​code>​
 +<​code>​
 +crab.cfg
 + ​[USER]
 +return_data=0
 +email=kurca@in2p3.fr
 +
 +copy_data = 1 
 +storage_element = T2_FR_CCIN2P3
 +user_remote_dir = /test
 +</​code>​
 +
 +====== Chapitre 3 : Test if gridftp is working correctly ======
 +
 +from any machine with UI
 +<​code>​
 +voms-proxy-init --voms cms
 +edg-gridftp-ls -v gsiftp://​ccgridli03.in2p3.fr/​
 +</​code>​
 +CRAB servers in Legnaro resp. Bari
 +<​code>​
 +edg-gridftp-ls -v gsiftp://​crabas.lnl.infn.it/​data/​
 +edg-gridftp-ls -v gsiftp://​dot1-prod-2.ba.infn.it/​data
 +</​code>​
 +
 +====== Chapitre 4 : Using CRAB Server ======
 +
 +=====  Setup your environment ​ =====
 +
 +<​code>​
 +1) Grid UI
 +    lcg_env
 +2) CMSSW environment
 +   ​source $VO_CMS_SW_DIR/​cmsset_default.(c)sh ​ (alias " cms_def "​)  ​
 +  cd into the CMSSW release directory
 +  (created before with: "​scramv1 project CMSSW CMSSW_3_3_1 " )
 +  eval `scramv1 runtime -(c)sh` ​            ​(alias " cms_sw ")
 +3) CRAB environment ​
 +  source $VO_CMS_SW_DIR/​CRAB/​crab.(c)sh ​      ​(alias " crabS ")
 +   .....
 +  (at CERN: source /​afs/​cern.ch/​cms/​ccs/​wm/​scripts/​Crab/​crab.csh)
 +  ...
 +4)  crab -create
 +5)  crab -submit
 +</​code>​
 +
 +====== Chapitre 5 : Using CRAB Client - Standalone ======
 +
 +=====  Setup your environment ​ =====
 +
 +<​code>​
 +If  you are in your CMSSW_x_y_z working directory do simply:
 +      cms_env
 +</​code>​
 +<​code>​
 +This alias is doing for you the following setups: ​
 +1) lcg_env
 +2) CMSSW environment ​ ( * needed also for CMSSW installation,​ i.e. to create your working area  CMSSW_x_y_z )
 +   ​source $VO_CMS_SW_DIR/​cmsset_default.(c)sh ​  ​(alias " cms_def ")
 +    (* see alias cmsrel)
 +  eval `scramv1 runtime -(c)sh` ​   (alias " cms_sw " )
 +3) CRAB environment : 
 +   ​source $VO_CMS_SW_DIR/​CRAB/​crabX.(c)sh ​  ​(alias " crabX ")
 + 
 +......
 +4) crab -create
 +5) crab -submit
 +
 +   ​Comment:​ "​crabX ​ " ​ is in fact "crabS " +  ​
 +setenv PYTHONPATH ${PYTHONPATH}:​${GLITE_LOCATION}/​lib:​${GLITE_LOCATION}/​lib/​python
 +  ​
 +!!! pour bash shell:
 +   ​export PYTHONPATH=${PYTHONPATH}:​${GLITE_LOCATION}/​lib:​${GLITE_LOCATION}/​lib/​python
 +  ​
 +
 +
 +</​code>​
 +
 +====== Chapitre 6 : SiteDB Check - HN-extraction ======
 +
 +<​code>​
 +https://​twiki.cern.ch/​twiki/​bin/​view/​CMS/​SiteDBForCRAB#​Check_HN_name_extraction
 +</​code>​
 +Check HN name extraction\\
 +\\
 +* In order to check if your HN name can be extracted from SiteDB please download and run this script:\\
 +o check_HN_Name.py\\
 +* usage:\\
 +o "​python check_HN_Name.py"​ will check the extraction in a standalone way\\
 +o "​python check_HN_Name.py crab" will check the extraction using the crab utilities. NOTE: this require the crab environment properly configured. ​
 +====== Chapitre 7 : Getting the results small datasets by "​hand"​ to local UI ======
 +
 +If you still have the crab working directory, this can be done automatically by doing:
 +<​code>​
 +crab -copyData
 +</​code>​
 +or, you can have the srm path by doing
 +<​code>​
 +crab -report
 +</​code>​
 +and then do manually srmcp 
 +====== Chapitre 8 : Erase "​wrong"​ data from SE ======
 +
 +<​code>​
 +edg-gridftp-ls ​  ​(edg-gridftp-rm )
 +gsiftp://​polgrid4.in2p3.fr:​2811/​dpm/​in2p3.fr/​home/​cms/​trivcat/​store/​user/​kurca/​ZZ_4l_10TeV_GEN/​ZZ_4l_10TeV_GEN_HLT/​3ff4a4f43d11ae9733d067a20ed87255
 +</​code>​
 +
 +====== Chapitre 9 : Inspect your data in DBS ======
 +
 +<​code>​
 +InspectDBS2.py --DBSURL=https://​vocms31.cern.ch:​8443/​cms_dbs_ph_analysis_01_writer/​servlet/​DBSServlet --datasetPath=/​ZZ_4l_10TeV_GEN/​kurca-ZZ_4l_10TeV_GEN_HLT-3ff4a4f43d11ae9733d067a20ed87255/​USER/​
 +</​code>​
 +<​code>​
 +https://​cmsweb.cern.ch/​dbs_discovery/​aSearch?​caseSensitive=on&​userMode=expert&​sortOrder=desc&​sortName=&​grid=0&​method=dbsapi&​dbsInst=cms_dbs_ph_analysis_01&​userInput=find+dataset+where+dataset+like+%2F*ZZ_4l_10TeV_GEN%2F*HLT*%2FUSER+
 + 
 +</​code>​
 +
  
  • en/crab_-_user.txt
  • Last modified: 2016/12/16 10:16
  • (external edit)