Differences

This shows you the differences between two versions of the page.

Link to this comparison view

en:different_issues [2016/12/05 14:43] (current)
Line 1: Line 1:
 +Last modified: Jul 16, 2009 by Kurca\\
 +\\
 +
 +====== Different Issues ======
 +
 +\\
 +\\
 +
 +====== Chapitre 1 :  dCache ======
 +
 +dc_open(dcap:​%%//​%%ccdcatsn004/​pnfs/​in2p3.fr/​data/​cms/​data/​store/​mc/​2007/​5/​25/​Spring07-W0j-1550/​0005/​003A7CE7-260D-DC11-A1B9-0015C5E5B9C5.root,​0,​438)\\
 +failed DCache error 26: Server rejected "​hello"​\\
 +\\
 +> La connexion a été refusée car la limite du nombre maximum de\\
 +> connexions simultanées a été atteinte, je l'ai augmentée un peu.
 +
 +
 +====== Chapitre 2 :  TURL ======
 +
 +<​code>​
 +root://​ccxroot.in2p3.fr:​1094/​
 +</​code>​
 +<​code>​
 +dcap://​ccdcatsn004/​
 +</​code>​
 +<​code>​
 +srm://​ccsrm.in2p3.fr:​8443/​srm/​managerv1?​SFN=
 +</​code>​
 +<​code>​
 +VO_Path:​   :​pnfs/​in2p3.fr/​data/​cms
 +</​code>​
 +srmcp fi [[file:///​sps/​|le:​%%//​%%/​sps/​]] ...        srm:​%%//​%%...cern.ch:​8443/​castor/​...
 +
 +globus-url-copy -p 10 -tcp-bs 4194304 -bs 4194304 -vb file:///​sps/​cms/​cms02/​\\
 +cmsprod/​zp1000-0.root gsiftp:​%%//​%%castorgrid.cern.ch/​castor/​cern.ch/​cms/​generation/​madgraph/​zp1000-0.root
 +
 +
 +====== Chapitre 3 :  T2 vs T1 ======
 +
 +T2 currently "​virtual"​ tier behind T1_IN2P3_Buffer
 +
 +- all requests for T3_IN2P3_IPNL data transfers have to be approved by T1 data manager
 +
 +--> fill site configuration for T2 CC-IN2P3 AF i  SiteDB in particular with "​Storage Element"​ and "​Phedex configuration"​    --> make it clear that SE is shared and this should allow me to subscribe data directly?
 +
 +<​code>​
 +improvements,​ upgrades ?  - separate PhEDEx node without triggering any real replication
 +
 +possible by running "​customized"​ agents for T2 part   --> T2 agents just look at what is already available on dCache before triggering any transfer  (similar as T1_IN2P3-Buffer -->MSS)
 +
 +feasibility ?
 +
 +time to make it operationel ? (people used to T1== T2)
 +</​code>​
 +
 +
 +====== Chapitre 4 :  dCacheSpaceChecker ======
 +
 +<​code>​
 +ccali11:​tcsh[208] /​usr/​local/​products/​dcache/​scripts/​dCacheSpaceChecker.pl --vo cms
 +</​code>​
 +<​code>​
 +INFO : Numbers are in GigaBytes
 +INFO : Here is a listing of configured endpoints for cms:
 +* /​pnfs/​in2p3.fr/​data/​cms (Global space allocated)
 +        Quota     : 263734
 +        Used      : 36889
 +        Available : 226844
 +                    86.0127875186348319%
 +* Import (Import)
 +        Quota     : 900
 +        Used      : 245
 +        Available : 654
 +                    72.6945667322725058%
 +* Prod (Endpoint for production outputs)
 +        Quota     : 28068
 +        Used      : 15719
 +        Available : 12348
 +                    43.9958023900680857%
 +* Analysis (Workers access)
 +        Quota     : 206352
 +        Used      : 3248
 +        Available : 203103
 +                    98.4257782541763223%
 +</​code>​
 +
 +====== Chapitre 5 :  dCache :   cacheinfo.sh ======
 +
 +from PNFS server ccdcamcli07
 +
 +<​code>​
 + /​var/​spool/​dcache/​tools/​public/​cacheinfo.sh ​ <full path file name>
 +</​code>​
 +<​code>​
 +/​pnfs/​in2p3.fr/​data/​cms/​data/​store/​data/​Commissioning08/​Cosmics/​RECO
 +</​code>​
 +<​code>​
 +/​CRAFT_ALL_V9_225-v2/​0009/​34AB6238-1AFA-DD11-992C-001A92971B28.root 
 +</​code>​
 +
 +====== Chapitre 6 :  File Status   pnfsidinfo ======
 +
 +pnfsid itself one can get from cacheinfo e.g.
 +
 +<​code>​
 + ​ http://​cctoolsafs.in2p3.fr/​dcache/​monitoring/​lcg/​pnfsidinfo.php
 +</​code>​
 +
 +====== Chapitre 7 :  SRM ======
 +
 +<​code>​
 +ccali38:​tcsh[264] srmls srm://​ccsrm.in2p3.fr:​8443/​pnfs/​in2p3.fr/​data/​cms/​data/​store/​mc
 +WARNING: SRM_PATH is defined, which might cause a wrong version of srm client to be executed
 +WARNING: SRM_PATH=/​usr/​local/​grid/​glite/​3.1.5-0/​UI/​d-cache/​srm
 +  0 /​pnfs/​in2p3.fr/​data/​cms/​data/​store/​mc/​
 +      512 /​pnfs/​in2p3.fr/​data/​cms/​data/​store/​mc/​2006/​
 +      512 /​pnfs/​in2p3.fr/​data/​cms/​data/​store/​mc/​2007/​
 +      512 /​pnfs/​in2p3.fr/​data/​cms/​data/​store/​mc/​2008/​
 +      512 /​pnfs/​in2p3.fr/​data/​cms/​data/​store/​mc/​PreCSA08/​
 +      512 /​pnfs/​in2p3.fr/​data/​cms/​data/​store/​mc/​CSA08/​
 +</​code>​
 +<​code>​
 + lcg-cp srm://​gridka-dCache.fzk.de:​8443/​srm/​managerv2?​SFN=pnfs/​..... ​    ​srm://​srm-cms.cern.ch:​8443/​castor/​cern/​user/​.....
 +</​code>​
 +<​code>​
 +lcg-cp srm://​ccsrm.in2p3.fr:​8443/​pnfs/​in2p3.fr/​data/​cms/​data/​store/​data/​Commissioning08/​
 +</​code>​
 +<​code>​
 +Cosmics/​RECO/​CRAFT_ALL_V9_225-v2/​0009/​34AB6238-1AFA-DD11-992C-001A92971B28.root file:///​sps/​cms/​kurca/​local.test
 +</​code>​
 +<​code>​
 +srmcp srm://​ccsrm.in2p3.fr:​8443/​pnfs/​in2p3.fr/​data.... ​   file:////​sps/​cms/​kurca/​local.test ​  ​(local.test ...is an existing directory !!!)
 +</​code>​
 +<​code>​
 +srmcp -srm_protocol_version=2 -pushmode=true -debug=true ​  ​srm://​... ​  ​srm://​...
 +</​code>​
 +<​code>​
 +srmping -debug -2 srm://​gridka-dCache.fzk.de:​8443/​srm/​managerv2
 +</​code>​
 +<​code>​
 + srmping -debug -2 srm://​ccsrm.in2p3.fr:​8443/​srm/​managerv1(2)
 +</​code>​
 +
 +====== Chapitre 8 :  dccp ======
 +
 +<​code>​
 +dccp dcap://​ccdcapcms/​pnfs/​in2p3.fr/​data/​cms/​data/​store/​data/​Commissioning08/​
 +</​code>​
 +<​code>​
 +Cosmics/​RECO/​CRAFT_ALL_V9_225-v2/​0009/​34AB6238-1AFA-DD11-992C-001A92971B28.root ​ /​sps/​cms/​kurca
 +</​code>​
 +<​code>​
 +file:////​sps/​cms/​kurca ​ .... doesn'​t work but ................................ in fact this command can be used to test accessibility of data ... or pool 
 +</​code>​
 +
 +====== Chapitre 9 :  Get remote end point to stage out file from CRAB ? ======
 +
 +( curl -H '​Accept:​ text/​xml'​\\
 +" http://​t2.unl.edu/​phedex/​tfc/​map/​$node?​lfn=$lfn&​protocol=$protocol ")
 +
 +curl -H '​Accept:​ text/​xml'​ " [[http://​t2.unl.edu/​phedex/​tfc/​map/​$node?​lfn=$lfn&​protocol=$protocol]] T1_FR_CCIN2P3_MSS?​lfn=/​store/​user/​afanfani&​protocol=srmv2"​
 +
 +srm:​%%//​%%ccsrm.in2p3.fr:​8443/​srm/​managerv1?​SFN=/​pnfs/​in2p3.fr/​data/​cms/​data/​store/​user/​afanfani
 +
 +>  I apologize...I was aware of the limitation of the port 8443 in CRAB\\
 +>  and also that some endpoints were not compliant,​\\
 +>  but in the various attempts to stage out I did yesterday\\
 +>  I forgot to check that UCL fall in the category.\\
 +>\\
 +>  The solution is to have some SiteDB or wathever API\\
 +>  that provide the full srm endpoints to CRAB. That's already in\\
 +>  the plan. Brian'​s tools that query TMDB were not allowed\\
 +>  to be used by CRAB.\\
 +I would not cache pfn information in siteDB for configuring CRAB\\
 +remote stage out. The pfns can dynamically change and siteDB would\\
 +become obsolete quite fast.\\
 +Currently the only reliable source for the conversion LFN->PFN at\\
 +every site, which is dynamically updated, is TMDB (which is caching\\
 +the TFC for all sites, exported by the export agent). The conversion\\
 +can be accessed using Brian'​s interface. I think the plan is to move\\
 +that service to some central maintained machine and make some changes\\
 +so that it can support a high query rate. I think the idea would be to\\
 +let CRAB at job preparation time talk to that service to configure the\\
 +remote stage out PFN.\\
 +In the meantime, given that one in has to manually configure the\\
 +remote stage out pfn in CRAB I would suggest to use Brian'​s web\\
 +interface ( curl -H '​Accept:​ text/​xml'​\\
 +" http://​t2.unl.edu/​phedex/​tfc/​map/​$node?​lfn=$lfn&​protocol=$protocol ")\\
 +when preparing the iCSA08/CCRC jobs.\\
 +E.g.,  curl -H '​Accept:​ text/​xml'"​ http://​t2.unl.edu/​phedex/​tfc/​map/​T2_BE_UCL?​lfn=/​store/​user/​afanfani&​protocol=srmv2\\
 +returns\\
 +srm:​%%//​%%ingrid-se02.cism.ucl.ac.be:​8444/​srm/​managerv2?​SFN=/​storage/​data/​cms/​store/​user/​afanfani
 +
 +So, to save time and avoid any confusion, I would suggest to manually\\
 +use Brian'​s web interface to TMDB to configure the remote stage out in\\
 +CRAB cfg file. Once a central and robust service is available, CRAB\\
 +can have an API to automatically configure the remote pfn at job\\
 +preparation time.
 +
 +Cheers,\\
 +Jose\\
 +\\
 +
 +
 +
  
  • en/different_issues.txt
  • Last modified: 2016/12/05 14:43
  • (external edit)