Differences

This shows you the differences between two versions of the page.

Link to this comparison view

en:crab_server_administration [2016/12/16 10:16] (current)
Line 1: Line 1:
 +Last modified: Apr 15, 2010 by Kurca\\
 +\\
 +
 +====== CRAB Server Administration ======
 +
 +\\
 +\\
 +
 +====== Chapitre 1 : How to put in DRAIN mode a CrabServer ======
 +
 +Change the configuration file $PRODAGENT_WORKDIR/​CrabServerConfig.xml replacing the value "​-1"​ with "​0"​ for the parameter "​acceptableThroughput"​ in the CommandManager configuration block:\\
 +\\
 +<​ConfigBlock Name="​CommandManager">​\\
 +.....\\
 +<​Parameter Name="​acceptableThroughput"​ Value="​0"/>​\\
 +\\
 +Restart the CommandManager component:​\\
 +\\
 +crabServerd --component=CommandManager --sh\\
 +crabServerd --component=CommandManager --start ​
 +====== Chapitre 2 : How to upgrade to higher version ======
 +
 +Even while previous server is still running, within usual crab environment,​ do:\\
 +\\
 +export MYTESTAREA=/​vo/​crab/​sw\\
 +export SCRAM_ARCH=slc4_ia32_gcc345\\
 +\\
 +source $MYTESTAREA/​$SCRAM_ARCH/​external/​apt/​0.5.15lorg3.2-cmp/​etc/​profile.d/​init.sh\\
 +apt-get update\\
 +apt-get upgrade\\
 +apt-cache search crab-server\\
 +apt-get install cms+crab-server+CRABSERVER_1_0_10-cmp\\
 +\\
 +then update the environment settings script changing the CrabServer version:\\
 +\\
 +CRABSERVER_version="​CRABSERVER_1_0_10-cmp"​\\
 +\\
 +Before restarting the server, a couple of changes should be done in the\\
 +$PRODAGENT_WORKDIR/​CrabServerConfig.xml file. In particular the following lines\\
 +should be added in the CrabServerConfigurations block section:\\
 +\\
 +<​Parameter Name="​baseConfUrl"​ Value="​https:​%%//​%%cmsweb.cern.ch/​crabconf/​files/"/>​\\
 +<​Parameter Name="​configFileName"​ Value="​glite_wms_CERN.conf"/> ​
 +====== Chapitre 3 : GgridFtp start ======
 +
 +as user "​root":​
 +<​code>​
 +cd ~crab
 +source GridFTPenv.csh
 +./​globus-gridftp start
 +</​code>​
 +=====  Test GridFtp ​ =====
 +
 +from your UI :
 +<​code>​
 +uberftp ccgridli03.in2p3.fr "ls /​vo/​crab/​storage"​
 +</​code>​
 +
 +====== Chapitre 4 : Parameters Tuning ======
 +
 +28.8.2008
 +<​code>​
 +<​ConfigBlock Name="​JobTracking">​
 +    <​Parameter Name="​QueryInterval"​ Value="​30"/> ​   originally "​3"​
 +</​code>​
 +<​code>​
 +<​ConfigBlock Name="​TaskTracking">​
 +    <​Parameter Name="​PollInterval"​ Value="​30"/> ​ originally was "​4"​
 +
 +</​code>​
 +
 +====== Chapitre 5 : Errors, Messages ======
 +
 +=====  Submitting ​ =====
 +
 +Command Manager has received the tasks, but the wasn't registered by the TaskRegister (known issue, but not solved...).\\
 +Restart TaskRegister component:
 +<​code>​
 +craServerd --component=TaskRegister restart
 +</​code>​
 +=====  NotSubmitted ​ =====
 +
 +Jobs submission problems:
 +<​code>​
 +"​NotSubmitted"  ​
 +--> reason: user proxy problem
 +
 +check:
 +edg-gridftp-ls -v gsiftp://​ccgridli03.in2p3.fr/​
 +  --proxy=/​tmp/​del_proxies/​ab7e74329e030a0c-086da606f0c05edf ...
 +
 +</​code>​
 +<​code>​
 +P.S. I've a script that test the few steps the crab client-server
 +is performing:
 +
 +/​afs/​cern.ch/​user/​a/​afanfani/​public/​CRABSERVER/​testCRABSERVER_delegationGridFTP.sh
 +however it doesn'​t cover the case of using the proxy that is on the
 +server itself (since I don't know how to get that proxy) and that seems
 +the cause of the "​NotSubmitted"​ behaviour.
 +</​code>​
 +furhter upgrade of the lcg-vomscerts RPM:\\
 +iyou can download the RPM where you want, we typically use:\\
 +$MYTESTAREA/​RPMs\\
 +where $MYTESTAREA is the area you specified during the first installation\\
 +( the one specified as path option: DelegationRPMs.sh -path $MYTESTAREA )\\
 +then you can do:\\
 +\\
 +cd $MYTESTAREA/​RPMs\\
 +\\
 +wget -O lcg-vomscerts-5.0.0-1.noarch.rpm http:​%%//​%%glitesoft.cern.ch/​EGEE/​gLite/​R3.1/​generic/​sl4/​i386/​RPMS.updates/​lcg-vomscerts-5.0.0-1.noarch.rpm\\
 +/​opt/​glite/​etc/​init.d/​glite-proxy-renewald stop ;\\
 +/​opt/​asap/​etc/​init.d/​delegation stop;\\
 +rpm -ev lcg-vomscerts-4.9.0-1 ;\\
 +rpm -ivh lcg-vomscerts-5.0.0-1.noarch.rpm\\
 +/​opt/​glite/​etc/​init.d/​glite-proxy-renewald start;\\
 +/​opt/​asap/​etc/​init.d/​delegation start;\\
 +\\
 +\\
 +Just as a doublecheck:​\\
 +1) firewall (*)\\
 +2) /​etc/​grid-security/​grid-mapfile\\
 +/​etc/​grid-security/​groupmapfile\\
 +3) was globus-gridftp started as root fron GridFTPenv.(c)sh environment?​\\
 +4) what /​var/​log/​gridftp-session.log\\
 +and /​var/​log/​globus-gridftp.log\\
 +say?\\
 +\\
 +(*)\\
 +if firewall is active, at least there should be:\\
 +\\
 +iptables -I INPUT -p TCP --dport 20000:25000 -m state --state NEW -j ACCEPT\\
 +iptables -I INPUT -p TCP --dport 2811 -m state --state NEW -j ACCEPT\\
 +iptables -I INPUT -p udp --dport 20000:25000 -j ACCEPT\\
 +\\
 +\\
 +Yes. Restart the server'​s component. Is there a PostMortem file in\\
 +ls -trl $PRODAGENT_WORKDIR/​CrabServerWorker/​PostMortem*\\
 +?\\
 +In principle though there should be a periodic restart if a\\
 +component goes down. 
 +====== Chapitre 6 : CRAB Components ======
 +
 +CRAB Server restart shouldn'​t cause any losses, damages for the running jobs.\\
 +Nevertheless it's better to restart only individual components if needed.
 +
 +====== Chapitre 7 : CPU Sensors ======
 +
 +<​code>​
 +crabServer-SysStat --status
 +  crabServer-SysStat --start
 +</​code>​
 +
 +====== Chapitre 8 : CRAB Configuration ======
 +
 +<​code>​
 +crabServer-edit-config --component=CommandManager --parameter=Port --value=20081
 +</​code>​
 +<​code>​
 +crab: Error during SOAP communication with server ccgridli03.in2p3.fr
 +        .... not open port 20081 for CRAB server communication
 +</​code>​
 +asap-user-register --server ccgridli03.in2p3.fr --myproxy\\
 +crab. proxy successfully registered in ASAP renewal service\\
 +crab. Proxy successfully delegated to the server.\\
 +\\
 +crab. Task name: afanfani_TestAF3_5c4ff395-ee43-45d5-9799-5779ac14895d\\
 +crab. List of ISB files: [u'​TestAF3/​share/​default.tgz',​\\
 +u'​TestAF3/​job/​CMSSW.sh'​]\\
 +crab. Starting sending the project to the storage ccsrm.in2p3.fr...\\
 +crab. Sending default.tgz to ccsrm.in2p3.fr\\
 +crab. Sending CMSSW.sh to ccsrm.in2p3.fr\\
 +crab. Project afanfani_TestAF3_5c4ff395-ee43-45d5-9799-5779ac14895d files\\
 +successfully submitted to the supporting storage element.\\
 +\\
 +\\
 +crab: Error during SOAP communication with server ccgridli03.in2p3.fr.\\
 +----- The server could be under maintainance. -----
 +<​code>​
 +Anyway, I've tried right now and today I get another delegation error (*)
 +that is usually related to voms certificates on the
 +machine (lcg-vomscerts.noarch) as mentioned in:
 + ​https://​twiki.cern.ch/​twiki/​bin/​view/​CMS/​ProxyDelegationSLC4
 +
 +cheers,
 + Ale
 +
 +
 +(*)
 +crab. Registering a valid proxy to the server:
 +  crab. executing:
 +    asap-user-register --server ccgridli03.in2p3.fr --myproxy
 +crab. SOAP 1.1 fault: SOAP-ENV:​Server [no subcode]
 +"​getProxyReq"​
 +Detail: Not a member of the CMS VO
 +</​code>​
 +
 +====== Chapitre 9 : CRAB Commands ======
 +
 +<​code>​
 +crabServerd --status
 +crabServerd --sh(utdown)
 +crabServerd --start
 +
 +crabServerd --component HTTPFrontend --restart
 +
 +crabServer-viewer --lsTasks --max 100
 +prodAgent-viewer --lsMsg --max 100
 +
 +</​code>​
 +Well, after the CrabServerWorker component restart still submission of yesterday'​s\\
 +jobs doesn'​t work.\\
 +It stucks in "​Arrived task" ... as before\\
 +\\
 +tail -f loggingTaskInfo\\
 +\\
 +%%**%%*Tue Aug 5 14:38:49 2008%%**%%*\\
 +[CRAB_Cmd_Mgr:​NewTask]:​ Arrived task:\\
 +ndefilip_ZBB-COMPHEP-Mis100pb_HLT_87ed8a09-693c-4df3-a1d7-6bfcd432642a\\
 +\\
 +%%**%%*Tue Aug 5 14:38:49 2008%%**%%*\\
 +[CRAB_Cmd_Mgr:​NewCommand]:​ New command:\\
 +ndefilip_ZBB-COMPHEP-Mis100pb_HLT_87ed8a09-693c-4df3-a1d7-6bfcd432642a\\
 +count: 5\\
 +This means that the server receives the submission commands on is\\
 +soap/​channel (CommandManager component), but the CrabServerWorker\\
 +component seems to be unable to catch the message...\\
 +\\
 +In the CommandManager component there are two logs. I'd like to see\\
 +what the FrontendLog file says..\\
 +\\
 +Also, can you provide with the result of those two commands:\\
 +crabServer-viewer --lsTasks --max 100\\
 +prodAgent-viewer --lsMsg --max 100
 +<​code>​
 +Checks related to Storage:
 +
 +1. Does this SE storage path exist ?
 +$lcg-ls -b -D srmv2
 + ​srm://​ccsrm.in2p3.fr:​8443/​srm/​managerv2?​SFN=/​pnfs/​in2p3.fr/​data/​cms/​prod/​store
 +
 +</​code>​
 +<​code>​
 +Checks related to Proxy renewal resp. Delegation Service:
 +1. Is proxyrenewal (delegation) service running?
 +/​opt/​glite/​etc/​init.d/​glite-proxy-renewald status
 +/​opt/​asap/​etc/​init.d/​delegation status
 +
 +asap-user-register --server ccgridli03.in2p3.fr --myproxy
 +
 +2. host certificate registered at CERN
 +"/​O=GRID-FR/​C=FR/​O=CNRS/​OU=CC-LYON/​CN=ccgridli03.in2p3.fr"​
 +</​code>​
 +
 +====== Chapitre 10 : MySQL ======
 +
 +<​code>​
 +mysql -u root -p --socket=$PRODAGENT_WORKDIR/​mysqldata/​mysql.sock CrabServerDB
 +password:
 +</​code>​
 +<​code>​
 +select * from ms_message where payload like "​mcinquil%";​
 +select * from we_Job where owner like "​mcinquil%";​
 +show tables;
 +
 +</​code>​
 +=====  Problems with MySQL  =====
 +
 +Characteristics:​\\
 +jobs "​arrived"​ but not submitted\\
 +jobs "​submitted"​ but not proceeding to the next step\\
 +\\
 +Reason:\\
 +too many open connections to the DB\\
 +\\
 +Cure:\\
 +stop TaskLifeManager
 +<​code>​
 +Hi Mattia,
 +you are right,
 +at least I found there an empty (corrupted) file
 +"​queueSerializationFile"​ with last "​update"​ from the day when disk was full.
 +Removing files helped at least for the TaskTracking, ​ Componentlog file
 +has no more the error message as before.
 +
 +  But probably the tasks already submitted are lost for the database?
 +  I.e. only new submissions should be seen as expected?
 +I think yes..but there is a way to recover it (if it is not registered wth the
 +crabServer-viewer utility):
 +
 +python $PRODAGENT_ROOT/​util/​publish.py CRAB_Cmd_Mgr:​NewTask TASKNAME
 +
 +where TASKNAME is the Nicola'​s task name on server side.
 +(eg: ndefilip_TEST_B_ZBB-COMPHEP-Mis100pb_543072e2-cf1f-45c2-a574-eb8809544834)
 +
 +</​code>​
 +<​code>​
 +crabServerd --sh
 +$PRODAGENT_WORKDIR/​CrabServerConfig.xml
 +<​!--<​Component Name="​TaskLifeManager"/>​-->​
 +crabServerd --start
 +</​code>​
 +Ok..this could be the point: the DB is overloaded with too many connections (for the next\\
 +release this is going to be fixed).\\
 +For the moment you can stop the TaskLifeManager component (it is not vital for the\\
 +task/job workflow) and it makes many queries to the db. You can follow the steps below:\\
 +* shut down the server\\
 +* to prevent the start of the TaskLifeManager component comment the related line from the\\
 +CrabServerConfig.xml (like this: <​!--<​Component Name="​TaskLifeManager"/>​-->​)\\
 +* restart the server\\
 +\\
 +The shut-down of the TLM component will stop the sand-boxes cleaning (to rehabilitate it\\
 +is enough to restart the component for a while, removing the commented line). ​
 +====== Chapitre 11 : MySQL start ======
 +
 +<​code>​
 +mysqld_safe --datadir=${PRODAGENT_WORKDIR}/​mysqldata --socket=${PRODAGENT_WORKDIR}/​mysqldata/​mysql.sock --skip-networking ​
 +--log-error=${PRODAGENT_WORKDIR}/​mysqldata/​error.log --pid-file=${PRODAGENT_WORKDIR}/​mysqldata/​mysqld.pid &
 +</​code>​
 +In case you want to use a specific MySQL configuration you should use the --defaults-file option:
 +<​code>​
 +mysqld_safe --defaults-file=${PRODAGENT_WORKDIR}/​mysqldata/​my.cnf --socket=${PRODAGENT_WORKDIR}/​mysqldata/​mysql.sock --skip-networking --log-error=${PRODAGENT_WORKDIR}/​mysqldata/​error.log --pid-file=${PRODAGENT_WORKDIR}/​mysqldata/​mysqld.pid &
 +</​code>​
 +
 +====== Chapitre 12 : MySQL - direct check ======
 +
 +<​code>​
 +mysql -u root -p --socket=$PRODAGENT_WORKDIR/​mysqldata/​mysql.sock CrabServerDB
 +</​code>​
 +<​code>​
 +show databases;
 +use CrabServerDB;​
 +show tables;
 +describe we_Job;
 +select global_sanbox from bl_task where name like '​ranjan%';​
 +</​code>​
 +
 +====== Chapitre 13 : MySQL - modification ======
 +
 +<​code>​
 +mysql> update user set password=password(''​) where user="​root"​ and host="​localhost";​
 +
 +</​code>​
 +<​code>​
 +mysqladmin -u root  -p password ''​ --socket=$PRODAGENT_WORKDIR/​mysqldata/​mysql.sock ​
 +
 +</​code>​
 +
 +====== Chapitre 14 : Proxy Renewal and Delegation Service ======
 +
 +as a user "​root"​ !!!!
 +<​code>​
 +/​opt/​glite/​etc/​init.d/​glite-proxy-renewald start  (stop)
 +
 +/​opt/​asap/​etc/​init.d/​delegation start (stop)
 +
 +</​code>​
 +=====  vomscert upgrade ​ =====
 +
 +<​code>​
 +cd  $MYTESTAREA/​RPMs
 +wget -O lcg-vomscerts-5.1.0-1.noarch.rpm http://​glitesoft.cern.ch/​EGEE/​gLite/​R3.1/​generic/​sl4/​i386/​RPMS.updates/​lcg-vomscerts-5.1.0-1.noarch.rpm
 +/​opt/​glite/​etc/​init.d/​glite-proxy-renewald stop ;
 +/​opt/​asap/​etc/​init.d/​delegation stop;
 +rpm -ev lcg-vomscerts-5.0.0-1 ;
 +rpm -ivh lcg-vomscerts-5.1.0-1.noarch.rpm
 +/​opt/​glite/​etc/​init.d/​glite-proxy-renewald start;
 +/​opt/​asap/​etc/​init.d/​delegation start;
 +
 +</​code>​
 +
 +====== Chapitre 15 : GridFTP ======
 +
 +as a "​root"​\\
 +get the proper environment and\\
 +Start GridFTP server daemon through the script
 +<​code>​
 +cd ~crab
 + ​source GridFTPenv.c|sh
 + ​./​globus-gridftp start
 +</​code>​
 +Test if gridftp is working correctly\\
 +\\
 +1.\\
 +script refers to a proxy in /​tmp/​del_proxies
 +<​code>​
 +cd ~crab
 +python testGridFTPSE_Lyon.py
 +
 +
 +</​code>​
 +2. from any machine
 +<​code>​
 +voms-proxy-init --voms cms
 +edg-gridftp-ls -v gsiftp://​ccgridli03.in2p3.fr/​
 +</​code>​
 +CRAB servers in Legnaro resp. Bari
 +<​code>​
 +edg-gridftp-ls -v gsiftp://​crabas.lnl.infn.it/​data/​
 +edg-gridftp-ls -v gsiftp://​dot1-prod-2.ba.infn.it/​data
 +</​code>​
 +
 +====== Chapitre 16 : Using CRAB ======
 +
 +=====  Setup your environment ​ =====
 +
 +<​code>​
 +1) Grid UI
 +  source /​afs/​cern.ch/​cms/​LCG/​LCG-2/​UI/​cms_ui_env.csh source
 +2) CMSSW environment
 +  cd into the CMSSW release directory
 +  eval `scramv1 runtime -csh`
 +3) CRAB environment
 +  source /​afs/​cern.ch/​cms/​ccs/​wm/​scripts/​Crab/​crab.csh
 +</​code>​
 +
 +====== Chapitre 17 : SiteDB Check - HN-extraction ======
 +
 +<​code>​
 +https://​twiki.cern.ch/​twiki/​bin/​view/​CMS/​SiteDBForCRAB#​Check_HN_name_extraction
 +</​code>​
 +Check HN name extraction\\
 +\\
 +* In order to check if your HN name can be extracted from SiteDB please download and run this script:\\
 +o check_HN_Name.py\\
 +* usage:\\
 +o "​python check_HN_Name.py"​ will check the extraction in a standalone way\\
 +o "​python check_HN_Name.py crab" will check the extraction using the crab utilities. NOTE: this require the crab environment properly configured.
 +<​code>​
 +class check_HN_name:​
 +    def init(self): ​
 +        pass
 +    ​
 +    def stdaloneCheck(self):  ​
 + 
 +        import urllib
 +        from commands import getstatusoutput
 +        print 'start standalone check ...'
 +        status, dn = getstatusoutput('​voms-proxy-info -identity'​)
 +        if status == 0:
 +           print "my DN is: %s "%dn
 +        dn = dn.split('​
 +')[-1]
 +        dn  = urllib.urlencode({'​dn':​dn})
 +        print 'Using urlencoded DN: 
 +  %s '%dn
 +        f = urllib.urlopen("​https://​cmsweb.cern.ch/​sitedb/​json/​index/​dnUserName?​%s"​ % dn)
 +        print 'my HN user name is: %s 
 +'​%str(f.read())
 +        f.close()
 +        print '
 +end check.....................'​
 +
 +    def crabCheck(self):​
 +        from crab_logger import Logger
 +        from WorkSpace import *
 +        import tempfile, urllib, os 
 +      ​
 +        dname = tempfile.mkdtemp( "",​ "​crab_",​ '/​tmp'​ )
 +        os.system("​mkdir %s/​log"​%dname )
 +        os.system("​touch %s/​crab.log"​%dname )
 +        ​
 +        cfg_params={'​USER.logdir'​ : dname }
 +        common.work_space = WorkSpace(dname,​ cfg_params)
 +        log = Logger()
 +        common.logger = log
 +        ​
 +        from crab_util import getDN,​gethnUserNameFromSiteDB
 +        print 'start using CRAB utils ...'
 +        print "my DN is: %s 
 +"​%getDN()
 +        try:
 +            print 'my HN user name is: %s 
 +'​%gethnUserNameFromSiteDB()
 +        except:
 +            print '
 + ​WARNING native crab_utils failed! ' ​
 +            dn=urllib.urlencode({'​dn':​getDN()})
 +            print '​trying now using urlencoded DN: 
 +%s '%dn
 +            status,​hnName = self.gethnName_urlenc(dn)
 +            if status == 1: 
 +                print '
 + ​WARNING:​ failed also using urlencoded DN '
 +            else: 
 +                print 'my HN user name is: %s 
 +'%name
 +                print '​problems with crab_utils' ​  
 +        print '
 + end check.....................'​
 +        ​
 +        os.system("​rm -rf %s"​%dname )
 +         
 +    def gethnName_urlenc(self,​dn):​
 +        from WMCore.Services.SiteDB.SiteDB import SiteDBJSON
 +        hnUserName = None
 +        userdn = dn
 +        mySiteDB = SiteDBJSON()
 +        status = 0 
 +        try:
 +            hnUserName = mySiteDB.dnUserName(dn=userdn)
 +        except:
 +            status = 1 
 +        return status,​hnUserName
 +
 +
 +if __name__ == '​__main__'​ :
 +    import sys
 +    args = sys.argv[1:​]
 +    check = check_HN_name() ​
 +    if '​crab'​ in args:
 +        check.crabCheck()  ​
 +    else:
 +        check.stdaloneCheck()  ​
 +
 +</​code>​
 +
 +====== Chapitre 18 : Monitoring ======
 +
 +Something wrong with HW sensors!\\
 +Attention on the LANG variable !
 +<​code>​
 +export LANG=en_US
 +and not:
 +export LANG=fr_FR
 +</​code>​
 +crabServer-SysStat --status --resources all\\
 +\\
 +tail $PRODAGENT_WORKDIR/​HTTPFrontend/​sensors/​{MEM,​CPU,​SWAP}-pidstat.dat\\
 +\\
 +sar -r 1 2\\
 +sar -W 1 2\\
 +sar -u 1 2
 +<​code>​
 +1) stop all services sensors:
 +
 +crabServer-SysStat --sh --resources all
 +
 +2) clear services sensor dat files:
 +
 +rm -f $PRODAGENT_WORKDIR/​HTTPFrontend/​sensors/​{MEM,​CPU,​SWAP}-pidstat.dat
 +
 +3) export the en_US language env var:
 +
 +export LANG=en_US
 +
 +4) restart all services sensors
 +
 +crabServer-SysStat --start --resources all
 +
 +</​code>​
 +tail -1 $PRODAGENT_WORKDIR/​HTTPFrontend/​sensors/​*-pidstat.dat\\
 +So the rest is here:\\
 +[crab@ccgridli03 sensors]$ tail -1\\
 +$PRODAGENT_WORKDIR/​HTTPFrontend/​sensors/​*-pidstat.dat\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​AdminControl-pidstat.dat <==\\
 +03/​10/​09#​09:​19:​36 1236673176 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​CommandManager-pidstat.dat <==\\
 +03/​10/​09#​09:​18:​45 1236673125 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​CPU-pidstat.dat <==\\
 +03/​13/​09#​15:​12:​44 1236953564 5.73 0.00 0.78 1.61\\
 +91.89\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​CrabServerWorker-pidstat.dat <==\\
 +03/​10/​09#​09:​19:​40 1236673180 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​ErrorHandler-pidstat.dat <==\\
 +03/​10/​09#​09:​19:​30 1236673170 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​GetOutput-pidstat.dat <==\\
 +03/​10/​09#​09:​18:​45 1236673125 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​GridFTP-pidstat.dat <==\\
 +03/​10/​09#​09:​19:​25 1236673165 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​hda-pidstat.dat <==\\
 +03/​13/​09#​15:​12:​15 1236953535 0.00 0.00 0 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​HTTPFrontend-pidstat.dat <==\\
 +03/​10/​09#​09:​19:​38 1236673178 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​JobKiller-pidstat.dat <==\\
 +03/​10/​09#​09:​19:​34 1236673174 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​JobTracking-pidstat.dat <==\\
 +03/​10/​09#​09:​18:​55 1236673135 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​LOAD-pidstat.dat <==\\
 +03/​13/​09#​15:​12:​41 1236953561 0.62 0.64 0.63\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​MEM-pidstat.dat <==\\
 +03/​13/​09#​15:​12:​38 1236953558 299180 5827720 774464 0.00\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​mySQL-pidstat.dat <==\\
 +03/​10/​09#​09:​19:​27 1236673167 6.10 0.00 6.1\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​Notification-pidstat.dat <==\\
 +03/​10/​09#​09:​19:​41 1236673181 0.07 0.00 0.07\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​sda-pidstat.dat <==\\
 +03/​13/​09#​15:​13:​12 1236953592 0.00 433.20 0 25992\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​SWAP-pidstat.dat <==\\
 +03/​13/​09#​15:​12:​47 1236953567 0.00 0.00\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​TaskLifeManager-pidstat.dat <==\\
 +03/​10/​09#​09:​18:​46 1236673126 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​TaskRegister-pidstat.dat <==\\
 +03/​10/​09#​09:​18:​43 1236673123 0.00 0.00 0\\
 +\\
 +==> /​vo/​crab/​work/​HTTPFrontend/​sensors/​TaskTracking-pidstat.dat <==\\
 +03/​10/​09#​09:​19:​20 1236673160 0.48 0.00 0.48\\
 +[crab@ccgridli03 sensors]$
 +
  
  • en/crab_server_administration.txt
  • Last modified: 2016/12/16 10:16
  • (external edit)