Differences

This shows you the differences between two versions of the page.

Link to this comparison view

en:dtm_configuration_and_customization [2016/12/16 10:16] (current)
Line 1: Line 1:
 +Last modified: Jan 18, 2013 by Cardenas\\
 +\\
 +
 +====== DTM Configuration and Customization ======
 +
 +\\
 +\\
 +
 +=====  Introduction ​ =====
 +
 +DTM can be configured using customized shell scripts (called plugins). In this way, specific characteristics of a technology or installation can be set by user. It permits add and customize new technologies or instances.\\
 +\\
 +For a general description of DTM see: [[:​en:​distributed_task_manager|Distributed Task Manager]]\\
 +\\
 +This document describes the configuration options of the file **"​config/​configuration"​**\\
 +\\
 +Configuration file has several sections:\\
 +[[#​DTM_Manager_configuration|DTM Manager configuration]]\\
 +[[#​TORQUE_Configuration|TORQUE Configuration]]\\
 +[[#​SGE_Configuration|SGE Configuration]]\\
 +[[#​GRID_Configuration|GRID Configuration]]\\
 +[[#​LOCAL_HOST_Configuration|LOCAL HOST Configuration]]
 +=====  DTM Manager configuration ​ =====
 +
 +It is necessary to indicate the plugin to submit and stop the DTM manager job or process.\\
 +\\
 +\\
 +MANAGER_START_COMMANDS_PATH\\
 +\\
 +Indicates the full path to the shell script used by dtm-start command to submit a DTM manager. It permits to set parameters and options to the job or process that execute the DTM manager. For example, use a special demon queue for long jobs.\\
 +\\
 +This plugin must establish if there is a manager running, if not it submits a DTM manager.\\
 +\\
 +The job-name nomenclature is used to identify the name of manager as indicated on dtm-start or dtm-task-add commands.\\
 +\\
 +MANAGER_STOP_COMMANDS_PATH\\
 +\\
 +Indicates the full path to the shell script used by dtm-cancel and dtm-stop commands to delete a DTM manager.\\
 +\\
 +The job-name nomenclature is used to identify the name of manager as indicated on dtm-start or dtm-task-add commands.
 +=====  TORQUE configuration ​ =====
 +
 +TORQUE_USE\\
 +Set "​YES"​ to enable or "​NOT"​ to disable the utilization of the TORQUE batch system. DTM manager job or process can submit jobs (DTM agents). User must be able to submit jobs to this system.\\
 +\\
 +TORQUE_COMMANDS_PATH\\
 +set path to batch commands (qsub, qdel, ...)\\
 +set ""​ (void quotes) to use default system PATH.\\
 +\\
 +TORQUE_STOP_COMMANDS_PATH\\
 +set path to batch commands (eg. "​qdel",​ "​qdel-torque"​).\\
 +set ""​ (void quotes) to use default system PATH.\\
 +\\
 +TORQUE_QSUB_COMMAND\\
 +set name of qsub command (eg. "​qsub",​ "​qsub-torque"​ )\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +TORQUE_QUEUE_NAME\\
 +set name of default queue used to DTM agents (eg. "​long",​ "​1hour",​...)\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +TORQUE_BATCH_OPTIONS\\
 +set default additional batch system options ( eg. "-p 100 -j oe" )\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +TORQUE_OUTPUT\\
 +set pathfilename to DTM agent stdout\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +TORQUE_DEBUG="​NOT"​\\
 +set "​YES"​ to enable or "​NOT"​ to disable expanding all scripts commands
 +=====  SGE configuration ​ =====
 +
 +SGE_USE\\
 +set "​YES"​ to enable or "​NOT"​ to disable the utilization of the SGE batch system. DTM manager job or process can submit jobs (DTM agents). User must be able to submit jobs to this system.\\
 +\\
 +SGE_COMMANDS_PATH\\
 +set path to batch commands (qsub, qdel, ...)\\
 +set ""​ (void quotes) to use default system PATH.\\
 +\\
 +SGE_STOP_COMMANDS_PATH\\
 +set path to batch commands (eg. "​qdel"​).\\
 +set ""​ (void quotes) to use default system PATH.\\
 +\\
 +SGE_QSUB_COMMAND\\
 +set name of qsub command (eg. "​qsub"​ )\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +SGE_QUEUE_NAME\\
 +set name of default queue used to DTM agents (eg. "​long",​ "​medium",​...)\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +SGE_BATCH_OPTIONS\\
 +set default additional batch system options ( eg. "sps=1 hpss=1"​ )\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +SGE_OUTPUT\\
 +set pathfilename to DTM agent stdout\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +SGE_DEBUG\\
 +Set "​YES"​ to enable or "​NOT"​ to disable expanding all scripts commands
 +=====  GRID configuration ​ =====
 +
 +Require an available gLite UI (User Interface). User must be able to submit jobs to grid, it must have a grid certificate and to be VO active member.\\
 +\\
 +GRID_USE\\
 +set "​YES"​ to enable or "​NOT"​ to disable the utilization of the grid. DTM manager job or process can submit jobs (DTM agents).\\
 +\\
 +GRID_COMMANDS_PATH\\
 +set path to CREAMCE glite commands (glite-ce-job-submit,​glite-ce-job-cancel ...)\\
 +set ""​ (void quotes) to use default UI system configuration.\\
 +\\
 +GRID_STOP_COMMANDS_PATH\\
 +set path to grid cancel commands (eg. "​glite-ce-job-cancel"​).\\
 +set ""​ (void quotes) to use default UI system configuration.\\
 +\\
 +GRID_QSUB_COMMAND\\
 +set name of grid job submit command (eg."​glite-ce-job-submit"​)\\
 +set ""​ (void quotes) to use default UI system configuration.\\
 +\\
 +GRID_QUEUE_NAME\\
 +set name of default grid queue (eg. "​long",​ "​short",​...)\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +GRID_BATCH_OPTIONS\\
 +set default additional submit grid options ( eg. "-D <​ID>"​ )\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +GRID_OUTPUT\\
 +set pathfilename to DTM agent stdout\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +GRID_VO_LIST **((obligatory)**\\
 +set list of VO to use (eg. "​vo.rhone-alpes.idgrilles.fr vo.france-asia.org"​\\
 +\\
 +GRID_DIR_USER_PROXY **(obligatory)**\\
 +set pathfile to proxy grid certificate to create (eg. $HOME/​.dtm-grid)\\
 +Note: use place in a shared file system\\
 +\\
 +GRID_FILTER_LIST\\
 +set list of query filters to grid information system. It permits to select a subset of creamce services. Example, for select the creamce at CC-IN2P3 "​cccreamceli"​\\
 +\\
 +GRID_DEBUG="​NOT"​\\
 +set "​YES"​ to enable or "​NOT"​ to disable expanding all scripts commands
 +=====  LOCAL HOST Configuration ​ =====
 +
 +Local section permits to submit DTM agents in local host, for example an Unix/Linux standalone computer. It is possible to use the "​at"​ or "​batch"​ command to submit jobs, other implementations can be integrated.\\
 +\\
 +\\
 +LOCAL_USE\\
 +set "​YES"​ to enable or "​NOT"​ to disable the utilization of local host.\\
 +\\
 +LOCAL_STOP_COMMANDS_PATH\\
 +set path to shell script used to terminate a DTM process at local host.\\
 +\\
 +LOCAL_COMMANDS_PATH\\
 +set path to commands or shell scripts to submit a DTM process at local host.\\
 +\\
 +LOCAL_QSUB_COMMAND\\
 +set name of command to submit a DTM agent (eg."​at"​)\\
 +\\
 +LOCAL_MAX_AGENTS\\
 +set the number of agents in a local host\\
 +\\
 +LOCAL_BATCH_OPTIONS\\
 +set additional arguments to submit command\\
 +\\
 +LOCAL_OUTPUT\\
 +set pathfilename to DTM agent stdout\\
 +set ""​ (void quotes) to use default system.\\
 +\\
 +LOCAL_DEBUG\\
 +set "​YES"​ to enable or "​NOT"​ to disable expanding all scripts commands
 +
  
  • en/dtm_configuration_and_customization.txt
  • Last modified: 2016/12/16 10:16
  • (external edit)