TORQUE管理员指南：集群作业管理详解

需积分: 10 56 浏览量更新于2024-07-22 收藏 1013KB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

"《TORQUE管理员指南》(version 3.0.2)是一份详尽的文档，介绍了Torque集群作业管理系统，它在IT领域中被广泛用于高效管理和调度分布式计算任务。该指南覆盖了从安装、配置到高级功能的全面指导，旨在帮助管理员有效地管理服务器节点、作业提交与监控、节点管理、队列设置、数据管理和与其他系统的集成。 **1. 安装与初始化** 这部分包括了对Torque系统的基本安装过程，以及如何在服务器（pbs_server）上初始化和配置系统，确保所有组件能够协同工作。管理员需掌握如何进行手动配置，特别是对于NUMA系统和多MOM（Multiple Object Manager）环境的支持。 **2. 作业提交与管理** 这部分深入解析了如何提交作业，包括如何监控作业状态，何时取消作业，以及利用Job Checkpoint和Restart功能来实现任务的持久化处理。此外，还包括服务作业和服务队列的设置，以确保作业按照预期执行。 **3. 节点管理** 管理员在这一部分学习如何添加新节点，配置节点属性，如CPU和GPU资源，以及处理节点安全性和Linux Cpuset支持。还介绍了如何调整节点状态以适应不同任务需求。 **4. 服务器政策设置** 涵盖了队列配置，如优先级设定，以及如何实现服务器的高可用性策略，确保系统稳定运行。 **5. 与调度器交互** 文档详细讲解了如何与不同的调度器集成，以便优化资源分配和任务调度，提高整体工作效率。 **6. 数据管理** 这部分涉及文件传输工具（如SCP/RCP）的设置，以及如何使用NFS和其他网络文件系统进行数据共享，以及文件的In/Out操作，确保数据在集群间的高效传输。 **7. 消息传递接口** 最后，管理员会了解如何与消息传递系统交互，以进一步增强系统间的通信和协调，确保任务之间的无缝协作。《TORQUE管理员指南》不仅提供了实用的安装和配置步骤，还包含了大量的最佳实践和故障排查技巧，是任何想要精通Torque系统管理的IT专业人士不可或缺的参考资料。"

资源详情

资源推荐

1.3 Advanced Configuration

1.3.1 Customizing the Install

The TORQUE configure command has several options available. Listed below are some suggested options to

use when running ./configure.

By default, TORQUE does not install the admin manuals. To enable this, use --enable-docs

By default, TORQUE does not enable syslog usage. To enable this, use --enable-syslog

Full Configure Options List

Optional features:

Option Description

--disable-clients directs torque not to build and install the TORQUE client utilities such as qsub, qstat,

qdel, etc.

--disable-

dependency-

tracking

directs TORQUE build system to only rebuild changed source files and not rebuild

dependent files.

--disable-

FEATURE

do not include FEATURE (same as --enable-FEATURE=no).

--disable-gcc-

warnings

Disable gcc strictness and warnings. If using gcc, default is to error on any warning.

--disable-gui do not include the GUI-clients.

--disable-

libtool-lock

avoid locking (might break parallel builds).

--disable-mom do not include the MOM daemon.

--disable-mom-

checkspool

Don't check free space on spool directory and set an error.

--disable-

posixmemlock

disable the moms use of mlockall. Some versions of OSs seem to have buggy POSIX

MEMLOCK.

--disable-

privports

disable the use of privileged ports for authentication. Some versions of OSX have a

buggy bind() and cannot bind to privileged ports.

--disable-qsub-

keep-override

do not allow the qsub -k flag to override -o -e.

--disable-rpp By default, TORQUE uses RPP/UDP for resource queries from the PBS server to the

MOMs. If disabled, TCP is used. This does not affect general node/job status messages,

job launch/exit messages, inter-mom messages, etc.

--disable-server do not include server and scheduler.

--disable-shell-

pipe

give the job script file as standard input to the shell instead of passing its name via a

pipe.

--disable-spool if disabled, TORQUE will create output and error files directly in $HOME/.pbs_spool if it

exists or in $HOME otherwise. By default, TORQUE will spool files in TORQUE_HOME/spool

and copy them to the users home directory when the job completes.

--disable- With HPUX and GCC, don't force usage of XOPEN and libxnet.

xopen-

networking

--enable-acct-x enable adding x attributes to accounting log.

--enable-array setting this under IRIX enables the SGI Origin 2000 parallel support. Normally

autodetected from the /etc/config/array file.

--enable-

autorun

allows jobs to run automatically as soon as they are queued if resources are available

(available in TORQUE 2.3.1 and later).

--enable-blcr enable BLCR support.

--enable-cpa enable Cray's CPA support.

--enable-cpuset enable Linux 2.6 kernel cpusets (in development).

--enable-debug turn on the compilation of DEBUG code.

--enable-

dependency-

tracking

do not reject slow dependency extractors.

--enable-drmaa build the DRMAA 1.0 library.

--enable-fast-

install[=PKGS]

optimize for fast installation [default=yes].

--enable-

FEATURE[=ARG]

include FEATURE [ARG=yes].

--enable-

filesync

Open files with sync on each write operation. This has a negative impact on TORQUE

performance. This is disabled by default.

--enable-force-

nodefile

forces creation of nodefile regardless of job submission parameters. Not on by default.

--enable-

geometry-

requests

TORQUE is compiled to use procs_bitmap during job submission.

--enable-high-

availability

enables enhanced high availability (high availability is enabled by default, but this option

enables the enhanced version)

--enable-

maintainer-

mode

this is for the autoconf utility and tells autoconf to enable so called rebuild rules. See

maintainer mode for more information.

--enable-

maxdefault

turn on the RESOURCEMAXDEFAULT flag.

Versions of TORQUE earlier than 2.4.5 attempted to apply queue and server

defaults to a job that didn't have defaults specified. If a setting still did not have a

value after that, TORQUE applied the queue and server maximum values to a job

(meaning, the maximum values for an applicable setting were applied to jobs that

had no specified or default value).

In TORQUE 2.4.5 and later, the queue and server maximum values are no longer

used as a value for missing settings. To reenable this behavior in TORQUE 2.4.5

and later, use --enable-maxdefault.

--enable-

nochildsignal

turn on the NO_SIGCHLD flag.

--enable-

nodemask

enable nodemask-based scheduling on the Origin 2000.

--enable-

pemask

enable pemask-based scheduling on the Cray T3e.

--enable-plock-

daemons[=ARG]

enable daemons to lock themselves into memory: logical-or of 1 for pbs_server, 2 for

pbs_scheduler, 4 for pbs_mom (no argument means 7 for all three).

--enable-

quickcommit

turn on the QUICKCOMMIT flag.

--enable-

shared[=PKGS]

build shared libraries [default=yes].

--enable-shell-

use-argv

enable this to put the job script name on the command line that invokes the shell. Not on

by default. Ignores --enable-shell-pipe setting.

--enable-sp2 build PBS for an IBM SP2.

--enable-srfs enable support for SRFS on Cray.

--enable-

static[=PKGS]

build static libraries [default=yes].

--enable-syslog enable (default) the use of syslog for error reporting.

--enable-tcl-

qstat

setting this builds qstat with Tcl interpreter features. This is enabled if Tcl is enabled.

--enable-

unixsockets

enable the use of Unix Domain sockets for authentication.

Optional Packages:

Option Description

--with-blcr=DIR BLCR installation prefix (Note: Available in versions 2.5.6 and 3.0.2 and

later.)

--with-blcr-include=DIR include path for libcr.h (Note: Available in versions 2.5.6 and 3.0.2 and later.)

--with-blcr-lib=DIR lib path for libcr (Note: Available in versions 2.5.6 and 3.0.2 and later.)

--with-blcr-bin=DIR bin path for BLCR utilities (Note: Available in versions 2.5.6 and 3.0.2 and

later.)

--with-cpa-include=DIR include path for cpalib.h.

--with-cpa-lib=DIR lib path for libcpalib.

--with-debug compile with debugging symbols.

--with-default-

server=HOSTNAME

set the name of the computer that clients will access when no machine name

is specified as part of the queue name. It defaults to the hostname of the

machine on which PBS is being compiled.

--with-environ=PATH set the path containing the environment variables for the daemons. For SP2

and AIX systems, suggested setting is to /etc/environment. Defaults to the

file "pbs_environment" in the server-home. Relative paths are interpreted

within the context of the server-home.

--with-gnu-ld assume the C compiler uses GNU ld [default=no].

--with-

maildomain=MAILDOMAIN

override the default domain for outgoing mail messages, i.e.

"user@maildomain". The default maildomain is the hostname where the job

was submitted from.

--with-modulefiles[=DIR] use module files in specified directory [/etc/modulefiles].

--with-momlogdir use this directory for MOM logs.

--with-momlogsuffix use this suffix for MOM logs.

--without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no).

--without-readline do not include readline support (default: included if found).

--with-PACKAGE[=ARG] use PACKAGE [ARG=yes].

--with-pam=DIR Directory that holds the system PAM modules. Defaults to /lib(64)/security on

Linux.

--with-pic try to use only PIC/non-PIC objects [default=use both].

--with-qstatrc-file=FILE set the name of the file that qstat will use if there is no ".qstatrc" file in the

directory where it is being invoked. Relative path names will be evaluated

relative to the server home directory (see above). If this option is not

specified, the default name for this file will be set to "qstatrc" (no dot) in the

server home directory.

--with-rcp one of "scp", "rcp", "mom_rcp", or the fullpath of a remote file copy program.

scp is the default if found, otherwise mom_rcp is used. Some rcp programs

don't always exit with valid error codes in case of failure. mom_rcp is a copy

of BSD rcp included with this source that has correct error codes, but it is also

old, unmaintained, and doesn't have largefile support.

--with-sched=TYPE sets the scheduler type. If TYPE is "c", the scheduler will be written in C. If

TYPE is "tcl" the server will use a Tcl based scheduler. If TYPE is "basl",

TORQUE will use the rule based scheduler. If TYPE is "no", then no scheduling

is done. "c" is the default.

--with-sched-code=PATH sets the name of the scheduler to use. This only applies to BASL schedulers

and those written in the C language. For C schedulers this should be a

directory name and for BASL schedulers a filename ending in ".basl". It will be

interpreted relative to srctree/src/schedulers.SCHD_TYPE/samples. As an

example, an appropriate BASL scheduler realtive path would be "nas.basl".

The default scheduler code for "C" schedulers is "fifo".

--with-scp In TORQUE 2.1 and later, SCP is the default remote copy protocol. See --

with-rcp if a different protocol is desired.

--with-sendmail[=FILE] sendmail executable to use.

--with-server-home=DIR set the server home/spool directory for PBS use. Defaults to

/var/spool/torque.

--with-server-name-

file=FILE

set the file that will contain the name of the default server for clients to use.

If this is not an absolute pathname, it will be evaluated relative to the server

home directory that either defaults to /usr/spool/PBS or is set using the --

with-server-home option to configure. If this option is not specified, the

default name for this file will be set to "server_name".

--with-tags[=TAGS] include additional configurations [automatic].

--with-tcl directory containing tcl configuration (tclConfig.sh).

--with-tclatrsep=CHAR set the Tcl attribute separator character this will default to "." if unspecified.

--with-tclinclude directory containing the public Tcl header files.

--with-tclx directory containing tclx configuration (tclxConfig.sh).

--with-tk directory containing tk configuration (tkConfig.sh).

--with-tkinclude directory containing the public Tk header files.

--with-tkx directory containing tkx configuration (tkxConfig.sh).

--with-tmpdir=DIR set the tmp directory that pbs_mom will use. Defaults to "/tmp". This is a

Cray-specific feature.

--with-xauth=PATH specify path to xauth program.

1.3.1.1 HAVE_WORDEXP

Wordxp() performs a shell-like expansion, including environment variables. By default, HAVE_WORDEXP is set

to 1 in src/pbs_config.h. If set to 1, TORQUE will limit the characters that can be used in a job name to those

allowed for a file in the current environment, such as BASH. If set to 0, any valid character for the file system

can be used.

If a user would like to disable this feature by setting HAVE_WORDEXP to 0 in src/include/pbs_config.h, it is

important to note that the error and the output file names will not expand environment variables, including

$PBS_JOBID. The other important consideration is that characters that BASH dislikes, such as (), will not be

allowed in the output and error file names for jobs by default.

1.3.2 Server Configuration

1.3.2.1 Server Configuration Overview

There are several steps to ensure that the server and the nodes are completely aware of each other and able

to communicate directly. Some of this configuration takes place within TORQUE directly using the qmgr

command. Other configuration settings are managed using the pbs_server nodes file, DNS files such as

/etc/hosts and the /etc/hosts.equiv file.

1.3.2.2 Name Service Configuration

Each node, as well as the server, must be able to resolve the name of every node with which it will interact.

This can be accomplished using /etc/hosts, DNS, NIS, or other mechanisms. In the case of /etc/hosts,

the file can be shared across systems in most cases.

A simple method of checking proper name service configuration is to verify that the server and the nodes can

"ping" each other.

1.3.2.3 Configuring Job Submission Hosts

Using RCmd Authentication

When jobs can be submitted from several different hosts, these hosts should be trusted via the R*

commands (such as rsh and rcp). This can be enabled by adding the hosts to the /etc/hosts.equiv file of the

machine executing the pbs_server daemon or using other R* command authorization methods. The exact

specification can vary from OS to OS (see the man page for ruserok to find out how your OS validates

remote users). In most cases, configuring this file is as simple as adding a line to your /etc/hosts.equiv

file, as in the following:

/etc/hosts.equiv:

剩余237页未读，继续阅读

donzds_0412

粉丝: 0
资源: 2

TORQUE管理员指南：集群作业管理详解

torque文档--torqueAdminGuide

TORQUE_Administrator's_Guide

如何部署torque集群

torque sensor怎么连接

design 3rd order Adaptive Torque Filter

3rd order Adaptive Torque Filter

centos7安装pbs_torque

load_torque

怎么定义load_torque(rpm)函数

matlab 怎么定义load_torque(rpm)函数

计算负载扭矩的函数load_torque

This process provides the capability to set torque commands for testing of the torque control function

优化以下数据：[date,txt,raw] = xlsread('490V_1_0-360Nm_0813.xlsx'); speed=date(:,strcmp('Speed1[rpm]',raw(1,:))); torque=date(:,strcmp('Torque1[Nm]',raw(1,:))); plot(speed,torque); xlabel('速度'); ylabel('扭矩')

torque+maui

为什么返回值是这个Series([], Name: Torque_engine, dtype: float64)

torque里面的scheduler用哪个更好？如何设置？

电机控制系统里速度环里Ki_vel_soft乘以bit_conv和Torque_Conv

使用matlab读取excel表，将首行为‘speed1’与‘touque1’的一列提取出来并进行绘图，x轴为‘speed’，y轴为‘torque1’，

读取excel表，将首行为‘speed1’与‘touque1’的一列提取出来并进行绘图，x轴为‘speed’，y轴为‘torque1’，

write a C++ example shows how to do force or torque control in offboard mode from a ROS 2 node with XRCE-DDS

最新资源