TORQUE管理员指南：配置与管理分布式计算资源

需积分: 9 120 浏览量更新于2024-07-27 收藏 992KB PDF 举报

"TORQUE Administrator's Guide 是一份针对TORQUE资源管理系统的管理员指南，版本3.0.2。该文档详细介绍了如何安装、配置和管理TORQUE，以及与PBS（Portable Batch System）和MPI（Message Passing Interface）的集成。" 在TORQUE系统中，管理员需要掌握以下关键知识点： 1. **概述**：TORQUE是一个开源的工作队列系统，用于管理计算集群中的作业调度和资源分配。它提供了一个中央服务器（pbs_server）来处理作业提交、状态跟踪和资源分配。 2. **安装与初始化**：安装TORQUE涉及在服务器上配置pbs_server。这一过程包括安装软件包、设置环境变量和启动服务。初始化配置需要根据集群的具体需求进行调整。 3. **高级配置**：高级配置涉及对服务器、节点和队列的定制，如设置资源限制、优先级和调度策略。 4. **手动设置初始服务器配置**：这一步骤可能包括编写和编辑配置文件，如`pbs_mom.conf`和`pbs_server.conf`，以确保所有节点能正确通信。 5. **服务器节点文件配置**：每个节点都需要在服务器的节点列表文件中注册，定义其资源容量（如CPU、内存等）。 6. **测试服务器配置**：提交测试作业来验证服务器配置是否正确，确保作业可以被调度、执行和完成。 7. **TORQUE在NUMA系统上的使用**：NUMA（Non-Uniform Memory Access）系统需要特别的配置，以优化多核节点上的内存访问。 8. **TORQUE多-MOM**：多-MOM配置允许在一个节点上运行多个Mom（Mesage Oriented Middleware）进程，提高资源利用率和故障恢复能力。 9. **提交和管理作业**：管理员应了解如何使用`qsub`命令提交作业，使用`qstat`监控作业状态，以及使用`qdel`取消作业。此外，还包括理解预占机制、保留已完成的作业记录、检查点和重启功能，以及服务作业的管理。 10. **管理节点**：添加新节点、配置节点属性、改变节点状态（如离线、维护模式）以及实施主机安全策略是日常管理任务。Linux Cpuset支持允许更精细的CPU资源隔离，而调度核心和GPU则涉及到更复杂的资源分配策略。 11. **设置服务器策略**：队列配置是控制作业执行的关键，包括设置资源限制、队列优先级和作业保持时间。同时，服务器高可用性配置保证了服务的连续性。 12. **与调度器接口**：TORQUE可以与多种调度器（如Maui或Moab）集成，以实现更智能的作业调度策略。 13. **数据管理配置**：包括设置SCP/RCP（Secure Copy/Remote Copy）协议，利用NFS（网络文件系统）或其他网络文件系统，以及实现作业的输入/输出文件阶段操作。 14. **与消息传递接口**：TORQUE支持MPI作业，管理员需要了解如何配置MPI运行时环境，以便作业能在分布式环境中正确通信。这份指南详尽地覆盖了TORQUE的各个方面，是集群管理员管理和优化资源分配的必备参考资料。通过深入理解和实践这些知识点，可以确保高效、可靠地运行大规模并行计算任务。

1.3 Advanced Configuration

1.3.1 Customizing the Install

The TORQUE configure command has several options available. Listed below are some suggested options to

use when running ./configure.

By default, TORQUE does not install the admin manuals. To enable this, use --enable-docs

By default, TORQUE does not enable syslog usage. To enable this, use --enable-syslog

Full Configure Options List

Optional features:

Option Description

--disable-clients directs torque not to build and install the TORQUE client utilities such as qsub, qstat,

qdel, etc.

--disable-

dependency-

tracking

directs TORQUE build system to only rebuild changed source files and not rebuild

dependent files.

--disable-

FEATURE

do not include FEATURE (same as --enable-FEATURE=no).

--disable-gcc-

warnings

Disable gcc strictness and warnings. If using gcc, default is to error on any warning.

--disable-gui do not include the GUI-clients.

--disable-

libtool-lock

avoid locking (might break parallel builds).

--disable-mom do not include the MOM daemon.

--disable-mom-

checkspool

Don't check free space on spool directory and set an error.

--disable-

posixmemlock

disable the moms use of mlockall. Some versions of OSs seem to have buggy POSIX

MEMLOCK.

--disable-

privports

disable the use of privileged ports for authentication. Some versions of OSX have a

buggy bind() and cannot bind to privileged ports.

--disable-qsub-

keep-override

do not allow the qsub -k flag to override -o -e.

--disable-rpp By default, TORQUE uses RPP/UDP for resource queries from the PBS server to the

MOMs. If disabled, TCP is used. This does not affect general node/job status messages,

job launch/exit messages, inter-mom messages, etc.

--disable-server do not include server and scheduler.

--disable-shell-

pipe

give the job script file as standard input to the shell instead of passing its name via a

pipe.

--disable-spool if disabled, TORQUE will create output and error files directly in $HOME/.pbs_spool if it

exists or in $HOME otherwise. By default, TORQUE will spool files in TORQUE_HOME/spool

and copy them to the users home directory when the job completes.

--disable- With HPUX and GCC, don't force usage of XOPEN and libxnet.

xopen-

networking

--enable-acct-x enable adding x attributes to accounting log.

--enable-array setting this under IRIX enables the SGI Origin 2000 parallel support. Normally

autodetected from the /etc/config/array file.

--enable-

autorun

allows jobs to run automatically as soon as they are queued if resources are available

(available in TORQUE 2.3.1 and later).

--enable-blcr enable BLCR support.

--enable-cpa enable Cray's CPA support.

--enable-cpuset enable Linux 2.6 kernel cpusets (in development).

--enable-debug turn on the compilation of DEBUG code.

--enable-

dependency-

tracking

do not reject slow dependency extractors.

--enable-drmaa build the DRMAA 1.0 library.

--enable-fast-

install[=PKGS]

optimize for fast installation [default=yes].

--enable-

FEATURE[=ARG]

include FEATURE [ARG=yes].

--enable-

filesync

Open files with sync on each write operation. This has a negative impact on TORQUE

performance. This is disabled by default.

--enable-force-

nodefile

forces creation of nodefile regardless of job submission parameters. Not on by default.

--enable-

geometry-

requests

TORQUE is compiled to use procs_bitmap during job submission.

--enable-high-

availability

enables enhanced high availability (high availability is enabled by default, but this option

enables the enhanced version)

--enable-

maintainer-

mode

this is for the autoconf utility and tells autoconf to enable so called rebuild rules. See

maintainer mode for more information.

--enable-

maxdefault

turn on the RESOURCEMAXDEFAULT flag.

Versions of TORQUE earlier than 2.4.5 attempted to apply queue and server

defaults to a job that didn't have defaults specified. If a setting still did not have a

value after that, TORQUE applied the queue and server maximum values to a job

(meaning, the maximum values for an applicable setting were applied to jobs that

had no specified or default value).

In TORQUE 2.4.5 and later, the queue and server maximum values are no longer

used as a value for missing settings. To reenable this behavior in TORQUE 2.4.5

and later, use --enable-maxdefault.

--enable-

nochildsignal

turn on the NO_SIGCHLD flag.

--enable-

nodemask

enable nodemask-based scheduling on the Origin 2000.

--enable-

pemask

enable pemask-based scheduling on the Cray T3e.

--enable-plock-

daemons[=ARG]

enable daemons to lock themselves into memory: logical-or of 1 for pbs_server, 2 for

pbs_scheduler, 4 for pbs_mom (no argument means 7 for all three).

--enable-

quickcommit

turn on the QUICKCOMMIT flag.

--enable-

shared[=PKGS]

build shared libraries [default=yes].

--enable-shell-

use-argv

enable this to put the job script name on the command line that invokes the shell. Not on

by default. Ignores --enable-shell-pipe setting.

--enable-sp2 build PBS for an IBM SP2.

--enable-srfs enable support for SRFS on Cray.

--enable-

static[=PKGS]

build static libraries [default=yes].

--enable-syslog enable (default) the use of syslog for error reporting.

--enable-tcl-

qstat

setting this builds qstat with Tcl interpreter features. This is enabled if Tcl is enabled.

--enable-

unixsockets

enable the use of Unix Domain sockets for authentication.

Optional Packages:

Option Description

--with-blcr=DIR BLCR installation prefix (Note: Available in versions 2.5.6 and 3.0.2 and

later.)

--with-blcr-include=DIR include path for libcr.h (Note: Available in versions 2.5.6 and 3.0.2 and later.)

--with-blcr-lib=DIR lib path for libcr (Note: Available in versions 2.5.6 and 3.0.2 and later.)

--with-blcr-bin=DIR bin path for BLCR utilities (Note: Available in versions 2.5.6 and 3.0.2 and

later.)

--with-cpa-include=DIR include path for cpalib.h.

--with-cpa-lib=DIR lib path for libcpalib.

--with-debug compile with debugging symbols.

--with-default-

server=HOSTNAME

set the name of the computer that clients will access when no machine name

is specified as part of the queue name. It defaults to the hostname of the

machine on which PBS is being compiled.

--with-environ=PATH set the path containing the environment variables for the daemons. For SP2

and AIX systems, suggested setting is to /etc/environment. Defaults to the

file "pbs_environment" in the server-home. Relative paths are interpreted

within the context of the server-home.

--with-gnu-ld assume the C compiler uses GNU ld [default=no].

--with-

maildomain=MAILDOMAIN

override the default domain for outgoing mail messages, i.e.

"user@maildomain". The default maildomain is the hostname where the job

was submitted from.

--with-modulefiles[=DIR] use module files in specified directory [/etc/modulefiles].

--with-momlogdir use this directory for MOM logs.

--with-momlogsuffix use this suffix for MOM logs.

--without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no).

--without-readline do not include readline support (default: included if found).

--with-PACKAGE[=ARG] use PACKAGE [ARG=yes].

--with-pam=DIR Directory that holds the system PAM modules. Defaults to /lib(64)/security on

Linux.

--with-pic try to use only PIC/non-PIC objects [default=use both].

--with-qstatrc-file=FILE set the name of the file that qstat will use if there is no ".qstatrc" file in the

directory where it is being invoked. Relative path names will be evaluated

relative to the server home directory (see above). If this option is not

specified, the default name for this file will be set to "qstatrc" (no dot) in the

server home directory.

--with-rcp one of "scp", "rcp", "mom_rcp", or the fullpath of a remote file copy program.

scp is the default if found, otherwise mom_rcp is used. Some rcp programs

don't always exit with valid error codes in case of failure. mom_rcp is a copy

of BSD rcp included with this source that has correct error codes, but it is also

old, unmaintained, and doesn't have largefile support.

--with-sched=TYPE sets the scheduler type. If TYPE is "c", the scheduler will be written in C. If

TYPE is "tcl" the server will use a Tcl based scheduler. If TYPE is "basl",

TORQUE will use the rule based scheduler. If TYPE is "no", then no scheduling

is done. "c" is the default.

--with-sched-code=PATH sets the name of the scheduler to use. This only applies to BASL schedulers

and those written in the C language. For C schedulers this should be a

directory name and for BASL schedulers a filename ending in ".basl". It will be

interpreted relative to srctree/src/schedulers.SCHD_TYPE/samples. As an

example, an appropriate BASL scheduler realtive path would be "nas.basl".

The default scheduler code for "C" schedulers is "fifo".

--with-scp In TORQUE 2.1 and later, SCP is the default remote copy protocol. See --

with-rcp if a different protocol is desired.

--with-sendmail[=FILE] sendmail executable to use.

--with-server-home=DIR set the server home/spool directory for PBS use. Defaults to

/var/spool/torque.

--with-server-name-

file=FILE

set the file that will contain the name of the default server for clients to use.

If this is not an absolute pathname, it will be evaluated relative to the server

home directory that either defaults to /usr/spool/PBS or is set using the --

with-server-home option to configure. If this option is not specified, the

default name for this file will be set to "server_name".

--with-tags[=TAGS] include additional configurations [automatic].

--with-tcl directory containing tcl configuration (tclConfig.sh).

--with-tclatrsep=CHAR set the Tcl attribute separator character this will default to "." if unspecified.

--with-tclinclude directory containing the public Tcl header files.

--with-tclx directory containing tclx configuration (tclxConfig.sh).

--with-tk directory containing tk configuration (tkConfig.sh).

--with-tkinclude directory containing the public Tk header files.

--with-tkx directory containing tkx configuration (tkxConfig.sh).

--with-tmpdir=DIR set the tmp directory that pbs_mom will use. Defaults to "/tmp". This is a

Cray-specific feature.

--with-xauth=PATH specify path to xauth program.

1.3.1.1 HAVE_WORDEXP

Wordxp() performs a shell-like expansion, including environment variables. By default, HAVE_WORDEXP is set

to 1 in src/pbs_config.h. If set to 1, TORQUE will limit the characters that can be used in a job name to those

allowed for a file in the current environment, such as BASH. If set to 0, any valid character for the file system

can be used.

If a user would like to disable this feature by setting HAVE_WORDEXP to 0 in src/include/pbs_config.h, it is

important to note that the error and the output file names will not expand environment variables, including

$PBS_JOBID. The other important consideration is that characters that BASH dislikes, such as (), will not be

allowed in the output and error file names for jobs by default.

1.3.2 Server Configuration

1.3.2.1 Server Configuration Overview

There are several steps to ensure that the server and the nodes are completely aware of each other and able

to communicate directly. Some of this configuration takes place within TORQUE directly using the qmgr

command. Other configuration settings are managed using the pbs_server nodes file, DNS files such as

/etc/hosts and the /etc/hosts.equiv file.

1.3.2.2 Name Service Configuration

Each node, as well as the server, must be able to resolve the name of every node with which it will interact.

This can be accomplished using /etc/hosts, DNS, NIS, or other mechanisms. In the case of /etc/hosts,

the file can be shared across systems in most cases.

A simple method of checking proper name service configuration is to verify that the server and the nodes can

"ping" each other.

1.3.2.3 Configuring Job Submission Hosts

Using RCmd Authentication

When jobs can be submitted from several different hosts, these hosts should be trusted via the R*

commands (such as rsh and rcp). This can be enabled by adding the hosts to the /etc/hosts.equiv file of the

machine executing the pbs_server daemon or using other R* command authorization methods. The exact

specification can vary from OS to OS (see the man page for ruserok to find out how your OS validates

remote users). In most cases, configuring this file is as simple as adding a line to your /etc/hosts.equiv

file, as in the following:

/etc/hosts.equiv:

剩余237页未读，继续阅读

net629

粉丝: 1
资源: 6

TORQUE管理员指南：配置与管理分布式计算资源

TORQUE Administrator's Guide: Understanding and Managing Batch Systems

PMSM电机控制策略：MTPA、MTPV与弱磁技术研究

"E1系列驱动器V1.5使用手册更新内容总结

TORQUE_Administrator's_Guide

torque_speed_curve_mod.rar_MOD

Direct_Torque_Control_Voltage_Model.rar_dtc pmsm_pmsm dtc_torque

Direct_Torque_Control_Lookup_Tables.zip_DTC modified_Voltage spa

Python库 | torque_drag-0.0.2.tar.gz

torque_jiao3_2_MTPV_2_26.mdl

Torque_&_Speed_Estimation_Induction_Motor.zip_Motors2_Speed Esti

最新资源