HTCondor并行运算系统用户手册

需积分: 11 7 浏览量更新于2024-07-24 收藏 6.49MB PDF 举报

"HTCondor并行运算系统详解" HTCondor是一种专门为计算intensive jobs设计的工作负载管理系统，旨在提供高效、可靠的计算资源管理和调度服务。 HTCondor提供了一个job queueing机制，调度策略，优先级机制，资源监控和资源管理，能够满足用户提交的串行或并行作业的需求。 HTCondor的主要特点包括： 1. 高 THROUGHPUT COMPUTING（HTC）： HTCondor支持高throughput计算，能够处理大量的计算任务，提高计算效率和速度。 2. 作业队列管理：HTCondor提供了一个job queueing机制，能够将用户提交的作业排队，按照调度策略和优先级机制执行作业。 3. 资源监控和管理：HTCondor能够实时监控计算资源的使用情况，动态分配计算资源，提高计算效率和资源利用率。 4. 作业管理：HTCondor提供了多种作业管理机制，包括作业提交、作业监控、作业管理等，能够满足用户的多种需求。 HTCondor的优势包括： 1. 高效的计算资源管理：HTCondor能够高效地管理计算资源，提高计算效率和速度。 2. 灵活的作业管理：HTCondor提供了多种作业管理机制，能够满足用户的多种需求。 3. 可靠的调度策略：HTCondor提供了多种调度策略，能够满足用户的多种需求。 4. 广泛的应用前景：HTCondor能够应用于多种领域，包括科学计算、工程计算、数据分析等。 HTCondor的主要组件包括： 1. HTCondor核心引擎：负责作业调度和资源管理。 2. HTCondor调度器：负责作业调度和资源分配。 3. HTCondor监控器：负责实时监控计算资源的使用情况。 4. HTCondor用户接口：提供了用户友好的接口，方便用户提交和管理作业。 HTCondor的安装和配置过程包括： 1. 下载HTCondor安装包。 2. 安装HTCondor核心引擎和调度器。 3. 配置HTCondor监控器和用户接口。 4. 测试HTCondor系统。 HTCondor的常见应用场景包括： 1. 科学计算：HTCondor能够应用于科学计算领域，例如气候模式、天文模拟等。 2. 工程计算：HTCondor能够应用于工程计算领域，例如结构分析、流体力学分析等。 3. 数据分析：HTCondor能够应用于数据分析领域，例如数据挖掘、机器学习等。 HTCondor是一种功能强大、灵活的工作负载管理系统，能够满足用户的多种需求，提高计算效率和速度。

CONTENTS xv

condor_reconﬁg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 823

condor_release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 826

condor_reschedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 828

condor_restart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 830

condor_rm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 833

condor_rmdir . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836

condor_router_history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 838

condor_router_q . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 840

condor_router_rm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 842

condor_run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 844

condor_set_shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 848

condor_ssh_to_job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 850

condor_stats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854

condor_status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 858

condor_store_cred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866

condor_submit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 868

condor_submit_dag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 908

condor_suspend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 915

condor_tail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 917

condor_transfer_data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 919

condor_updates_stats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 921

condor_userlog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 924

condor_userprio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 927

condor_vacate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 932

condor_vacate_job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 935

condor_version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 938

condor_wait . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 940

condor_who . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 943

HTCondor Version 8.1.2 Manual

CONTENTS xvi

gidd_alloc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 947

procd_ctl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 948

12 Appendix A: ClassAd Attributes 951

13 Appendix B: Codes and Other Needed Values 1004

LICENSING AND COPYRIGHT

HTCondor is released under the Apache License, Version 2.0.

Apache License

Version 2.0, January 2004

http://www.apache.org/licenses/

sity of Wisconsin-Madison, WI.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this ﬁle except in com-

pliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is dis-

tributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express

or implied. See the License for the speciﬁc language governing permissions and limitations under the License.

TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

1. Deﬁnitions.

"License" shall mean the terms and conditions for use, reproduction, and distribution as deﬁned by

Sections 1 through 9 of this document.

"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting

the License.

"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled

by, or are under common control with that entity. For the purposes of this deﬁnition, "control" means (i)

the power, direct or indirect, to cause the direction or management of such entity, whether by contract or

otherwise, or (ii) ownership of ﬁfty percent (50outstanding shares, or (iii) beneﬁcial ownership of such

entity.

"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this Li-

cense.

"Source" form shall mean the preferred form for making modiﬁcations, including but not limited to

software source code, documentation source, and conﬁguration ﬁles.

"Object" form shall mean any form resulting from mechanical transformation or translation of a Source

form, including but not limited to compiled object code, generated documentation, and conversions to

other media types.

HTCondor Version 8.1.2 Manual

CONTENTS xvii

"Work" shall mean the work of authorship, whether in Source or Object form, made available under the

License, as indicated by a copyright notice that is included in or attached to the work (an example is

provided in the Appendix below).

"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived

from) the Work and for which the editorial revisions, annotations, elaborations, or other modiﬁcations

represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works

shall not include works that remain separable from, or merely link (or bind by name) to the interfaces

of, the Work and Derivative Works thereof.

"Contribution" shall mean any work of authorship, including the original version of the Work and any

modiﬁcations or additions to that Work or Derivative Works thereof, that is intentionally submitted to

Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized

to submit on behalf of the copyright owner. For the purposes of this deﬁnition, "submitted" means any

form of electronic, verbal, or written communication sent to the Licensor or its representatives, includ-

ing but not limited to communication on electronic mailing lists, source code control systems, and issue

tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and im-

proving the Work, but excluding communication that is conspicuously marked or otherwise designated

in writing by the copyright owner as "Not a Contribution."

"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution

has been received by Licensor and subsequently incorporated within the Work.

2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby

grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright li-

cense to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and

distribute the Work and such Derivative Works in Source or Object form.

3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby

grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as

stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise

transfer the Work, where such license applies only to those patent claims licensable by such Contributor

that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s)

with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any

entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution

incorporated within the Work constitutes direct or contributory patent infringement, then any patent

licenses granted to You under this License for that Work shall terminate as of the date such litigation is

ﬁled.

4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in

any medium, with or without modiﬁcations, and in Source or Object form, provided that You meet the

following conditions:

(a) You must give any other recipients of the Work or Derivative Works a copy of this License; and

(b) You must cause any modiﬁed ﬁles to carry prominent notices stating that You changed the ﬁles; and

trademark, and attribution notices from the Source form of the Work, excluding those notices that do not

pertain to any part of the Derivative Works; and

(d) If the Work includes a "NOTICE" text ﬁle as part of its distribution, then any Derivative Works that

You distribute must include a readable copy of the attribution notices contained within such NOTICE

ﬁle, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of

the following places: within a NOTICE text ﬁle distributed as part of the Derivative Works; within the

Source form or documentation, if provided along with the Derivative Works; or, within a display gener-

ated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of

the NOTICE ﬁle are for informational purposes only and do not modify the License. You may add Your

HTCondor Version 8.1.2 Manual

CONTENTS xviii

own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the

NOTICE text from the Work, provided that such additional attribution notices cannot be construed as

modifying the License.

You may add Your own copyright statement to Your modiﬁcations and may provide additional or dif-

ferent license terms and conditions for use, reproduction, or distribution of Your modiﬁcations, or for

any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work

otherwise complies with the conditions stated in this License.

5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally sub-

mitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this

License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall

supersede or modify the terms of any separate license agreement you may have executed with Licensor

regarding such Contributions.

6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks,

or product names of the Licensor, except as required for reasonable and customary use in describing the

origin of the Work and reproducing the content of the NOTICE ﬁle.

7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides

the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WAR-

RANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation,

any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS

FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of

using or redistributing the Work and assume any risks associated with Your exercise of permissions

under this License.

8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence),

contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts)

or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect,

special, incidental, or consequential damages of any character arising as a result of this License or out

of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work

stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if

such Contributor has been advised of the possibility of such damages.

9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof,

You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other

liability obligations and/or rights consistent with this License. However, in accepting such obligations,

You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other

Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any

liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such

warranty or additional liability.

END OF TERMS AND CONDITIONS

HTCondor Version 8.1.2 Manual

CHAPTER

ONE

Overview

1.1 High-Throughput Computing (HTC) and its Requirements

For many research and engineering projects, the quality of the research or the product is heavily

dependent upon the quantity of computing cycles available. It is not uncommon to ﬁnd problems

that require weeks or months of computation to solve. Scientists and engineers engaged in this

sort of work need a computing environment that delivers large amounts of computational power

over a long period of time. Such an environment is called a High-Throughput Computing (HTC)

environment. In contrast, High Performance Computing (HPC) environments deliver a tremendous

amount of compute power over a short period of time. HPC environments are often measured in

terms of FLoating point Operations Per Second (FLOPS). A growing community is not concerned

about operations per second, but operations per month or per year. Their problems are of a much

larger scale. They are more interested in how many jobs they can complete over a long period of

time instead of how fast an individual job can complete.

The key to HTC is to efﬁciently harness the use of all available resources. Years ago, the en-

gineering and scientiﬁc community relied on a large, centralized mainframe or a supercomputer to

do computational work. A large number of individuals and groups needed to pool their ﬁnancial re-

sources to afford such a machine. Users had to wait for their turn on the mainframe, and they had a

limited amount of time allocated. While this environment was inconvenient for users, the utilization

of the mainframe was high; it was busy nearly all the time.

As computers became smaller, faster, and cheaper, users moved away from centralized main-

frames and purchased personal desktop workstations and PCs. An individual or small group could

afford a computing resource that was available whenever they wanted it. The personal computer is

slower than the large centralized machine, but it provides exclusive access. Now, instead of one giant

computer for a large institution, there may be hundreds or thousands of personal computers. This

剩余1084页未读，继续阅读

tjjhk

粉丝: 0
资源: 2

HTCondor并行运算系统用户手册

PyPI 官网下载 | htcondor-8.7.6-cp27-cp27mu-manylinux1_x86_64.whl

Python库 | condor-0.0.4-py2-none-any.whl

贝岭的matlab的代码-belle2-condor-sd:belle2-condor-sd

condor-copasi:从 code.google.compcondor-copasi 自动导出

The-Legend-of-the-Condor-Heroes:射雕英雄传

condor_espresso_glue:使用ESPResSo和condor优雅地运行参数扫描的python中间件

Adding Graphics Card Support into Condor-开源

condor-kubernetes:为 Kubernetes dockerized HTCondor

condor-mongoose：与猫鼬和condor GRPC框架一起使用的实用程序

Motorola-Moto-E-condor-unbrick:摩托罗拉Moto E XT1021，XT1022，XT1025的工具

最新资源