Rocky8.4集群搭建：Warewulf与SLURM集成指南

需积分: 10 190 浏览量更新于2024-07-09 收藏 548KB PDF 举报

"该文档是'Install_guide-Rocky8-Warewulf-SLURM-2.3-x86_64.pdf'，主要内容涉及使用Rocky Linux 8.4操作系统、Warewulf和SLURM搭建高性能计算(HPC)集群的指南，更新日期为2021年6月26日，修订版本号为411abc72d。" 本文档主要面向对Linux HPC集群构建感兴趣的用户，提供了从基础操作系统安装到OpenHPC组件部署，再到Warewulf和SLURM集群管理软件设置的详细步骤。 1. **目标受众**: 文档的目标读者是那些想要构建基于Rocky Linux 8.4的HPC集群的技术人员，他们可能具有一定的Linux系统管理经验，对高性能计算有基本的了解，并希望通过Warewulf和SLURM实现集群的高效运行。 2. **需求与假设**: 在开始之前，用户需要具备一台或多台支持x86_64架构的硬件设备，准备安装Rocky Linux 8.4作为基础操作系统。此外，用户应该熟悉基本的Linux命令行操作，以及网络配置等基础知识。 3. **输入**: 按照文档进行操作时，用户需要准备的是硬件设备、Rocky Linux 8.4的安装介质，以及访问OpenHPC仓库的权限，以便安装必要的组件。 4. **安装基础操作系统(BOS)**: 这一步骤涉及在所有集群节点上安装Rocky Linux 8.4。用户需要遵循标准的Linux安装流程，确保所有节点配置一致，特别是网络设置，以便于后续的集群配置。 5. **安装OpenHPC组件**: OpenHPC是一套用于简化HPC集群搭建和管理的开源工具集。用户首先需要在master节点上启用OpenHPC仓库，然后利用模板进行安装。这将包括如MPI库、性能监控工具和其他集群必需的组件。 6. **添加配置服务到master节点**: 在master节点上安装 Warewulf，它是一个用于快速部署和管理计算节点的工具。通过Warewulf，用户可以创建虚拟机映像并分发到计算节点，实现统一的系统配置。 7. **在master节点上添加资源管理服务**: SLURM（Simple Linux Utility for Resource Management）是集群的资源调度器，负责分配计算任务到各个节点。用户需要在master节点上安装并配置SLURM，包括设置账户、队列策略和调度算法。 8. **节点配置与测试**: 完成上述步骤后，用户需要配置计算节点，连接到master节点，并通过SLURM验证节点的加入和功能正确性。这通常包括网络验证、SSH通信测试和SLURM的简单作业提交测试。 9. **故障排查与优化**: 文档可能还会提供一些常见问题的解决方案，以及如何根据实际需求调整和优化SLURM配置以提高集群效率。 10. **维护与更新**: 最后，用户应了解如何对集群进行日常维护，包括更新组件、监控性能、备份重要数据等，以确保集群的稳定运行。总体来说，这份"Install_guide-Rocky8-Warewulf-SLURM-2.3-x86_64.pdf"是针对HPC集群建设者的一份详尽指南，涵盖了从基础到高级的多个层面，旨在帮助用户成功构建一个高效、可靠的计算环境。

Install Guide (v2.3): Rocky 8.4/x86 64 + Warewulf + SLURM

3.4 Add resource management services on master node

OpenHPC provides multiple options for distributed resource management. The following command adds the

Slurm workload manager server components to the chosen master host. Note that client-side components

will be added to the corresponding compute image in a subsequent step.

# Install slurm server meta-package

[sms]# yum -y install ohpc-slurm-server

# Use ohpc-provided file for starting SLURM configuration

[sms]# cp /etc/slurm/slurm.conf.ohpc /etc/slurm/slurm.conf

# Identify resource manager hostname on master host

[sms]# perl -pi -e "s/ControlMachine=\S+/ControlMachine=${sms_name}/" /etc/slurm/slurm.conf

There are a wide variety of conﬁguration options and plugins available for Slurm and the example conﬁg

ﬁle illustrated above targets a fairly basic installation. In particular, job completion data will be stored in a

text ﬁle (/var/log/slurm jobcomp.log) that can be used to log simple accounting information. Sites who

desire more detailed information, or want to aggregate accounting data from multiple clusters, will likely

want to enable the database accounting back-end. This requires a number of additional local modiﬁcations

(on top of installing slurm-slurmdbd-ohpc), and users are advised to consult the online documentation for

more detailed information on setting up a database conﬁguration for Slurm.

Tip

SLURM requires enumeration of the physical hardware characteristics for compute nodes under its control.

In particular, three conﬁguration parameters combine to deﬁne consumable compute resources: Sockets,

CoresPerSocket, and ThreadsPerCore. The default conﬁguration ﬁle provided via OpenHPC assumes dual-

socket, 8 cores per socket, and two threads per core for this 4-node example. If this does not reﬂect your

local hardware, please update the conﬁguration ﬁle at /etc/slurm/slurm.conf accordingly to match your

particular hardware. Note that the SLURM project provides an easy-to-use online conﬁguration tool that

can be accessed here.

Other versions of this guide are available that describe installation of alternate resource management

systems, and they can be found in the docs-ohpc package.

3.5 Optionally add InﬁniBand support services on master node

The following command adds OFED and PSM support using base distro-provided drivers to the chosen

master host.

[sms]# yum -y groupinstall "InfiniBand Support"

# Load IB drivers

[sms]# systemctl start rdma

Tip

InﬁniBand networks require a subnet management service that can typically be run on either an

administrative node, or on the switch itself. The optimal placement and conﬁguration of the subnet

manager is beyond the scope of this document, but Rocky 8.4 provides the opensm package should

you choose to run it on the master node.

10 Rev: 411abc72d

Install Guide (v2.3): Rocky 8.4/x86 64 + Warewulf + SLURM

With the InﬁniBand drivers included, you can also enable (optional) IPoIB functionality which provides

a mechanism to send IP packets over the IB network. If you plan to mount a Lustre ﬁle system over

InﬁniBand (see §3.8.4.6 for additional details), then having IPoIB enabled is a requirement for the Lustre

client. OpenHPC provides a template conﬁguration ﬁle to aid in setting up an ib0 interface on the master

host. To use, copy the template provided and update the ${sms ipoib} and ${ipoib netmask} entries to

match local desired settings (alter ib0 naming as appropriate if system contains dual-ported or multiple

HCAs).

[sms]# cp /opt/ohpc/pub/examples/network/centos/ifcfg-ib0 /etc/sysconfig/network-scripts

# Define local IPoIB address and netmask

[sms]# perl -pi -e "s/master_ipoib/${sms_ipoib}/" /etc/sysconfig/network-scripts/ifcfg-ib0

[sms]# perl -pi -e "s/ipoib_netmask/${ipoib_netmask}/" /etc/sysconfig/network-scripts/ifcfg-ib0

# configure NetworkManager to *not* override local /etc/resolv.conf

[sms]# echo "[main]" > /etc/NetworkManager/conf.d/90-dns-none.conf

[sms]# echo "dns=none" >> /etc/NetworkManager/conf.d/90-dns-none.conf

# Start up NetworkManager to initiate ib0

[sms]# systemctl start NetworkManager

3.6 Optionally add Omni-Path support services on master node

The following command adds Omni-Path support using base distro-provided drivers to the chosen master

host.

[sms]# yum -y install opa-basic-tools

# Load RDMA services

[sms]# systemctl start rdma

Tip

Omni-Path networks require a subnet management service that can typically be run on either an

administrative node, or on the switch itself. The optimal placement and conﬁguration of the subnet

manager is beyond the scope of this document, but Rocky 8.4 provides the opa-fm package should

you choose to run it on the master node.

3.7 Complete basic Warewulf setup for master node

At this point, all of the packages necessary to use Warewulf on the master host should be installed. Next, we

need to update several conﬁguration ﬁles in order to allow Warewulf to work with Rocky 8.4 and to support

local provisioning using a second private interface (refer to Figure 1).

Tip

By default, Warewulf is conﬁgured to provision over the eth1 interface and the steps below include updating

this setting to override with a potentially alternatively-named interface speciﬁed by ${sms eth internal}.

11 Rev: 411abc72d

剩余54页未读，继续阅读

lesite

粉丝: 0
资源: 3

Rocky8.4集群搭建：Warewulf与SLURM集成指南

openssl-1.0.2k-24.el7-9.x86-64.rpm包

openstack安装包（一）

kernel-lt-5.4.264-1.el7.elrepo.x86-64.rpm

安装过程中提示Unable to find a match: mingw-w64-x86_64-toolchain

(mysql-8.0.33-linux-glibc2.28-x86_64.tar.gz

ubuntu22.04使用qemu创建一个Rocky9的虚拟主机

kylin v10 使用qemu命令安装Rocky9虚拟主机

用ubuntu系统使用qemu创建一个Rocky的虚拟主机

升级内核RockyLinux9.2

最新资源