OpenHPC CentOS 7.4 安装指南：Warewulf SLURM v1.3.4 x86_64集群部署教程

需积分: 25 95 浏览量更新于2024-07-16 收藏 541KB PDF 举报

本篇文档是关于在CentOS 7.4平台上安装OpenHPC v1.3.4软件栈的详细指南，特别强调了使用Warewulf和SLURM版本的Linux系统，适用于x86_64架构。文档创建日期为2018年4月2日，修订版本为520a42ff2，适用于构建一个小型、无状态的四节点集群，并提供了通用步骤，同时允许用户进行适当的定制以适应本地环境。目标受众包括希望部署和管理HPC（高性能计算）Linux集群的系统管理员和技术人员。该指南假设读者对基础Linux操作和HPC工具有一定了解，如理解如何配置存储和网络、以及对SLURM资源管理器和Warewulf分发模型有所认知。安装过程分为几个关键步骤： 1. **安装基础操作系统** (BOS)：文档首先介绍了如何在CentOS 7.4上安装并配置基本的Linux环境，确保系统稳定且准备好后续的HPC组件安装。 2. **启用OpenHPC本地仓库**：为了获取预构建的OpenHPC软件包，需要配置本地仓库，这一步骤确保了后续安装的软件包能够正确地从官方源获取。 3. **安装模板**：文档指导用户如何使用OpenHPC提供的安装模板，这是一种自动化脚本，可以简化组件的安装过程，如SLURM管理和 Warewulf的配置。 4. **添加服务到主节点**：在主节点上，安装过程中会涉及添加必要的服务，如Provisioning服务（可能涉及到用户账户管理、硬件配置等）和Resource Management服务（如SLURM的调度器和监控工具）。 5. **扩展到其他节点**：根据集群规模，指南将指导如何在剩余的节点上重复这些步骤，可能涉及到网络配置和数据复制。 6. **可选定制**：虽然文档提供的是标准流程，但作者强调可以根据本地需求进行修改，比如调整配置文件、安装特定的科学库或I/O客户端。在整个安装过程中，文档遵循了OpenHPC的统一框架和Linux发行版的标准，确保了系统的兼容性和性能优化。此外，文档还包含了版权信息和许可条款，指出所有内容受到Creative Commons Attribution 4.0 International License的保护，并提到了Intel商标的使用情况。通过阅读这篇文档，读者将能掌握在一个小规模的HPC集群环境中，如何利用OpenHPC v1.3.4和Warewulf/SLURM来构建、配置和维护一个功能强大的高性能计算平台。

Install Guide (v1.3.4): CentOS7.4/x86 64 + Warewulf + SLURM

Tip

SLURM requires enumeration of the physical hardware characteristics for compute nodes under its control.

In particular, three conﬁguration parameters combine to deﬁne consumable compute resources: Sockets,

CoresPerSocket, and ThreadsPerCore. The default conﬁguration ﬁle provided via OpenHPC assumes dual-

socket, 8 cores per socket, and two threads per core for this 4-node example. If this does not reﬂect your

local hardware, please update the conﬁguration ﬁle at /etc/slurm/slurm.conf accordingly to match your

particular hardware. Note that the SLURM project provides an easy-to-use online conﬁguration tool that

can be accessed here.

Other versions of this guide are available that describe installation of alternate resource management

systems, and they can be found in the docs-ohpc package.

3.5 Optionally add InﬁniBand support services on master node

The following command adds OFED and PSM support using base distro-provided drivers to the chosen

master host.

[sms]# yum -y groupinstall "InfiniBand Support"

[sms]# yum -y install infinipath-psm

# Load IB drivers

[sms]# systemctl start rdma

Tip

InﬁniBand networks require a subnet management service that can typically be run on either an

administrative node, or on the switch itself. The optimal placement and conﬁguration of the subnet

manager is beyond the scope of this document, but CentOS7.4 provides the opensm package should

you choose to run it on the master node.

With the InﬁniBand drivers included, you can also enable (optional) IPoIB functionality which provides

a mechanism to send IP packets over the IB network. If you plan to mount a Lustre ﬁle system over

InﬁniBand (see §3.8.4.6 for additional details), then having IPoIB enabled is a requirement for the Lustre

client. OpenHPC provides a template conﬁguration ﬁle to aid in setting up an ib0 interface on the master

host. To use, copy the template provided and update the ${sms ipoib} and ${ipoib netmask} entries to

match local desired settings (alter ib0 naming as appropriate if system contains dual-ported or multiple

HCAs).

[sms]# cp /opt/ohpc/pub/examples/network/centos/ifcfg-ib0 /etc/sysconfig/network-scripts

# Define local IPoIB address and netmask

[sms]# perl -pi -e "s/master_ipoib/${sms_ipoib}/" /etc/sysconfig/network-scripts/ifcfg-ib0

[sms]# perl -pi -e "s/ipoib_netmask/${ipoib_netmask}/" /etc/sysconfig/network-scripts/ifcfg-ib0

# Initiate ib0

[sms]# ifup ib0

10 Rev: 520a42ﬀ2

Install Guide (v1.3.4): CentOS7.4/x86 64 + Warewulf + SLURM

3.6 Optionally add Omni-Path support services on master node

The following command adds Omni-Path support using base distro-provided drivers to the chosen master

host.

[sms]# yum -y install opa-basic-tools

# Load RDMA services

[sms]# systemctl start rdma

Tip

Omni-Path networks require a subnet management service that can typically be run on either an

administrative node, or on the switch itself. The optimal placement and conﬁguration of the subnet

manager is beyond the scope of this document, but CentOS7.4 provides the opa-fm package should

you choose to run it on the master node.

3.7 Complete basic Warewulf setup for master node

At this point, all of the packages necessary to use Warewulf on the master host should be installed. Next,

we need to update several conﬁguration ﬁles in order to allow Warewulf to work with CentOS7.4 and to

support local provisioning using a second private interface (refer to Figure 1).

Tip

By default, Warewulf is conﬁgured to provision over the eth1 interface and the steps below include updating

this setting to override with a potentially alternatively-named interface speciﬁed by ${sms eth internal}.

# Configure Warewulf provisioning to use desired internal interface

[sms]# perl -pi -e "s/device = eth1/device = ${sms_eth_internal}/" /etc/warewulf/provision.conf

# Enable tftp service for compute node image distribution

[sms]# perl -pi -e "s/^\s+disable\s+= yes/ disable = no/" /etc/xinetd.d/tftp

# Enable internal interface for provisioning

[sms]# ifconfig ${sms_eth_internal} ${sms_ip} netmask ${internal_netmask} up

# Restart/enable relevant services to support provisioning

[sms]# systemctl restart xinetd

[sms]# systemctl enable mariadb.service

[sms]# systemctl restart mariadb

[sms]# systemctl enable httpd.service

[sms]# systemctl restart httpd

[sms]# systemctl enable dhcpd.service

3.8 Deﬁne compute image for provisioning

With the provisioning services enabled, the next step is to deﬁne and customize a system image that can

subsequently be used to provision one or more compute nodes. The following subsections highlight this

process.

11 Rev: 520a42ﬀ2

Install Guide (v1.3.4): CentOS7.4/x86 64 + Warewulf + SLURM

3.8.1 Build initial BOS image

The OpenHPC build of Warewulf includes speciﬁc enhancements enabling support for CentOS7.4. The

following steps illustrate the process to build a minimal, default image for use with Warewulf. We begin

by deﬁning a directory structure on the master host that will represent the root ﬁlesystem of the compute

node. The default location for this example is in /opt/ohpc/admin/images/centos7.4.

Tip

Warewulf is conﬁgured by default to access an external repository (mirror.centos.org) during the wwmkchroot

process. If the master host cannot reach the public CentOS mirrors, or if you prefer to access a locally

cached mirror, set the ${YUM MIRROR} environment variable to your desired repo location prior to running the

wwmkchroot command below. For example:

# Override default OS repository (optional) - set YUM_MIRROR variable to desired repo location

[sms]# export YUM_MIRROR=${BOS_MIRROR}

# Define chroot location

[sms]# export CHROOT=/opt/ohpc/admin/images/centos7.4

# Build initial chroot image

[sms]# wwmkchroot centos-7 $CHROOT

3.8.2 Add OpenHPC components

The wwmkchroot process used in the previous step is designed to provide a minimal CentOS7.4 conﬁguration.

Next, we add additional components to include resource management client services, NTP support, and

other additional packages to support the default OpenHPC environment. This process augments the chroot-

based install performed by wwmkchroot to modify the base provisioning image and will access the BOS

and OpenHPC repositories to resolve package install requests. We begin by installing a few common base

packages:

# Install compute node base meta-package

[sms]# yum -y --installroot=$CHROOT install ohpc-base-compute

To access the remote repositories by hostname (and not IP addresses), the chroot environment needs to

be updated to enable DNS resolution. Assuming that the master host has a working DNS conﬁguration in

place, the chroot environment can be updated with a copy of the conﬁguration as follows:

[sms]# cp -p /etc/resolv.conf $CHROOT/etc/resolv.conf

Now, we can include additional components to the compute instance using yum to install into the chroot

location deﬁned previously:

# Add Slurm client support meta-package

[sms]# yum -y --installroot=$CHROOT install ohpc-slurm-client

# Add Network Time Protocol (NTP) support

[sms]# yum -y --installroot=$CHROOT install ntp

12 Rev: 520a42ﬀ2

剩余55页未读，继续阅读

frank0060071

粉丝: 2030
资源: 103

OpenHPC CentOS 7.4 安装指南：Warewulf SLURM v1.3.4 x86_64集群部署教程

Install_guide-CentOS7-Warewulf-SLURM-1.3.9-x86_64.pdf

warewulf3:Warewulf是可伸缩的系统管理套件，最初是为管理大型高性能Linux集群而开发的

slurm安装文档

下载httpd-2.4.6-80.el7.centos.1.x86_64.rpm软件包到/root并安装

这个http://vault.centos.org/5.8/os/x86_64/CentOS/pcre-devel-6.6-9.el5_6.1.x86_64.rpm地址下载不到，再换一个

centos-release-7-9.2009.1.el7.centos.x86_64.rpm

wget http://mirror.centos.org/centos/7/os/x86_64/Packages/glibc-devel-2.17-222.el7.x86_64.rpm

这个链接也失效了http://mirror.centos.org/centos/5/os/x86_64/CentOS/pcre-devel-6.6-9.el5.x86_64.rpm，还有其他的吗

linux如何下载libcap-2.16-5.5.el6.x86_64.rpm和libcap-devel-2.16-5.5.el6.x86_6

httpd-2.4.6-88.el7.centos.x86_64.rpm

最新资源