高性能计算现代系统与实践：Sterling & Anderson 著

需积分: 25 30 浏览量更新于2024-07-18 收藏 47.25MB PDF 举报

《高性能计算：现代系统与实践》(High Performance Computing: Modern Systems and Practices)是由托马斯·斯特林(Thomas Sterling)、马修·安德森(Matthew Anderson)和马切克·布罗多维奇(Maciej Brodowicz)合著，于2017年出版的一本专著。本书属于印第安纳大学信息科学、计算机科学与工程学院的著作，由摩根 Kaufmann出版社发行，是Elsevier集团的一部分。该书在高性能计算领域具有权威性，旨在介绍现代高性能计算系统的最新理论和技术实践。本书的封面由计算机科学先驱C. Gordon Bell作序，强调了高性能计算在当今科技领域的关键地位。版权方面，所有内容受到严格保护，未经Elsevier出版社书面许可，不得以任何形式复制或传播，包括但不限于影印、录音或任何信息存储和检索系统。读者可以在www.elsevier.com/permissions网站上获取关于版权许可政策和与版权管理机构如Copyright Clearance Center和Copyright Licensing Agency合作的详细信息。作为一本专业教材，书中内容涵盖了高性能计算的核心概念，包括但不限于并行处理、分布式计算、网格计算、云计算、数据并行算法设计、优化技术以及高级编程模型等。作者们分享了他们的专业知识和实践经验，强调随着研究的不断推进和新技术的涌现，高性能计算领域的知识和最佳实践在持续演变。书中可能会涉及的具体知识点包括： 1. **并行计算基础**：介绍了如何利用多核处理器、GPU和其他硬件资源来加速计算任务，通过多线程、多进程和向量指令集提高性能。 2. **分布式系统架构**：探讨了分布式计算环境中的节点间通信、数据分布和负载均衡策略，如Hadoop、Spark等框架的应用。 3. **网格计算**：讨论了通过网络连接的异构资源池，实现大规模计算任务的分布式执行，如GridFTP和Globus Toolkit。 4. **云计算和虚拟化**：分析云计算对高性能计算的影响，以及如何在云环境中部署和优化高性能计算工作负载。 5. **内存管理和优化**：关注内存层次结构、缓存策略以及如何减少内存访问延迟，提升计算效率。 6. **算法优化**：提供针对特定应用的优化算法，如矩阵运算、大规模数据分析等，以及并行算法的设计原则。 7. **编程模型与工具**：介绍了MPI（Message Passing Interface）、OpenMP、CUDA等并行编程模型，以及相关开发工具的使用。 8. **系统监控与调试**：讲述了如何有效地监控和诊断高性能计算系统的性能瓶颈，以及故障恢复和维护策略。 9. **案例研究与实践**：通过实际项目展示高性能计算的应用实例，如气象预报、生物信息学、物理学模拟等。《高性能计算：现代系统与实践》是一本深入浅出的指南，不仅适合研究人员、工程师和学生，也为企业提供了最新的技术和最佳实践，帮助他们在这个快速发展的领域保持竞争力。随着技术的不断发展，读者可以从中持续学习和更新自己的知识。

1.6 This Textbook as a Guide and Tool for the Student............................................................................38

1.7 Summary and Outcomes of Chapter 1............................................................................................... 39

1.8 Questions and Problems..................................................................................................................40

References ............................................................................................................................................41

Supercomputing, which means supercomputers and their application, is among the most important

developments of the modern age, with unequaled impact across a vast diversity of ﬁelds of inquiry

and practical effect. From the extremes of arcane sciences to the most immediate practical concerns,

supercomputers play an essential role in the progress and advancement of human capabilities, envi-

ronments, and understanding. No other single technology in the history of humanity has experienced a

similar rate of growth, even in its relatively short existence. Within the span of a single human lifetime,

supercomputers have expanded their ability to perform calculations by a factor of 10 trillion or 13 orders

of magnitude, and this is a conservative estimate. From less than a 1000 basic operations per second in

the late 1940s to today’s performance in excess of a 100 quadrillion ﬂoating-point operations per second

(over 100 petaﬂops), supercomputer speed has steadily improved by about a factor of 200 times every

decade through a series of advances in technology, architecture, programming methods, algorithms, and

system software (Fig. 1.1). High performance computing (HPC), synonymous with supercomputing, is a

principal means of exploration complementing empirical methods used for more than 2 millennia and

theory practiced in the age of enlightenment of the last 4 centuries. As the “third pillar” of investigation,

supercomputing enables new paths of inquiry, new techniques of design, and new methods of operating

process. Even discoveries correctly credited to other classes of tools and instrumentation, such as giant

telescopes or particle accelerators, require the use of supercomputers as well to produce their ﬁnal results

through data analysis (sometimes referred to as “big data”). It can be asserted that supercomputing

allows us to understand the past, to control the present, and in limited cases to predict the future.

The skills required to employ HPC are multiple and complex, while the means of acquiring such

skills to a sufﬁcient degree require potentially years of study and experience at least in normal practice.

FIGURE 1.1

The Titan petaﬂops machine fully deployed at Oak Ridge National Laboratory in 2013. It takes up more than

4000 sq ft and consumes approximately 8 MW of electrical power. It has a theoretical peak performance of over

27 petaﬂops and delivers 17.6 petaﬂops R

max

sustained performance for the highly parallel Linpack (HPL)

benchmark. This architecture includes Nvidia graphics processing unit accelerators.

Photo courtesy of Oak Ridge National Laboratory, US Dept. of Energy

2 CHAPTER 1 INTRODUCTION

This often means lengthy apprenticeships in research facilities in academia, industry, or national

laboratories. There are many books written to teach particular program ming languages; others describe

in detail the structures and instruction sets of computer architectures; and still others discuss system

software such as operating systems. But missing has been a single textbook that serves as an entry-

level presentation of all these elements and their interrelationships in one place, combined with

guided hands-on experience. This work, High Performance Computing, is developed as a carefully

crafted synthesis of relevant elements of related disciplines, all of which contribute in critical ways to

supercomputing and its use. This book presents the foundation concepts, in-depth relevant knowledge,

and detailed skills that together will give you a meaningful understanding of HPC and an initial set of

techniques to make you an effective, albeit incipient, practitioner in its use. Throughout this text the

best practices employed by the community are presented with training, so you learn to do, the how,

even as you are gaining understanding of the what and the why.

This textbook provides a comprehensive introduction to the ﬁeld of HPC. It is presented in a form

that will be both intellectually rewarding and practical in teaching useful basic skills. It combines

perspectives about supercomputing concepts, knowledge about supercomputers, and techniques for

using and programming supercomputers. But teaching a complex subject like HPC is challe nging, in

that just about everything is deﬁned in terms of and relates to everything else. Yet by the nature of

pedagogy, material must be presented in some sequential order. This ﬁrst chapter is a brief introductory

presentation of the essential elements of HPC to provide an overview of everything; a ﬁrst pass that

will allow successive in- depth chapters to be related to this broad context.

The chapter looks at the many facets which comprise HPC. The importance of the material is that it

provides a complete, albeit simpliﬁed, perspective of HPC so that more detailed discussion of speciﬁcs

can be understood within the full context. Because no piece makes sense without the others, almost all

areas are brieﬂy introduced in this chapter. To reinforce the interrelated broad-brush presentation of

issues, this chapter concludes with a history of the ﬁeld and its rapid evolution.

1.1 HIGH PERFORMANCE COMPUTING DISCIPLINES

As previously noted, HPC is really a collection of multiple interrelated disciplines, each providing an

important aspect of the total ﬁeld. To master HPC as a useful tool is to develop an understanding and

associated skills in each of these corresponding areas. These broad areas are described here, including

a formal deﬁnition of “high performance computing” that applies throughout the treatment of the

ﬁeld, end-user application problems that are the intended purpose of HPC across a wide range of

science, engineering, societal, and security domains, the core concept of perform ance which is the

distinguishing characteristic of HPC compared to other forms of computing, the hardware and

software components that make up an HPC system, environments, tools, application programming,

and the interfaces used. Each of these is presented in some detail in the following sections and together

form a major portion of the concepts, knowledge content, and skills comprising this textbook.

1.1.1 DEFINITION

HPC is a ﬁeld of endeavor that relates to all facets of technology, methodology, and application

associated with achieving the greatest computing capability possible at any point in time and technology.

It engages a class of electronic digital machines referred to as “supercomputers” to perform a wide array

1.1 HIGH PERFORMANCE COMPUTING DISCIPLINES 3

of computational problems or “applications” (alternatively “workloads”) as fast as is possible. The

action of performing an application on a supercomputer is widely termed “supercomputing” and is

synonymous with HPC.

1.1.2 APPLICATION PROGRAMS

The purpose of HPC is to derive answers to questions that cannot be adequately addressed alone

through means of empiricism, theory, or even widely available or accessible commercial computers

(e.g., enterprise servers). Historically supercomputers have been applied to science and engineering,

and the methodology has been described as the “third pillar of science” alongside and complementing

both experimentation (empiricism) and mathematics (theory). But the range of problems that super-

computers can tackle extends far beyond classical scientiﬁc and engineering studies to include

challenges in socioeconomics, big-data management and learning, process control, and national

security. An application, then, is both the problem to be solved and the body of “code” or collection of

ordered computing instructions that represent the means of solving the problem. The code is the means

by which the user conveys to the supercomputer how it is to perform the necessary computations to

achieve the objectives of the problem. The full set of code used is a “computer program” or just

“program”, and the person developing the application code is the “programmer”.

1.1.3 PERFORMANCE AND METRICS

While the notion of performance may be intuitive, it is not simple. There is no single measure of

performance that fully reﬂects all aspects of the quality of computer operation. A “metric” is a

quantiﬁable observable operational parameter of a supercomputer. Multiple perspectives and related

metrics are routinely applied to characterize the behavioral properties and capabilities of an HPC

system. Two basic measures are employed individually or in combination and in differing contexts to

formulate the values used to represent the quality of a supercompute r. These two fundamental

measures are “time” and “number of operations” performed, both under prescribed conditions.

For HPC the most widely used metric is “ﬂoating-point operations per second” or “ﬂops”. A

ﬂoating-point operation is an addition or multiplication of two real (or ﬂoating-point) numbers

represented in some machine-readable and manipulatable form. Because supercomputers are so

“powerful”, to describe their capability would require phrases like “a trillion or quadrillion operations

per second”. The ﬁeld adopts the same system of notation as science and engineering, using the

Greek preﬁxes kilo, mega, giga, tera, and peta to represent 1000, 1 million, 1 billion, 1 trillion, and 1

quadrillion, respectively. The ﬁrst supercomputers barely achieved 1 kiloﬂops (Kﬂops). Today’s fastest

supercomputer exhibits a peak performance in the order of 125 petaﬂops. The laptop computer upon

which this textbook was written has a peak performance of a few gigaﬂops. A supercomputer is

millions of times more powerful than a laptop by this metric.

The true capability of a supercomputer is its ability to perform real work, to achieve useful results

toward an end goal such as simulating a particular physical phenomenon (e.g., colliding neutron stars

to determine resulting electromagnetic burst signatures). A better measure than ﬂops is how long a

given problem takes to complete. But because there are literally thousands (millions?) of such

problems, this measure is not particularly useful broadly. Thus the HPC community selects speciﬁc

problems around which to standardize. Such standar dized application programs are “benchmarks”.

4 CHAPTER 1 INTRODUCTION

One particularly widely used supercomputer benchmark is “Linpack”, or more precisely the “highly

parallel Linpack” (HPL), which solves a set of linear equations in dense matrix form [1]. A benchmark

gives a means of comparative evaluation between two independent system s by measuring their

respective times to perform the same calculation. Thus a second way to measure performance is time

to completion of a ﬁxed problem. The HPC community has selected HPL as a means of ranking

supercomputers, as represented by the “Top 500 list” begun in 1993 (Fig. 1.2). But other benchmarks

are also employed to stress certain aspects of a supercom puter or represent a certain class of programs.

1.1.4 HIGH PERFORMA NCE COMPUTING SYSTEMS

The most visible aspect of the ﬁeld of HPC is the high performance computers, or simply super-

computers, themselves. Today these machines appear as rows upon rows of many racks taking up

thousands of square feet and consuming potentially multiple megawatts of electrical power. To be in

the presence of one (which often means to be literally inside it) offers a whole other experience in

terms of noise, rapidly shif ting temperature gradients, and many blink ing lights. Even the most staid

observer cannot help but be awestruck by the imp ressive massiveness of such systems, the engineering

by which they are achieved, the commitment they represent to the edges of computing capability, and

the problems only they can solve. And beyond what is visible even to the not-so-casual observer is the

infrastructure that supports the operation of these systems, much of which is below ﬂoors, in adjacent

rooms, and outside the building that houses the machine. The deployment of a state-of-the-art

supercomputer is truly a major engineering undertaking involving time, expense, and expertise, as

well as responsible management and maintenance throughout the lifetime of the system. And yet the

FIGURE 1.2

The evolution of the R

max

from the HPL benchmark for supercomputing systems in the Top 500 list since the list

began in 1993. The top line indicates the cumulative performance of all the computers in the list. The middle line

shows the performance of the number one computer in the list. The bottom line shows the performance of the last

computer in the list (number 500).

Image courtesy Erich Strohmaier

1.1 HIGH PERFORMANCE COMPUTING DISCIPLINES 5

visible, audible, and other sensory experiences barely reﬂect the true nature of the accomplishment

embodied by these machines. At the heart of the HPC system is the structure and organization of its

myriad com ponents and the semantics or rules by which they operate and perform the user applications

offered to them. Even more than the hardware, the HPC system is a vast array of software components

that control the hierarchy of the physical components and manage the user workloads. If the physical

hardware, racks and all, is what the visitor experiences, the system software, its interfaces, and

functionality are what the user experiences (usually at a location far from the physical machine) when

developing and running the applications and analyzing the results.

In one important sense, the high performance computer system has basic functionality and sub-

systems in common with the laptop personal computer upon whi ch this textbook was written. These

principal capabilities, shared by both extremes, include the following.

• The operational functions that transform input data values to output results.

• The internal memory that stores the data upon which the system operates.

• The communication channels through which intermediate data is transferred between different

components and subsystems during application execution.

• The control hardware that coordinates the interoperability among the constituent components and

subsystems.

• The mass storage that organizes and holds the persistent data, system software, and application

programs.

• The input/output (I/O) channels and interfaces (like the keyboard I am typing on and the screen I

am looking at) that connect users to the system.

Similarly, the software of an HPC system has much in common with the desk-side workstation or

departmental ente rprise server. Like these more pervasive albeit more modest computers, the super-

computer has a software structure that serves many of the same purposes of interface, control, and

functionality, including but not limited to the following.

• The operating system that manages all aspects of the machine and its operation.

• The compilers that translate application programs written in human-readable syntactic languages

(and other interfaces) to machine-readable binary code.

• File systems that present a logical abstraction of mass storage and organize the data on mass-

storage devices (like hard-disk drives).

• The myriad software drivers of the I/O devices by which the computer communicates with the

external world and users.

• The many tools that make up much of the expected user environments.

What distinguishes an HPC system from a conventional computer is the organization, inter-

connectivity, and scale of the component resources and the ability of the supporting software to

manage the operation of the system at that scale (Fig. 1.3). By scale is meant the degree of physical

and logical parallelism, i.e., the replication of key physical components such as processors and

memory banks and the delineation of a number of tasks to be performed simultaneously. While even a

single socket laptop incorporates some parallelism, an HPC system is structure d in far more levels,

each of which is usually much more substantial (but there are exceptions to this). It is this parallel

organization, the methods by which the constituent subsystems are coordinated to solve a shared

problem, and the additional functionality of the system software and programming models providing

6 CHAPTER 1 INTRODUCTION

剩余694页未读，继续阅读

u010652219

粉丝: 2
资源: 65

高性能计算现代系统与实践：Sterling & Anderson 著

High Performance Computing 无水印原版pdf

论文阅读 Containerisation for High Performance Computing Systems

Introduction to High Performance Computing for Scientists and Engineers

High Performance Computing for Big Data Methodologies and Applications epub

High Performance Computing for Big Data Methodologies and Applications 无水印pdf

High Performance Computing for Big Data Methodologies and Applications azw3

New Frontiers in High Performance Computing and Big Data 无水印原版pdf

Introduction to High Performance Computing

Tools for High Performance Computing

HPCStack: A High Performance Computing Cloud based on KVM and OpenStack

最新资源