并行编程的挑战与应对策略

需积分: 10 164 浏览量更新于2024-07-15 收藏 7.15MB PDF 举报

身份认证购VIP最低享 7 折!

30元优惠券

"Is Parallel Programming Hard [v20191222a # Paul E. McKenney]" 是一本由Paul E. McKenney编辑的文档，主要探讨了并行编程的挑战以及如何应对这些问题。该文档发布于2019年12月22日，版本号为2019.12.22a。McKenney在Facebook上以paulmck@kernel.org的身份活动，并且可能与Linux社区有密切关联，因为文档中提到了Linux这一商标。文档内容可能涉及以下几个关键知识点： 1. **并行计算的复杂性**：并行编程通常比串行编程更为复杂，因为它涉及到多线程、多进程间的协调，以及数据同步和通信问题。作者可能深入讨论了这些挑战，包括死锁、竞态条件、活锁和资源争用等常见问题。 2. **并行编程模型**：文档可能介绍了不同的并行编程模型，如共享内存模型（如OpenMP）、分布式内存模型（如MPI）和数据流模型。对于每种模型，作者可能都阐述了其优点、缺点以及适用场景。 3. **Linux环境下的并行编程**：由于文档标签中提及了Linux，因此可能包含与Linux内核相关的并行编程技术，如POSIX线程（Pthreads）、Kernel-level Thread（Kthreads）以及Linux系统下的并行性能优化。 4. **硬件平台和架构**：文档可能涵盖了不同硬件平台对并行编程的影响，如IBM的zSystems和PowerPC架构，以及Intel的i386（x86）架构。这些平台的并行处理能力、指令集和内存管理可能都是讨论的重点。 5. **开源许可**：文档遵循了Creative Commons Attribution-ShareAlike 3.0 United States许可，意味着读者可以自由使用、修改和翻译内容，只要保持原作者的署名权。 6. **实践策略**：作者可能提供了实际的并行编程技巧和最佳实践，帮助开发者避免常见的陷阱，提高并行程序的效率和可维护性。 7. **案例研究或示例**：为了阐明理论概念，文档可能包含了一些实际的代码片段或案例研究，展示了如何在特定情况下解决并行编程问题。 "Is Parallel Programming Hard"是一份深入探讨并行编程复杂性和解决方案的资源，对于想要提升在Linux环境下进行并行计算技能的开发者来说，是一份宝贵的学习材料。通过理解和应用文档中的知识，读者将能更好地理解和解决并行编程中遇到的难题，从而实现更高效、可靠的并行程序设计。

资源详情

资源推荐

4 CHAPTER 1. HOW TO USE THIS BOOK

find CodeSamples -name rcu_rcpls.c -print

This command will locate the ﬁle

rcu_rcpls.c

, which

is called out in Appendix B. Other types of systems have

well-known ways of locating ﬁles by ﬁlename.

1.5 Whose Book Is This?

If you become a teacher, by your pupils you’ll be

taught.

Oscar Hammerstein II

As the cover says, the editor is one Paul E. McKenney.

However, the editor does accept contributions via the

perfbook@vger.kernel.org

email list. These contri-

butions can be in pretty much any form, with popular

approaches including text emails, patches against the

book’s L

X source, and even

git pull

requests. Use

whatever form works best for you.

To create patches or

git pull

requests, you will

need the L

X source to the book, which is at

git://git.kernel.org/pub/scm/linux/kernel/

git/paulmck/perfbook.git

. You will of course also

need

git

and L

X, which are available as part of most

mainstream Linux distributions. Other packages may be

required, depending on the distribution you use. The

required list of packages for a few popular distributions is

listed in the ﬁle

FAQ-BUILD.txt

in the L

X source to

the book.

To create and display a current L

X source tree of this

book, use the list of Linux commands shown in Listing 1.1.

In some environments, the

evince

command that displays

perfbook.pdf

may need to be replaced, for example,

with

acroread

. The

git clone

command need only be

used the ﬁrst time you create a PDF, subsequently, you

can run the commands shown in Listing 1.2 to pull in any

updates and generate an updated PDF. The commands in

Listing 1.2 must be run within the

perfbook

directory

created by the commands shown in Listing 1.1.

PDFs of this book are sporadically posted at

http://kernel.org/pub/linux/kernel/people/

paulmck/perfbook/perfbook.html

and at

http:

//www.rdrop.com/users/paulmck/perfbook/.

The actual process of contributing patches and send-

ing

git pull

requests is similar to that of the Linux

kernel, which is documented in the

Documentation/

SubmittingPatches

ﬁle in the Linux source tree. One

important requirement is that each patch (or commit, in

the case of a

git pull

request) must contain a valid

Signed-off-by: line, which has the following format:

Signed-off-by: My Name <myname@example.org>

Please see

http://lkml.org/lkml/2007/1/15/

219

for an example patch containing a

Signed-off-by:

line.

It is important to note that the

Signed-off-by:

line

has a very speciﬁc meaning, namely that you are certifying

that:

(a)

The contribution was created in whole or in part by

me and I have the right to submit it under the open

source license indicated in the ﬁle; or

(b)

The contribution is based upon previous work that, to

the best of my knowledge, is covered under an appro-

priate open source License and I have the right under

that license to submit that work with modiﬁcations,

whether created in whole or in part by me, under the

same open source license (unless I am permitted to

submit under a diﬀerent license), as indicated in the

ﬁle; or

(c)

The contribution was provided directly to me by

some other person who certiﬁed (a), (b) or (c) and I

have not modiﬁed it.

(d)

I understand and agree that this project and the contri-

bution are public and that a record of the contribution

(including all personal information I submit with it,

including my sign-oﬀ) is maintained indeﬁnitely and

may be redistributed consistent with this project or

the open source license(s) involved.

This is quite similar to the Developer’s Certiﬁcate

of Origin (DCO) 1.1 used by the Linux kernel. You

must use your real name: I unfortunately cannot accept

pseudonymous or anonymous contributions.

The language of this book is American English, however,

the open-source nature of this book permits translations,

and I personally encourage them. The open-source li-

censes covering this book additionally allow you to sell

your translation, if you wish. I do request that you send

me a copy of the translation (hardcopy if available), but

this is a request made as a professional courtesy, and

is not in any way a prerequisite to the permission that

you already have under the Creative Commons and GPL

licenses. Please see the

FAQ.txt

ﬁle in the source tree

If parallel programming is so hard, why are there so

many parallel programs?

Unknown

Chapter 2

Introduction

Parallel programming has earned a reputation as one of

the most diﬃcult areas a hacker can tackle. Papers and

textbooks warn of the perils of deadlock, livelock, race

conditions, non-determinism, Amdahl’s-Law limits to

scaling, and excessive realtime latencies. And these perils

are quite real; we authors have accumulated uncounted

years of experience dealing with them, and all of the

emotional scars, grey hairs, and hair loss that go with such

experiences.

However, new technologies that are diﬃcult to use at

introduction invariably become easier over time. For

example, the once-rare ability to drive a car is now com-

monplace in many countries. This dramatic change came

about for two basic reasons: (1) cars became cheaper

and more readily available, so that more people had the

opportunity to learn to drive, and (2) cars became easier to

operate due to automatic transmissions, automatic chokes,

automatic starters, greatly improved reliability, and a host

of other technological improvements.

The same is true for many other technologies, includ-

ing computers. It is no longer necessary to operate a

keypunch in order to program. Spreadsheets allow most

non-programmers to get results from their computers that

would have required a team of specialists a few decades

ago. Perhaps the most compelling example is web-surﬁng

and content creation, which since the early 2000s has been

easily done by untrained, uneducated people using various

now-commonplace social-networking tools. As recently

as 1968, such content creation was a far-out research

project [

Eng68

], described at the time as “like a UFO

landing on the White House lawn”[Gri00].

Therefore, if you wish to argue that parallel program-

ming will remain as diﬃcult as it is currently perceived

by many to be, it is you who bears the burden of proof,

keeping in mind the many centuries of counter-examples

in a variety of ﬁelds of endeavor.

2.1 Historic Parallel Programming

Diﬃculties

Not the power to remember, but its very opposite, the

power to forget, is a necessary condition for our

existence.

Sholem Asch

As indicated by its title, this book takes a diﬀerent ap-

proach. Rather than complain about the diﬃculty of

parallel programming, it instead examines the reasons

why parallel programming is diﬃcult, and then works to

help the reader to overcome these diﬃculties. As will be

seen, these diﬃculties have fallen into several categories,

including:

The historic high cost and relative rarity of parallel

systems.

The typical researcher’s and practitioner’s lack of

experience with parallel systems.

3. The paucity of publicly accessible parallel code.

The lack of a widely understood engineering disci-

pline of parallel programming.

The high overhead of communication relative to that

of processing, even in tightly coupled shared-memory

computers.

Many of these historic diﬃculties are well on the way

to being overcome. First, over the past few decades,

the cost of parallel systems has decreased from many

multiples of that of a house to a fraction of that of a

bicycle, courtesy of Moore’s Law. Papers calling out the

advantages of multicore CPUs were published as early

8 CHAPTER 2. INTRODUCTION

as 1996 [

ONH

]. IBM introduced simultaneous multi-

threading into its high-end POWER family in 2000, and

multicore in 2001. Intel introduced hyperthreading into

its commodity Pentium line in November 2000, and both

AMD and Intel introduced dual-core CPUs in 2005. Sun

followed with the multicore/multi-threaded Niagara in late

2005. In fact, by 2008, it was becoming diﬃcult to ﬁnd a

single-CPU desktop system, with single-core CPUs being

relegated to netbooks and embedded devices. By 2012,

even smartphones were starting to sport multiple CPUs.

Second, the advent of low-cost and readily available

multicore systems means that the once-rare experience

of parallel programming is now available to almost all

researchers and practitioners. In fact, parallel systems are

now well within the budget of students and hobbyists. We

can therefore expect greatly increased levels of invention

and innovation surrounding parallel systems, and that

increased familiarity will over time make the once pro-

hibitively expensive ﬁeld of parallel programming much

more friendly and commonplace.

Third, in the 20

century, large systems of highly par-

allel software were almost always closely guarded propri-

etary secrets. In happy contrast, the 21

century has seen

numerous open-source (and thus publicly available) paral-

lel software projects, including the Linux kernel [

Tor03

database systems [

Pos08

MS08

], and message-passing

systems [

The08

Uni08a

]. This book will draw primarily

from the Linux kernel, but will provide much material

suitable for user-level applications.

Fourth, even though the large-scale parallel

program-

ming projects of the 1980s and 1990s were almost all

proprietary projects, these projects have seeded other

communities with a cadre of developers who understand

the engineering discipline required to develop production-

quality parallel code. A major purpose of this book is to

present this engineering discipline.

Unfortunately, the ﬁfth diﬃculty, the high cost of com-

munication relative to that of processing, remains largely

in force. Although this diﬃculty has been receiving in-

creasing attention during the new millennium, according

to Stephen Hawking, the ﬁnite speed of light and the

atomic nature of matter is likely to limit progress in this

area [

Gar07

Moo03

]. Fortunately, this diﬃculty has been

in force since the late 1980s, so that the aforementioned

engineering discipline has evolved practical and eﬀective

strategies for handling it. In addition, hardware designers

are increasingly aware of these issues, so perhaps future

hardware will be more friendly to parallel software as

discussed in Section 3.3.

Quick Quiz 2.1:

Come on now!!! Parallel program-

ming has been known to be exceedingly hard for many

decades. You seem to be hinting that it is not so hard.

What sort of game are you playing?

However, even though parallel programming might not

be as hard as is commonly advertised, it is often more

work than is sequential programming.

Quick Quiz 2.2:

How could parallel programming

ever be as easy as sequential programming?

It therefore makes sense to consider alternatives to

parallel programming. However, it is not possible to

reasonably consider parallel-programming alternatives

without understanding parallel-programming goals. This

topic is addressed in the next section.

2.2 Parallel Programming Goals

If you don’t know where you are going, you will end

up somewhere else.

Yogi Berra

The three major goals of parallel programming (over and

above those of sequential programming) are as follows:

1. Performance.

2. Productivity.

3. Generality.

Unfortunately, given the current state of the art, it is

possible to achieve at best two of these three goals for any

given parallel program. These three goals therefore form

the iron triangle of parallel programming, a triangle upon

which overly optimistic hopes all too often come to grief.

Quick Quiz 2.3:

Oh, really??? What about correctness,

maintainability, robustness, and so on?

Quick Quiz 2.4:

And if correctness, maintainability,

and robustness don’t make the list, why do productivity

and generality?

Quick Quiz 2.5:

Given that parallel programs are much

harder to prove correct than are sequential programs, again,

shouldn’t correctness really be on the list?

Quick Quiz 2.6: What about just having fun?

Each of these goals is elaborated upon in the following

sections.

Kudos to Michael Wong for naming the iron triangle.

剩余560页未读，继续阅读

CodeArhat

粉丝: 13
资源: 55

并行编程的挑战与应对策略

Is Parallel programming hard

C++ Memory Model Meets High-Update-Rate Data Structures - Paul E. McKenney

Is Parallel Programming Hard, And, If So, What Can You Do About It?

常见算法优化技巧有哪些？

AttributeError: 'DataParallel' object has no attribute 'K'

AttributeError: 'DataParallel' object has no attribute 'generator'

使用distributed的方式进行多显卡训练时，会造成输出的网络层名称前多出一个module.如何删去

linux系统编译找不到iomp5点库

请利用joblib将其改写为并行执行

parallel programming: concepts and practice pdf

torch.nn.modules.module.moduleattributeerror: 'dataparallel' object has no a

如何编写pg_bulkload的控制文件ctl？

基于lammps如何输出裂痕的长度随着拉伸的变化而变化的数据？脚本是什么？

torch.nn.parallel.DataParallel

groupby apply 加速

pragma omp parallel for

6. What is the State chart Diagram? Explain the following. a. Process b.Data Flows c. Actor d.Data Stores

TypeError: parallel is not a function

最新资源