C++编程：并行与分布式系统实战指南

需积分: 9 117 浏览量更新于2024-07-20 收藏 5.94MB PDF 举报

"平行与分布式编程使用C++-Addison Wesley(2003).pdf" 本书《Parallel and Distributed Programming Using C++》由Cameron Hughes和Tracey Hughes合著，出版于2003年，是Addison Wesley出版社的一本720页的专业技术书籍，专注于利用C++进行并行和分布式编程。这本书不仅适合经验丰富的计算机程序员、软件开发者、设计师、研究人员和软件架构师，同时也适用于对计算机科学感兴趣的在校学生。书中的内容深入浅出地介绍了如何构建能够充分利用多处理器计算机的软件。作者提供了一种编程平行虚拟机的简单方法，并详细讲解了集群应用程序开发的基础知识。通过易于理解的多线程编程概述，读者可以学习到如何编写能够在网络上协同工作以解决问题和执行任务的软件组件。本书的核心知识点包括： 1. 并行编程基础：涵盖并行计算的基本概念，如进程与线程、同步与通信机制，以及并行算法设计策略，帮助读者理解和掌握并行程序的构建。 2. 平行虚拟机编程：介绍如何利用C++实现平行虚拟机，以提高程序的执行效率和资源利用率，同时减少程序复杂性。 3. 集群应用开发：阐述了在大规模分布式系统中构建应用程序的关键技术，如负载均衡、容错处理和分布式数据管理。 4. 多线程编程：详细讲解了C++中多线程的创建、管理及线程间的协作，包括线程安全、同步原语（如互斥量、条件变量等）的使用。 5. 网络编程：介绍网络通信的基础，包括套接字编程，以及如何设计能够跨网络协作的软件组件。 6. 软件架构：探讨并行和分布式系统的设计原则和最佳实践，强调可扩展性和可维护性。 7. 案例研究和实战练习：通过具体的实例和练习，帮助读者将理论知识应用于实际问题解决，增强实践经验。《Parallel and Distributed Programming Using C++》是一本全面而实用的教程，旨在帮助读者掌握并行和分布式编程的关键技能，从而在现代高性能计算领域取得成功。无论是对专业人士还是学习者，都能从中受益匪浅，提升自己在并行和分布式系统设计与实现方面的专业能力。

[ Team LiB ]

Acknowledgments

We could not have successfully pulled this project off without the help, suggestions, constructive

criticisms, and resources of many of our friends and colleagues. In particular, we would like to thank Terry

Lewis and Doug Johnson from OSC (Ohio Super-Computing) for providing us with complete access to a

32-node Linux-based cluster. To Mark Welton from YSU for his expertise and help with configuring the

cluster to support our PVM and MPI programs. To Sal Sanders from YSU for providing us with access to

Power-PCs running Mac OSX and Adobe Illustrator. To Brian Nelson from YSU for allowing us to test

many of our multithreaded and distributed programs on multiprocessor Sun E-250s and E-450s. We are

also indebted to Mary Ann Johnson and Jeffrey Trimble from YSU MAAG for helping us locate and hold

on to the technical references we required. Claudio M. Stanziola, Paulette Goldweber, and Jacqueline

Hansson from the IEEE Standards and Licensing and Contracts Office for obtaining permission to reprint

parts of the new Single-UNIX/POSIX standard; Andrew Josey and Gene Pierce from The Open Group

was also helpful in this regard. Thanks to Trevor Watkins of the Z-Group for all his help with the testing

of the program examples; his multi-Linux distribution environment was especially important in the testing

process. A special thanks to Steve Tarasweki for agreeing to provide a technical review for this book

while it was in its roughest form. To Dr. Eugene Santos for pointing us in the right direction as we

explored how categorical data structures could be used with PVMs. To Dr. Mike Crescimanno from the

Advanced Computing Work Group at YSU for allowing us to present some of the materials from this book

at one of the ACWG meetings. Finally, to Paul Petralia and the production team (especially Gail Cocker-

Bogusz) from Prentice Hall who had to put up with all of our missed deadlines and strange UNIX/Linux

file formats—we are extremely indebted to their patience, encouragement, enthusiasm, and

professionalism.

[ Team LiB ]

Chapter 1. The Joys of Concurrent Programming

"I suspect that concurrency is best supported by a library and that such a library can be

implemented without major language extensions."

—Bjarne Stroustrup, inventor of C++

In this Chapter

What is Concurrency?

The Benefits of Parallel Programming

The Benefits of Distributed Programming

The Minimal Effort Required

The Basic Layers of Software Concurrency

No Keyword Support for Parallelism in C++

Programming Environments for Parallel and Distributed Programming

Summary—Toward Concurrency

The software development process now requires a working knowledge of parallel and distributed

programming. The requirement for a piece of software to work properly over the Internet, on an intranet,

or over some network is almost universal. Once the piece of software is deployed in one or more of these

environments it is subjected to the most rigorous of performance demands. The user wants

instantaneous and reliable results. In many situations the user wants the software to satisfy many

requests at the same time. The capability to perform multiple simultaneous downloads of software and

data from the Internet is a typical expectation of the user. Software designed to broadcast video must also

be able to render graphics and digitally process sound seamlessly and without interruption. Web server

software is often subjected to hundreds of thousands of hits per day. It is not uncommon for frequently

used e-mail servers to be forced to survive the stress of a million sent and received messages during

business hours. And it's not just the quantity of the messages that can require tremendous work, it's also

the content. For instance, data transmissions containing digitized music, movies, or graphics devour

network bandwidth and can inflict a serious penalty on server software that has not been properly

designed. The typical computing environment is networked and the computers involved have multiple

processors. The more the software does, the more it is required to do. To meet the minimal user's

requirements, today's software must work harder and smarter. Software must be designed to take

advantage of computers that have multiple processors. Since networked computers are more the rule

than the exception, software must be designed to correctly and effectively run, with some of its pieces

executing simultaneously on different computers. In some cases, the different computers have totally

different operating systems with different network protocols! To accommodate these realities, a software

development repertoire must include techniques for implementing concurrency through parallel and

distributed programming.

[ Team LiB ]

1.1 What is Concurrency?

Two events are said to be concurrent if they occur within the same time interval. Two or more tasks

executing over the same time interval are said to execute concurrently. For our purposes, concurrent

doesn't necessarily mean at the same exact instant. For example, two tasks may occur concurrently within

the same second but with each task executing within different fractions of the second. The first task may

execute for the first tenth of the second and pause, the second task may execute for the next tenth of the

second and pause, the first task may start again executing in the third tenth of a second, and so on. Each

task may alternate executing. However, the length of a second is so short that it appears that both tasks

are executing simultaneously. We may extend this notion to longer time intervals. Two programs

performing some task within the same hour continuously make progress of the task during that hour,

although they may or may not be executing at the same exact instant. We say that the two programs are

executing concurrently for that hour. Tasks that exist at the same time and perform in the same time

period are concurrent. Concurrent tasks can execute in a single or multiprocessing environment. In a

single processing environment, concurrent tasks exist at the same time and execute within the same time

period by context switching. In a multiprocessor environment, if enough processors are free, concurrent

tasks may execute at the same instant over the same time period. The determining factor for what makes

an acceptable time period for concurrency is relative to the application.

Concurrency techniques are used to allow a computer program to do more work over the same time

period or time interval. Rather than designing the program to do one task at a time, the program is broken

down in such a way that some of the tasks can be executed concurrently. In some situations, doing more

work over the same time period is not the goal. Rather, simplifying the programming solution is the goal.

Sometimes it makes more sense to think of the solution to the problem as a set of concurrently executed

tasks. For instance, the solution to the problem of losing weight is best thought of as concurrently

executed tasks: diet and exercise. That is, the improved diet and exercise regimen are supposed to occur

over the same time interval (not necessarily at the same instant ). It is typically not very beneficial to do

one during one time period and the other within a totally different time period. The concurrency of both

processes is the natural form of the solution. Sometimes concurrency is used to make software faster or

get done with its work sooner. Sometimes concurrency is used to make software do more work over the

same interval where speed is secondary to capacity. For instance, some web sites want customers to

stay logged on as long as possible. So it's not how fast they can get the customers on and off of the site

that is the concern—it's how many customers the site can support concurrently. So the goal of the

software design is to handle as many connections as possible for as long a time period as possible.

Finally, concurrency can be used to make the software simpler. Often, one long, complicated sequence of

operations can be implemented easier as a series of small, concurrently executing operations. Whether

concurrency is used to make the software faster, handle larger loads, or simplify the programming

solution, the main object is software improvement using concurrency to make the software better.

1.1.1 The Two Basic Approaches to Achieving Concurrency

Parallel programming and distributed programming are two basic approaches for achieving concurrency

with a piece of software. They are two different programming paradigms that sometimes intersect.

Parallel programming techniques assign the work a program has to do to two or more processors within

a single physical or a single virtual computer. Distributed programming techniques assign the work a

program has to do to two or more processes—where the processes may or may not exist on the same

computer. That is, the parts of a distributed program often run on different computers connected by a

network or at least in different processes. A program that contains parallelism executes on the same

physical or virtual computer. The parallelism within a program may be divided into processes or threads.

We discuss processes in Chapter 3 and threads in Chapter 4. For our purposes, distributed programs

can only be divided into processes. Multithreading is restricted to parallelism. Technically, parallel

programs are sometimes distributed, as is the case with PVM (Parallel Virtual Machine) programming.

Distributed programming is sometimes used to implement parallelism, as is the case with MPI (Message

Passing Interface) programming. However, not all distributed programs involve parallelism. The parts of a

distributed program may execute at different instances and over different time periods. For instance, a

[ Team LiB ]

1.2 The Benefits of Parallel Programming

Programs that are properly designed to take advantage of parallelism can execute faster than their

sequential counterparts, which is a market advantage. In other cases the speed is used to save lives. In

these cases faster equates to better. The solutions to certain problems are represented more naturally as

a collection of simultaneously executing tasks. This is especially the case in many areas of scientific,

mathematical, and artificial intelligence programming. This means that parallel programming techniques

can save the software developer work in some situations by allowing the developer to directly implement

data structures, algorithms, and heuristics developed by researchers. Specialized hardware can be

exploited. For instance, in high-end multimedia programs the logic can be distributed to specialized

processors for increased performance, such as specialized graphics chips, digital sound processors, and

specialized math processors. These processors can usually be accessed simultaneously. Computers with

MPP (Massively Parallel Processors) have hundreds, sometimes thousands of processors and can be

used to solve problems that simply cannot realistically be solved using sequential methods. With MPP

computers, it's the combination of fast with pure brute force that makes the impossible possible. In this

category would fall environmental modeling, space exploration, and several areas in biological research

such as the Human Genome Project. Further parallel programming techniques open the door to certain

software architectures that are specifically designed for parallel environments. For example, there are

certain multiagent and blackboard architectures designed specifically for a parallel processor

environment.

1.2.1 The Simplest Parallel Model (PRAM)

The easiest method for approaching the basic concepts in parallel programming is through the use of the

PRAM (Parallel Random Access Machine). The PRAM is a simplified theoretical model where there are

n processors labeled as P

, P

, ... P

and each processor shares one global memory. Figure 1-2

shows a simple PRAM.

Figure 1-2. A Simple PRAM.

All the processors have read and write access to a shared global memory. In the PRAM the access can

be simultaneous. The assumption is that each processor can perform various arithmetic and logical

operations in parallel. Also, each of the theoretical processors in Figure 1-2 can access the global shared

memory in one uninterruptible unit of time. The PRAM model has both concurrent and exclusive read

algorithms. Concurrent read algorithms are allowed to read the same piece of memory simultaneously

with no data corruption. Exclusive read algorithms are used to ensure that no two processors ever read

the same memory location at the same time. The PRAM model also has both concurrent and exclusive

write algorithms. Concurrent write algorithms allow multiple processors to write to memory, while

exclusive write algorithms ensure that no two processors write to the same memory at the same time.

Table 1-1 shows the four basic types of algorithms that can be derived from the read and write

possibilities.

Table 1-1. Four Basic Read-Write Algorithms

Read-Write Algorithm Type Meaning

EREW Exclusive read exclusive write

剩余511页未读，继续阅读

hopyGreat

粉丝: 56
资源: 16

C++编程：并行与分布式系统实战指南

Parallel and Distributed Programming Using C++

Parallel.and.Distributed.Programming using c++

Parallel and Distributed Programming Using C++.

帮我写一个c++程序，要求如下：esim_tool --model=<model.bin> --input=<ifmap.bin> --output=<ofmap.bin> --infer_order=<depthfirst|breadthfirst|random|parallel>，要保持120的行宽，并且将命令行参数解析使用getopt_long单独封装成一个函数

$BWA mem -t $THREADS -R "@RG\tID:re\tSM:re\tLB:lib1" ref-100-2.fa ref_1.fq > ref.sam

近三年关于Java的外文带页码的期刊参考文献

if cuda and rank == -1 and torch.cuda.device_count() > 1

nmap -v scanme.nmap.org 扫描与 nmap -vv scanme.nmap.org 扫描的区别是什么

基于光场驱动的显微操作发展的参考文献

idea设置时间的重定向

最新资源