xvi Preface
be expected from the exciting new developments in multicore and many-core
hardware.
Chapter 2 is all about the methodology and the design patterns that can be
employed in the development of parallel and multicore software. Both work
decomposition patterns and program structure patterns are examined.
• Shared-memory programming: Two different approaches for shared-memory
parallel programming are examined: explicit and implicit parallelization. On the
explicit side, Chapter 3 covers threads and two of the most commonly used
synchronization mechanisms, semaphores and monitors. Frequently
encountered design patterns, such as producers-consumers and readers-writers,
are explained thoroughly and applied in a range of examples.
On the implicit side, Chapter 4 covers the OpenMP standard that has been
specically designed for parallelizing existing sequential code with minimum
effort. Development time is signicantly reduced as a result. There are still
complications, such as loop-carried dependencies, which are also addressed.
• Distributed memory programming: Chapter 5 introduces the de facto standard
for distributed memory parallel programming, i.e., the Message Passing
Interface (MPI). MPI is relevant to multicore programming as it is designed to
scale from a shared-memory multicore machine to a million-node
supercomputer. As such, MPI provides the foundation for utilizing multiple
disjoint multicore machines, as a single virtual platform.
The features that are covered include both point-to-point and collective
communication, as well as one-sided communication. A section is dedicated to
the Boost.MPI library, as it does simplify the proceedings of using MPI,
although it is not yet feature-complete.
• GPU programming: GPUs are one of the primary reasons why this book was put
together. In a similar fashion to shared-memory programming, we examine the
problem of developing GPU-specic software from two perspectives: on one
hand we have the “nuts-and-bolts” approach of Nvidia’s CUDA, where memory
transfers, data placement, and thread execution conguration have to be
carefully planned. CUDA is examined in Chapter 6.
On the other hand, we have the high-level, algorithmic approach of the Thrust
template library, which is covered in Chapter 7. The STL-like approach to
program design affords Thrust the ability to target both CPUs and GPU
platforms, a unique feature among the tools we cover.
• Load balancing : Chapter 8 is dedicated to an often under-estimated aspect of
multicore development. In general, load balancing has to be seriously
considered once heterogeneous computing resources come into play. For
example, a CPU and a GPU constitute such a set of resources, so we should not
think only of clusters of dissimilar machines as tting this requirement. Chapter
8 briey discusses the Linda coordination language, which can be considered a
high-level abstraction of dynamic load balancing.
The main focus is on static load balancing and the mathematical models that
can be used to drive load partitioning and data communication sequences.
评论0