![](https://csdnimg.cn/release/download_crawler_static/11413379/bg14.jpg)
8 CHAPTER 2. INTRODUCTION
2.
The typical researcher’s and practitioner’s lack of experience with parallel sys-
tems.
3. The paucity of publicly accessible parallel code.
4.
The lack of a widely understood engineering discipline of parallel programming.
5.
The high overhead of communication relative to that of processing, even in tightly
coupled shared-memory computers.
Many of these historic difficulties are well on the way to being overcome. First, over
the past few decades, the cost of parallel systems has decreased from many multiples of
that of a house to a fraction of that of a bicycle, courtesy of Moore’s Law. Papers calling
out the advantages of multicore CPUs were published as early as 1996 [
ONH
+
96
]. IBM
introduced simultaneous multi-threading into its high-end POWER family in 2000, and
multicore in 2001. Intel introduced hyperthreading into its commodity Pentium line
in November 2000, and both AMD and Intel introduced dual-core CPUs in 2005. Sun
followed with the multicore/multi-threaded Niagara in late 2005. In fact, by 2008, it
was becoming difficult to find a single-CPU desktop system, with single-core CPUs
being relegated to netbooks and embedded devices. By 2012, even smartphones were
starting to sport multiple CPUs.
Second, the advent of low-cost and readily available multicore systems means
that the once-rare experience of parallel programming is now available to almost all
researchers and practitioners. In fact, parallel systems are now well within the budget of
students and hobbyists. We can therefore expect greatly increased levels of invention
and innovation surrounding parallel systems, and that increased familiarity will over
time make the once prohibitively expensive field of parallel programming much more
friendly and commonplace.
Third, in the 20
th
century, large systems of highly parallel software were almost
always closely guarded proprietary secrets. In happy contrast, the 21
st
century has
seen numerous open-source (and thus publicly available) parallel software projects,
including the Linux kernel [
Tor03
], database systems [
Pos08
,
MS08
], and message-
passing systems [
The08
,
Uni08a
]. This book will draw primarily from the Linux kernel,
but will provide much material suitable for user-level applications.
Fourth, even though the large-scale parallel
-
programming projects of the 1980s and
1990s were almost all proprietary projects, these projects have seeded other communities
with a cadre of developers who understand the engineering discipline required to develop
production-quality parallel code. A major purpose of this book is to present this
engineering discipline.
Unfortunately, the fifth difficulty, the high cost of communication relative to that
of processing, remains largely in force. Although this difficulty has been receiving
increasing attention during the new millennium, according to Stephen Hawking, the
finite speed of light and the atomic nature of matter is likely to limit progress in this
area [
Gar07
,
Moo03
]. Fortunately, this difficulty has been in force since the late 1980s,
so that the aforementioned engineering discipline has evolved practical and effective
strategies for handling it. In addition, hardware designers are increasingly aware of
these issues, so perhaps future hardware will be more friendly to parallel software as
discussed in Section 3.3.
Quick Quiz 2.1:
Come on now!!! Parallel programming has been known to be
exceedingly hard for many decades. You seem to be hinting that it is not so hard. What
sort of game are you playing?