《编程并行处理器实战指南》第二版——掌握大规模并行计算

5星 · 超过95%的资源需积分: 9 118 浏览量更新于2024-07-24 4 收藏 4.19MB PDF 举报

"《Programming Massively Parallel Processors 2nd Edition》是一本关于大规模并行计算的权威指南，由David B. Kirk和Wen-mei W. Hwu合著，是平行计算领域的畅销书籍，拥有广泛的读者基础。本书旨在提供实践性的并行编程经验，特别关注CUDA、MPI、OpenMP和OpenCL等技术。" 本书第二版深入浅出地介绍了并行计算的核心概念，包括并行计算架构、多核心处理器和GPU的工作原理，以及如何有效地利用这些资源进行高性能计算。作者们通过实际案例和练习，引导读者掌握并行编程的技巧和最佳实践。首先，CUDA（Compute Unified Device Architecture）是NVIDIA推出的一种用于GPU编程的并行计算平台。书中详细讲解了CUDA编程模型，包括线程块、网格和共享内存的概念，以及如何利用CUDA C/C++进行高效的数据并行和任务并行编程。此外，还讨论了CUDA的性能优化策略，如减少数据传输和最大化内存效率。其次，MPI（Message Passing Interface）是分布式内存系统中的通信库，用于构建大规模并行应用。书中介绍了MPI的基本操作，如点对点通信、集合通信和进程管理，并通过实例展示了如何在MPI程序中实现并行计算。 OpenMP是一种在共享内存系统上的并行编程标准，支持多线程编程。书中涵盖了OpenMP的API，包括并行化指令、同步原语、工作共享构造和数据亲和性等主题，帮助读者理解和编写高效的OpenMP程序。最后，OpenCL是一个开放的并行计算框架，支持多种硬件平台，包括CPU和GPU。书中详细介绍了OpenCL的上下文、队列、内核和缓冲区等概念，以及如何编写跨平台的并行程序。此外，本书还讨论了并行算法设计、负载平衡、错误处理和并行性能分析等关键话题。它不仅适合初学者入门，并行计算，也为有经验的开发者提供了深入的洞见和新的挑战。《Programming Massively Parallel Processors 2nd Edition》是学习和提升并行编程技能不可或缺的参考资料。

A Three-Phased Approach

In ECE498AL the lectures and programming assignments are balanced with each

other and organized into three phases:

Phase 1: One lecture based on Chapter 3 is dedicated to teaching the basic

CUDA memory/threading model, the CUDA extensions to the C language,

and the basic programming/debugging tools. After the lecture, students can

write a simple vector addition code in a couple of hours. This is followed by a

series of four lectures that give students the conceptual understanding of the

CUDA memory model, the CUDA thread execution model, GPU hardware

performance features, and modern computer system architecture. These

lectures are based on Chapters 4, 5, and 6.

Phase 2: A series of lec tures covers floating-point considerations in parallel

computing and common data-parallel programming patterns needed to develop

a high-performance parallel application. These lectures are based on Chapters

710. The performance of their matrix multiplication codes increases by

about 10 times through this period. The students also complete assignments on

convolution, vector reduction, and prefix sum through this period.

Phase 3: Once the students have established solid CUDA programming skills,

the remaining lectures cover application case studies, computational thinking,

a broader range of parallel execution models, and parallel programming

principles. These lectures are based on Chapters 1120. (The voice and video

recordings of these lectures are available online at the ECE408 web site:

http://courses.engr.illinois.edu/ece408/.

Tying It All Together: The Final Project

While the lectures, labs, and chapters of this book help lay the intellectual foun-

dation for the students, what brings the lea rning experience together is the final

project. The final project is so important to the full-semester course that it is

prominently positioned in the course and commands nearly two months’ focus. It

incorporates five innovative aspects: mentoring, workshop, clinic, final report,

and symposium. (While much of the information about the final project is avail-

able at the ECE408 web site, we would like to offer the thinking that was behind

the design of these aspects.)

Students are encouraged to base their final projects on problems that represent

current challe nges in the research community. To seed the process, the instructors

should recruit several computational science research groups to propose problems

and serve as mentors. The mentors are asked to contribute a one- to two-page

project specification sheet that briefly describes the significance of the applica-

tion, what the mentor would like to accomplish with the student teams on the

application, the technical skills (particular type of math, physics, or chemistry

courses) required to understand and work on the application, and a list of web

xvPreface

and traditional reso urces that students can draw upon for technical background,

general information, and building blocks, along with specific URLs or FTP paths

to particular implementations and coding examples. These project specification

sheets also provide students with learning experiences in defining their own

research projects later in their careers. (Several examples are available at the

ECE408 course web site.)

Students are also encouraged to contact their potential mentors during their

project selection process. Once the students and the mentors agree on a project,

they enter into a close relationship, featuring frequent consultation and project

reporting. The instructors should attempt to facilitat e the collaborative relation-

ship between students and their mentors, making it a very valuable experience for

both mentors and students.

Project Workshop

The main vehicle for the whole class to contribute to each other’s final project

ideas is the project workshop. We usually dedicate six of the lecture slots to project

workshops. The workshops are designed for students’ benefit. For example, if a

student has identified a project, the workshop serves as a venue to present prelimi-

nary thinking, get feedback, and recruit tea mmates. If a student has not identified a

project, he or she can simply attend the presentations, participate in the discussions,

and join one of the project teams. Students are not grade d during the workshops, to

keep the atmosphere nonthreatening and enable them to focus on a meaningful dia-

log with the instructors, teaching assistants, and the rest of the class.

The workshop schedule is designed so the instructors and teaching assistants

can take some ti me to provide feedback to the project tea ms and so that students

can ask questions. Presentations are limited to 10 minutes so there is time for

feedback and questions during the class period. This limits the class size to about

36 presenters, assuming 90-minute lecture slots. All presentations are preloaded

into a PC to control the schedule strictly and maximize feedback time. Since not

all students present at the workshop, we have been able to accommodate up to 50

students in each class, with extra workshop time available as needed.

The instru ctors and teaching assistants must make a commitment to attend all

the presentations and to give useful feedback. Students typically need the most

help in answering the following questions: (1) Are the projects too big or too

small for the amount of time available? (2) Is there existing work in the field that

the project can benefit from? (3) Are the computations being targeted for parallel

execution appropriate for the CUDA programming model?

Design Document

Once the students decide on a project and form a team, they are required to sub-

mit a design document for the project. This helps them think through the project

steps before they jump into it. The ability to do such planning will be important

to their later career success. The design document should discuss the background

xvi Preface

and motivation for the project, application-level objectives and potential impact,

main features of the end application, an overview of their design, an implementa-

tion plan, their performance goals, a verification plan and acceptance test, and a

project schedule.

The teaching assistants hold a project clinic for final project teams during the

week before the class symposium. This clinic helps ensure that students are on

track and that they have identified the potential roadblocks early in the process.

Student teams are asked to come to the clinic with an initial draft of the following

three versions of their application: (1) the best CPU sequential code in terms of

performance, with SSE2 and other optimizations that establish a strong serial base

of the code for their speedup comparisons and (2) the best CUDA parallel code in

terms of performance—this version is the main output of the project. This version

is used by the students to char acterize the parallel algorithm overhead in terms of

extra computations involved.

Student teams are asked to be prepared to discuss the key ideas used in each

version of the code, any floating-point numerical issues, any comparison against

previous results on the application, and the potential impact on the field if they

achieve tremendous speedup. From our experience, the optimal schedule for the

clinic is one week before the class symposium. An earlier time typically results in

less mature projects and less meaningful sessions. A later time will not give stu-

dents sufficient time to revise their projects according to the feedback.

Project Report

Students are required to submit a project report on their team’s key findings. Six

lecture slots are com bined into a whole-day class symposium. During the sympo-

sium, students use presentation slots proportional to the size of the teams. During

the presentation, the students highlight the best parts of their project report for the

benefit of the whole class. The presentation accounts for a significant part of stu-

dents’ grades. Each stud ent must answer questions directed to him or her as indivi-

duals, so that different grades can be assigned to individuals in the same team. We

have recorded these presentations for viewing by future students at the ECE408

web site. The symposium is a major opportunity for students to learn to produce a

concise presentation that motivates their peers to read a full paper. After their pre-

sentation, the students also submit a full report on their final project.

Online Supplements

The lab assignments, final project guidelines, and sample project specifications

are available to instructors who use this book for their classes. While this book

provides the intellectual contents for these classes, the additional material will be

crucial in achieving the overall education goals. We would like to invite you to

xviiPreface

Acknowledgements

There are so many people who have made special contributions to the second

edition. We would like to first thank the contributing authors of the new chapters.

Yuan Lin and Vinod Grover wrote the original draft of the OpenACC chapter.

Nathan Bell and Jared Hoberock wrote the original draft of the Thrust chapter,

with additional contributions on the foundational concepts from Chris Rodrigues.

Greg Ruetsch and Massimiliano Fatica wrote the original draft of the CUDA

FORTRAN chapter. David Callahan wrote the C11AMP Chapter. Isaac Gelado

wrote the original draft of the MPI-CUDA chapter. Brent Oster contributed to

base material and code examples of the Kepler chapter. Without the expertise and

contribution of these individuals, we would not have been able to cover these

new programming models with the level of insight that we wanted to provide to

our readers.

We would like to give special thanks to Izzat El Hajj, who tirelessly helped to

verify the code examples and improved the quality of illustrations and exercises.

We would like to especially acknowledge Ian Buck, the father of CUDA and

John Nickolls, the lead architect of Tesla GPU Computing Architecture. Their teams

laid an excellent infrastructure for this course. John passed away while we were

working on the second edition. We miss him dearly. Nadeem Mohammad organized

the NVIDIA review efforts and also contributed to Appendix B. Bill Bean, Simon

Green, Mark Harris, Nadeem Mohammad, Brent Oster, Peter Shirley, Eric Young

and Cyril Zeller provided review comments and corrections to the manuscripts.

Calisa Cole helped with cover. Nadeem’s heroic efforts have been critical to the

completion of this book.

We would like to especially thank Jensen Huang for providing a great amount

of financial and human resources for developing the course that laid the founda-

tion for this book. Tony Tamasi’s team contributed heavily to the review and

revision of the book chapters. Jensen also took the time to read the early drafts of

the chapters and gave us valuable feedback. David Luebke has facilitated the

GPU computing resources for the course. Jonah Alben has provided valuabl e

insight. Michael Shebanow and Michael Garland have given guest lectures and

offered materials.

John Stone and Sam Stone in Illinois contributed much of the base material for

the case study and OpenCL chapters. John Stratton and Chris Rodrigues contributed

some of the base material for the computational thinking chapter. I-Jui “Ray” Sung,

John Stratton, Xiao-Long Wu, Nady Obeid contributed to the lab material and

helped to revise the course material as they volunteered to serve as teaching assis-

tants on top of their research. Jeremy Enos worked tirelessly to ensure that students

have a stable, user-friendly GPU computing cluster to work on their lab assignments

and projects.

xix

剩余518页未读，继续阅读

mzh8675

粉丝: 0
资源: 2

《编程并行处理器实战指南》第二版——掌握大规模并行计算

GPU编程：掌握大规模并行处理器

GPU并行编程实战：掌握大规模并行处理器

深度学习CUDA：大规模并行处理器编程第二版

教材Programming Massively Parallel Processors 2nd Edition

Programming Massively Parallel Processors, Second Edition

Programming Massively Parallel Processors

Programming Massively Parallel Processors - 2rd Edition

Programming Massively Parallel Processors.pdf

Programming Massively Parallel Proceprogramming massively parallel processors书籍

Programming Massively Parallel Processors - A Hands on Approach

最新资源