Intel多线程应用程序开发指南

3星 · 超过75%的资源需积分: 10 50 浏览量更新于2024-07-23 收藏 2.86MB PDF 举报

"Intel Guide for Developing Multithreaded Applications" 这篇Intel官方文档是为开发者提供的一份详尽指南，旨在帮助他们有效地开发多线程应用程序。多线程编程是一种技术，通过将任务分解到多个线程中，使得程序能同时执行多个独立的部分，从而提高效率，尤其是在多核处理器系统中。在并行编程中，正确地设计和实现多线程应用至关重要，因为这直接影响到程序的性能、可靠性和可维护性。文档中可能涵盖了以下几个关键知识点： 1. **并行计算基础**：解释并行计算的概念，包括进程与线程的区别，以及为何在现代计算环境中选择多线程作为优化性能的手段。 2. **线程创建与管理**：介绍如何在不同的编程语言（如C++、Java等）中创建和管理线程，包括线程的生命周期、同步机制（如互斥量、条件变量、信号量等）以及线程池的使用。 3. **数据共享与并发控制**：讨论了多线程环境下共享数据的挑战，如数据竞争和死锁问题，以及如何使用锁和其他同步原语来防止这些问题。 4. **并行算法设计**：探讨如何将串行算法转化为并行算法，包括任务划分、负载均衡和通信开销的考虑。 5. **性能分析与优化**：讲解如何使用性能分析工具来识别并优化瓶颈，以及如何调整线程数量以获得最佳性能。 6. **线程安全编程**：强调避免数据不一致性的重要性，讨论无锁编程、原子操作和内存模型等概念。 7. **异常处理**：在多线程环境中，异常处理需要特别注意，因为一个线程中的异常可能会对其他线程产生影响。 8. **平台特定的特性**：Intel的处理器通常包含特定的指令集和特性，如超线程技术，文档可能会介绍如何利用这些特性来提升多线程应用的性能。 9. **最佳实践和陷阱**：总结了在编写多线程应用时应遵循的最佳实践，以及常见的错误和陷阱，以避免潜在的问题。 10. **测试与调试**：提供多线程程序的测试策略和调试技巧，以确保代码的正确性和可靠性。这份Intel文档不仅仅是理论性的介绍，还可能包含实例代码和具体操作步骤，使得开发者能够将学到的知识应用到实际项目中。它对于任何想要利用多核处理器潜力的软件开发者来说，都是一份宝贵的参考资料。尽管Intel产品可能被特别提及，但文档中的许多原则和最佳实践适用于各种硬件和软件环境。

for (int i = 0; i < imx; i++)

ws[nv][k][j][i] = Process(w[nv][k][j][i]);

}

With any number other than five threads, parallelizing the outer loop will result in load imbalance

and idle threads. The inefficiency would be especially severe if the array dimensions imx, jmx,

and kmx are very large. Parallelizing one of the inner loops is a better option in this case.

Avoid the implicit barrier at the end of worksharing constructs when it is safe to do so. All

OpenMP worksharing constructs (for, sections, single) have an implicit barrier at the end of

the structured block. All threads must rendezvous at this barrier before execution can proceed.

Sometimes these barriers are unnecessary and negatively impact performance. Use the OpenMP

nowait clause to disable this barrier, as in the following example:

void processQuadArray (int imx, int jmx, int kmx,

double**** w, double**** ws)

{

#pragma omp parallel shared(w, ws)

{

int nv, k, j, i;

for (nv = 0; nv < 5; nv++)

for (k = 0; k < kmx; k++) // kmx is usually small

#pragma omp for shared(nv, k) nowait

for (j = 0; j < jmx; j++)

for (i = 0; i < imx; i++)

ws[nv][k][j][i] = Process(w[nv][k][j][i]);

}

Since the computations in the innermost loop are all independent, there is no reason for threads

to wait at the implicit barrier before going on to the next k iteration. If the amount of work per

iteration is unequal, the nowait clause allows threads to proceed with useful work rather than

sit idle at the implicit barrier.

If a loop has a loop-carried dependence that prevents the loop from being executed in parallel, it

may be possible to break up the body of the loop into separate loops than can be executed in

parallel. Such division of a loop body into two or more loops is known as ―loop fission‖. In the

following example, loop fission is performed on a loop with a dependence to create new loops

that can execute in parallel:

float *a, *b;

int i;

for (i = 1; i < N; i++) {

if (b[i] > 0.0)

a[i] = 2.0 * b[i];

else

a[i] = 2.0 * fabs(b[i]);

b[i] = a[i-1];

}

The assignment of elements within the a array are all independent, regardless of the sign of the

corresponding elements of b. Each assignment of an element in b is independent of any other

assignment, but depends on the completion of the assignment of the required element of a. Thus,

as written, the loop above cannot be parallelized.

By splitting the loop into the two independent operations, both of those operations can be

executed in parallel. For example, the Intel® Threading Building Blocks (Intel® TBB)

parallel_for algorithm can be used on each of the resulting loops as seen here:

float *a, *b;

parallel_for (1, N, 1,

[&](int i) {

if (b[i] > 0.0)

a[i] = 2.0 * b[i];

else

a[i] = 2.0 * fabs(b[i]);

});

parallel_for (1, N, 1,

[&](int i) {

b[i] = a[i-1];

});

The return of the first parallel_for call before execution of the second ensures that all the

updates to the a array have completed before the updates on the b array are started.

Another use of loop fission is to increase data locality. Consider the following sieve-like code :

for (i = 0; i < list_len; i++)

for (j = prime[i]; j < N; j += prime[i])

marked[j] = 1;

The outer loop selects the starting index and step of the inner loop from the prime array. The

inner loop then runs through the length of the marked array depositing a ‗1‘ value into the

chosen elements. If the marked array is large enough, the execution of the inner loop can evict

cache lines from the early elements of marked that will be needed on the subsequent iteration of

the outer loop. This behavior will lead to a poor cache hit rate in both serial and parallel versions

of the loop.

Through loop fission, the iterations of the inner loop can be broken into chunks that will better fit

into cache and reuse the cache lines once they have been brought in. To accomplish the fission

in this case, another loop is added to control the range executed over by the innermost loop:

for (k = 0; k < N; k += CHUNK_SIZE)

for (i = 0; i < list_len; i++) {

start = f(prime[i], k);

end = g(prime[i], k);

for (j = start; j < end; j += prime[i])

marked[j] = 1;

}

For each iteration of the outermost loop in the above code, the full set of iterations of the i-loop

will execute. From the selected element of the prime array, the start and end indices within the

chunk of the marked array (controlled by the outer loop) must be found. These computations

have been encapsulated within the f() and g() routines. Thus, the same chunk of marked will

be processed before the next one is processed. And, since the processing of each chunk is

independent of any other, the iteration of the outer loop can be made to run in parallel.

Merging nested loops to increase the iteration count is another optimization that may aid

effective parallelization of loop iterations. For example, consider the code on the left with two

nested loops having iteration counts of 23 and 1000, respectively. Since 23 is prime, there is no

way to evenly divide the outer loop iterations; also, 1000 iteration may not be enough work to

sufficiently minimize the overhead of threading only the inner loop. On the other hand, the loops

can be fused into a single loop with 23,000 iterations (as seen on the right), which could alleviate

the problems with parallelizing the original code.

#define N 23

#define M 1000

. . .

for (k = 0; k < N; k++)

for (j = 0; j < M; j++)

wn[k][j] = Work(w[k][j], k,

j);

#define N 23

#define M 1000

. . .

for (kj = 0; kj < N*M; kj++) {

k = kj / M;

j = kj % M;

wn [k][j] = Work(w[k][j], k,

j);

}

However, if the iteration variables are each used within the loop body (e.g., to index arrays), the

new loop counter must be translated back into the corresponding component values, which

creates additional overhead that the original algorithm did not have.

Fuse (or merge) loops with similar indices to improve granularity and data locality and to

minimize overhead when parallelizing. The first two loops in the left-hand example code can be

easily merged:

for (j = 0; j < N; j++)

a[j] = b[j] + c[j];

for (j = 0; j < N; j++)

d[j] = e[j] + f[j];

for (j = 5; j < N – 5; j++)

g[j] = d[j+1] + a[j+1];

for (j = 0; j < N; j++)

{

a[j] = b[j] + c[j];

d[j] = e[j] + f[j];

}

for (j = 5; j < N – 5; j++)

g[j] = d[j+1] + a[j+1];

Merging these loops increases the amount of work per iteration (i.e., granularity) and reduces

loop overhead. The third loop is not easily merged because its iteration count is different. More

important, however, a data dependence exists between the third loop and the previous two loops.

Use the OpenMP if clause to choose serial or parallel execution based on runtime information.

Sometimes the number of iterations in a loop cannot be determined until runtime. If there is a

negative performance impact for executing an OpenMP parallel region with multiple threads (e.g.,

a small number of iterations), specifying a minimum threshold will help maintain performance, as

in the following example:

#pragma omp parallel for if(N >= threshold)

for (i = 0; i < N; i++) { … }

Granularity and Parallel Performance

Abstract

One key to attaining good parallel performance is choosing the right granularity for the

application. Granularity is the amount of real work in the parallel task. If granularity is too fine,

then performance can suffer from communication overhead. If granularity is too coarse, then

performance can suffer from load imbalance. The goal is to determine the right granularity

(usually larger is better) for parallel tasks, while avoiding load imbalance and communication

overhead to achieve the best performance.

Background

The size of work in a single parallel task (granularity) of a multithreaded application greatly

affects its parallel performance. When decomposing an application for multithreading, one

approach is to logically

partition

the problem into as many parallel tasks as possible. Within the

parallel tasks, next determine the necessary

communication

in terms of shared data and

execution order. Since partitioning tasks, assigning the tasks to threads, and communicating

(sharing) data between tasks are not free operations, one often needs to

agglomerate

, or

combine partitions, to overcome these overheads and achieve the most efficient implementation.

The agglomeration step is the process of determining the best granularity for parallel tasks.

The granularity is often related to how balanced the work load is between threads. While it is

easier to balance the workload of a large number of smaller tasks, this may cause too much

parallel overhead in the form of communication, synchronization, etc. Therefore, one can reduce

parallel overhead by increasing the granularity (amount of work) within each task by combining

smaller tasks into a single task. Tools such as the Intel® Parallel Amplifier can help identify the

right granularity for an application.

The following examples demonstrate how to improve the performance of a parallel program by

decreasing the communication overhead and finding the right granularity for the threads. The

example used throughout this article is a prime-number counting algorithm that uses a simple

brute force test of all dividing each potential prime by all possible factors until a divisor is found

or the number is shown to be a prime. Because positive odd numbers can be computed by either

(4k+1) or (4k+3), for k ≥ 0, the code will also keep a count of the prime numbers that fall into

each form. The examples will count all of the prime numbers between 3 and 1 million.

The first variation of the code shows a parallel version using OpenMP*:

#pragma omp parallel

{ int j, limit, prime;

#pragma for schedule(dynamic, 1)

for(i = 3; i <= 1000000; i += 2) {

limit = (int) sqrt((float)i) + 1;

prime = 1; // assume number is prime

j = 3;

while (prime && (j <= limit)) {

if (i%j == 0) prime = 0;

j += 2;

}

if (prime) {

#pragma omp critical

{

剩余132页未读，继续阅读

拾月的枫叶

粉丝: 0
资源: 3

Intel多线程应用程序开发指南

Intel Guide for Developing Multithreaded Applications Part 2

Intel Guide for Developing Multithreaded Applications Part 1

Multithreaded Application

Threads v4.3 Multithreaded Application Model

A Simple Way To Use Asynchronous Call in Your Multithreaded Application

Deterministic Execution for Arbitrary Multithreaded Programs

multithreaded programming guide

Pthreads Primer: a guide to multithreaded programmingr

PThreads Primer: A Guide to Multithreaded Programming

Checkpointing for Multithreaded Programs-开源

最新资源