机器学习驱动编译器自动调优：性能优化新策略

需积分: 10 176 浏览量更新于2024-07-19 收藏 3.78MB PDF 举报

自动编译器调优是现代软件开发中的一个重要挑战，尤其是在多架构环境下。随着计算机硬件的多样性增加，程序员需要花费更多精力来适应不同的平台，并确保他们的代码在各种设备上运行高效。编译器优化在此过程中扮演关键角色，通过多层次的优化策略来针对应用程序的不同代码段进行定制化处理。然而，选择正确的优化组合及其应用顺序（即所谓的“阶段顺序问题”）对性能有着显著影响，这是一个长期存在的编译器研究难题。传统的方法依赖于编译器启发式算法来解决这个问题，但面对应用中的每个代码段优化选择的巨大复杂性，这些方法显得力不从心。近期的研究工作正转向利用机器学习技术来改进这一状况。《使用机器学习进行编译器自动调优》一书探讨了如何通过创新的方法识别出能最大化目标应用程序性能的编译器优化策略。作者们，来自加拿大多伦多大学和意大利米兰理工大学的专家，结合理论与实践，提出了将机器学习模型应用于编译器优化选择和阶段顺序决策的可能路径。书中可能包含的具体内容可能包括： 1. **机器学习基础应用**：介绍如何将监督学习、强化学习或深度学习等机器学习算法引入编译器优化领域，如使用监督学习建立性能预测模型，或者使用强化学习寻找最优的优化组合。 2. **特征工程**：解释如何从源代码和应用程序特性中提取有用的特征，作为机器学习模型的输入，以便更准确地预测不同优化的组合效果。 3. **模型训练与评估**：描述如何通过大量的实验数据训练模型，包括优化策略的效果数据，以及如何通过交叉验证等方法评估模型的性能。 4. **动态调整**：讨论如何实时或根据运行时环境变化动态调整优化策略，以进一步提升编译器的自适应性。 5. **案例研究**：通过实际的应用案例展示机器学习驱动的编译器优化在性能提升上的效果，可能涉及不同领域的软件，如科学计算、图形处理或Web服务。 6. **挑战与未来方向**：分析当前方法面临的挑战，如数据稀疏性、模型泛化能力以及如何处理不断变化的硬件环境，同时展望机器学习在编译器优化中的潜在发展趋势。《自动编译器调优使用机器学习》一书为了解决传统编译器优化的局限性提供了一种新颖且具有前瞻性的视角，有望在未来改善软件性能优化的效率和准确性。

Chapter 1

Background

Abstract Since the mid-1990s, researchers have been trying to use machine-

learning based approaches to solve a number of different compiler optimization prob-

lems. The techniques primarily enhance the quality of the obtained results and, more

importantly, make it feasible to tackle two main compiler optimization problems:

optimization selection (choosing which optimizations to use) and phase-ordering

(choosing the order of applying optimizations). The compiler optimization space

continues to grow due to the advancement of applications, increasing compiler opti-

mizations, and new target architectures. Generic optimization passes in compilers

cannot fully leverage newly introduced optimizations and, therefore, cannot keep

up with the pace of increasing options. This chapter summarizes and classiﬁes the

recent advances in using machine learning for the compiler optimization ﬁeld, par-

ticularly on the two major problems of (i) selecting the best optimizations and (ii) the

phase-ordering of optimizations. The chapter highlights the approaches taken, the

obtained results, the holistic comparisons among different approaches and ﬁnally,

the visionary path towards the near future.

1.1 Introduction

Recent developments in silicon production and fabrication led to the creation of

much faster computational units such as CPUs, GPUs, FPGAs, and similar devices

with different instruction set architectures (ISAs). Software (SW) programming par-

adigms including OpenMP, MPI, OpenCL, and OpenACC allow software developers

to exploit Hardware (HW) parallelism to port l egacy serial codes on these emerging

platforms to attain application speedups. Compilers struggle to keep up with the

increasing development pace of ever-expanding hardware and software program-

ming paradigms. Additionally, the growing complexity of modern compilers and the

concern over security are among the most important problems that compilers should

answer. Moore’s law [1] states that transistor density should double every two years;

however, the rate of compilers, which are faced with many open-research problems,

have not been able to improve more than a few percentage points each year [2].

A. H. Ashouri et al., Automatic Tuning of Compilers Using Machine Learning,

PoliMI SpringerBriefs, https://doi.org/10.1007/978-3-319-71489-9_1

2 1 Background

Usually, software applications are developed in a high-level programming lan-

guage (e.g. C, C++) and then passed through the compiler to emit an executable

binary. Compilers have been used for the past 50 years [2, 3] for generating machine-

dependent executable binary from high-level programming languages. Compiler

developers typically design optimization passes in order to transform each code

segment of a program to produce an optimized version of an application. The opti-

mizations can be applied at different stages of the compilation process since com-

pilers have three main layers: (i) front-end (ii) intermediate-representation (IR) and

(iii) backend. At the same time, optimizing source code by hand is a tedious task.

Compiler optimizations provide an automatic method to transform code. To this

end, optimizing the intermediate phase plays an important role on the performance

metrics. Enabling compiler optimization parameters (e.g. loop unrolling, register

allocation, etc.) might substantially beneﬁt several performance metrics. Depend-

ing on the objectives, these metrics could be execution time, code size, or power

consumption. A holistic exploration approach to trade-off these metrics represents a

challenging problem [4].

Autotuning addresses automatic code-generation and optimization by using dif-

ferent scenarios and architectures. It constructs techniques for automatic optimiza-

tion of different parameters to maximize or minimize the satisfaction of an objective

function. Historically, optimizations were mostly in the backend where scheduling,

resource-allocation, and code-generation are done [5, 6]. The constraints form a

linear problem (ILP), thus one should solve an ILP to ﬁnd an optimized design.

Recently, researchers have shown increased effort in introducing frontend and IR

optimizations. The reason is twofold: (i) the complexity of a backend compiler

requires strict and exclusive knowledge of compiler designers, and (ii) frontend

optimizations have lower overheads compared with the optimizations performed in

the back-end. Nonetheless, each has its beneﬁts and drawbacks and is subject to

further analyses.

The major challenge in choosing the right set of compiler optimizations is the

fact that the optimizations are programming language, application, and architecture

dependent. Additionally, the word optimization is a misnomer; there is no guarantee

that the transformed code will perform better than the original version. Aggressive

optimizations can degrade the performance of the code to which they are applied [7].

Understanding the behavior of the optimizations, the perceived effects on the source-

code, and the interaction of the optimizations with each other are complex modeling

problems. This understanding is particularly difﬁcult because compiler developers

must consider hundreds of optimizations that can be applied during the different

compilation phases. This optimization ordering dilemma creates the phase-ordering

problem.

There were a number of long-standing compiler optimization problems have not

yet been adequately addressed. These problems comprise of knowing what optimiza-

tions (by which conﬁguration (i.e., the tile size in loop tiling)), and in which order

the optimizations should be used to obtain the best improvement. The former yields

the so-called problem of selecting the best compiler optimizations and the latter is

the phase-ordering problem of compiler optimizations. The phase-ordering problem

1.1 Introduction 3

has been an open problem for many decades [8–11]. The inability of researchers to

solve the phase-ordering problem has led to advances in the less complex problem

of selecting the right set of optimizations, but even this has yet to be fully resolved

[12–14].

In order to understand the difﬁculty of the problems, one should consider that the

process of selecting the right optimizations for each code segment is typically done

manually and the sequence of optimizations is constructed with little insight into the

interaction between the preceding compiler optimizations in the sequence. The task

of constructing heuristics to select the right sequence of compiler optimizations is

infeasible given the ever-growing number of compiler optimizations being integrated

into compiler frameworks. As an example, GCC has more than 200 compiler passes,

referred to as compiler options,

and LLVM-clang and LLVM-opt both have more

than 100 transformation passes

each. Additionally, these optimizations are applied

at very different phases of the compilation, including analysis passes and loop-nest

passes. Most optimization ﬂags are turned off by default and compiler developers

rely on software developers to know, which optimizations will be beneﬁcial for their

code. Compiler developers provide standard optimization levels, i.e., -O1, -O2,

-Os, etc. to introduce a ﬁxed-sequence of compiler optimizations that, on average,

bring good performance on a set of benchmarks compiler developers tested the

optimization levels with. However, using predeﬁned optimizations usually is not good

enough to bring the best achievable application-speciﬁc performance. One of the key

approaches used recently in literature to ﬁnd the best optimizations to apply to an

application is inducing prediction models using different classes of machine learning

[15]. Approaches which leverage machine learning to ﬁnd the best optimizations to

apply will be the center focus of this book.

Contribution. In this chapter, we provide an overview of techniques and

approaches appeared so far to tackle the aforementioned problems. We elaborate

many recent papers proposed for compiler autotuning when Machine Learning (ML)

was involved. To the best of our knowledge, the ﬁrst application of machine learning

for the compiler autotuning problem was done by [16–18]. However, there were other

original works which tackled the problem of compiler autotuning without the use of

machine learning technique [10, 19–22], and we believe them to be the driving force

of using machine learning to the existing problems. Thus we decided to consider

the past 25 years as it covers the whole time span of the literature on the very ﬁeld.

Additionally, this chapter can be a connecting point for the already available surveys

[23, 24] on the compiler optimization ﬁeld.

We ﬁrst discuss the motivation and challenges involved in the compiler optimiza-

tion research in Sect. 1.2, followed by an analysis of the optimization space for the

two major optimization problems. Then, we discuss the machine learning models

used in Sect. 1.3 and provide a full classiﬁcation of different techniques used in the

recent research. Finally, we conclude the chapter with discussion and future trends.

https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html.

http://llvm.org/docs/Passes.html.

1.2 Compiler Optimizations 5

clarity, we do not use all these terms but rather use optimizing compilers, or compiler

autotuning. Moreover, under each classiﬁed subsection, we will point out the other

nomenclatures that have been used widely and our reference subsequently.

1.2.2 Compiler Optimization Beneﬁts and Challenges

The majority of potential speedup no longer arrives at the increase of processor core

clock frequencies. Automatic methods of exploiting parallelism and reducing depen-

dencies are needed. Compiler optimizations [20] allow a user to affect the generated

code without changing the original high-level source code. When these optimizations

are applied may result in a program running better on a target architecture. Since a

user is not able to manual tune a large code, automatic methods need to be intro-

duced. Additionally, manual tuning is not portable – transformations applied to code

running on one architecture is not guaranteed to yield the same performance increase

on another architecture. We discuss the challenges and beneﬁts at two levels:

Research Level

One clear beneﬁt of optimization passes is their portability – if necessary, they can

be easily adapted to newer architectures. However, there are some optimizations that

have been researched more than others. Speciﬁcally, we still have not reached the once

“holy grail” of auto-parallelizing compilers, but we have made signiﬁcant progress.

Polyhedral loop analysis and transformations paved the way for safe transformations

leading to auto-parallelizable code segments. The polyhedral model also aided with

the generation of architecture-dependent, cache-friendly access patterns.

User’s Level

Compiler writers expose general-purpose transformations to end users. Ulti-

mately, a subset of these transformations leads to better architecture ﬁtting given

source code. Over time we have seen an overall improvement of compilers and related

tools. New high-level languages and languages extensions make additional optimiza-

tions possible. Directive-based programming languages, such as OpenMP [25] and

OpenACC [26], automatically transform users’ code to exploit parallelism. One of

the most advantageous contributions has been the introduction of easily expandable

compiler infrastructures such as LLVM. An advanced compiler infrastructure makes

it possible to introduce new optimizations with minimal effort.

1.2.3 Compiler Optimization Problems

The problem of interdependency among phases of compiler optimizations is not

unique to compiler optimization ﬁeld. Phase inter-dependencies have been noted in

剩余129页未读，继续阅读

WindStand

粉丝: 35
资源: 367

机器学习驱动编译器自动调优：性能优化新策略

"Java应用程序中的内存问题解决方案-Poonam Parhar.pdf

Spark Autotuning: 藏经阁文件优化手册

TensorRT-*.*.*.*在Windows10 CUDA 10.2环境下的安装与使用指南

Automatic Tuning of Compilers Using Machine Learning

Pid Control System Design and Automatic Tuning Using Matlab Simulink (Wiley-IEEE.pdf

Hands－On.Machine.Learning.with.Scikit－Learn.and.TensorFlow.2017

P-Tuning v2.pdf

hotspot-virtual-machine-garbage-collection-tuning-guide.pdf

Performance-Tuning-Guide-for-Mellanox-Network-Adapters.pdf

bert-arch-1layer.pdf

最新资源

TensorRT-...在Windows10 CUDA 10.2环境下的安装与使用指南