优化方法：S.J.Wright的数值优化第二版解析

需积分: 41 105 浏览量更新于2024-07-20 1 收藏 4.45MB PDF 举报

"numerical optimization (S.J.Wright) 2ed" 是一本关于数值优化的经典书籍，适合初学者，作者S.J. Wright是该领域的权威专家。这本书在介绍数值优化方法时，注重理论与实践的结合，语言简洁易懂，受到了广泛的好评。数值优化是计算科学中的一个重要分支，它涉及寻找数学函数的最小值或最大值，常用于工程、经济学、机器学习等领域的最优化问题。在实际应用中，很多问题不能用解析方法解决，因此需要借助数值方法来逼近最优解。本书可能涵盖了以下核心知识点： 1. **线性规划**：这是优化的基础，Dantzig和Thapa的两本书对此进行了深入介绍。线性规划通过解决一组线性不等式和等式约束下的线性目标函数最大化或最小化问题，找到最优解。 2. **非线性规划**：当目标函数或约束是非线性的，问题变得复杂。书中可能讨论了梯度下降法、牛顿法、拟牛顿法等求解非线性优化问题的算法。 3. **约束优化**：除了无约束优化，实际问题往往包含各种类型（如等式、不等式）的约束。书中可能介绍了处理这些约束的方法，如拉格朗日乘子法、惩罚函数法等。 4. **动态规划**：在处理具有时间序列或顺序决策的问题时，动态规划是一种有效工具。书中可能会讲解基本的动态规划原理和应用实例。 5. **随机优化**：Bonnans和Shapiro的书涉及随机优化问题，这在不确定环境中特别有用。书中可能讨论了期望值最小化、风险敏感优化等概念。 6. **变分不等式与互补问题**：Facchinei和Pang的两卷本详细探讨了这些问题，它们在均衡问题、控制理论和经济学中有广泛应用。 7. **蒙特卡洛方法**：Fishman的两本书涉及蒙特卡洛模拟，这是一种基于随机抽样的数值方法，用于估算和分析复杂系统的行为。书中可能涵盖基本概念、算法实现以及实际应用。 8. **物流与供应链管理**：Bramel, Chen, 和Simchi-Levi的书讲述了物流和供应链管理的理论、算法和应用，这与优化问题紧密相关，因为供应链优化是企业运营中的关键问题。 9. **设施选址**：Drezner编著的书对设施选址问题进行了综述，这是运筹学中的一个重要子领域，涉及到如何最优地分配资源和服务。 10. **离散事件仿真**：Fishman的另一本书涉及仿真技术，这对于理解复杂系统的动态行为和优化决策过程非常有用。通过阅读这本书，读者不仅可以掌握数值优化的基本概念，还能学习到如何在实际问题中应用这些方法，为解决各种工程和管理问题打下坚实基础。

xvi C ONTENTS

Description of a Trust-Region Interior-Point Method . . ......... 582

19.6 ThePrimalLog-BarrierMethod...................... 583

19.7 GlobalConvergenceProperties ...................... 587

FailureoftheLineSearchApproach.................... 587

ModiﬁedLineSearchMethods ...................... 589

Global Convergence of the Trust-Region Approach . . . ......... 589

19.8 SuperlinearConvergence ......................... 591

19.9 PerspectivesandSoftware ......................... 592

NotesandReferences ............................... 593

Exercises...................................... 594

A Background Material 598

A.1 ElementsofLinearAlgebra ........................ 598

VectorsandMatrices............................ 598

Norms ................................... 600

Subspaces . ................................ 602

Eigenvalues, Eigenvectors, and the Singular-Value Decomposition . . . . 603

DeterminantandTrace .......................... 605

Matrix Factorizations: Cholesky, LU, QR ................. 606

SymmetricIndeﬁniteFactorization .................... 610

Sherman–Morrison–WoodburyFormula................. 612

InterlacingEigenvalueTheorem...................... 613

Error Analysis and Floating-Point Arithmetic ............... 613

Conditioning and Stability . . ....................... 616

A.2 ElementsofAnalysis,Geometry,Topology ................ 617

Sequences ................................. 617

RatesofConvergence ........................... 619

Topology of the Euclidean Space IR

.................... 620

Convex Sets in IR

............................. 621

ContinuityandLimits........................... 623

Derivatives................................. 625

DirectionalDerivatives .......................... 628

MeanValueTheorem ........................... 629

ImplicitFunctionTheorem ........................ 630

OrderNotation .............................. 631

Root-Finding for Scalar Equations . . .................. 633

B A Regularization Procedure 635

References 637

Index 653

xviii P REFACE

designing a portfolio of investments to maximize expected return while maintaining

an acceptable level of risk;

controlling a chemical process or a mechanical device to optimize performance or

meet standards of robustness;

computing the optimal shape of an automobile or aircraft component.

Every year optimization algorithms are being called on to handle problems that

are much larger and complex than in the past. Accordingly, the book emphasizes large-

scale optimization techniques, such as interior-point methods, inexact Newton methods,

limited-memory methods, and the role of partially separable functions and automatic

differentiation. It treats important topics such as trust-region methods and sequential

quadratic programming more thoroughly than existing texts, and includes comprehensive

discussion of such “core curriculum” topics as constrained optimization theory, Newton

and quasi-Newton methods, nonlinear least squares and nonlinear equations, the simplex

method, and penalty and barrier methods for nonlinear programming.

The Audience

We intend that this book will be used in graduate-level courses in optimization, as of-

fered in engineering, operations research, computer science, and mathematics departments.

There is enough material here for a two-semester (or three-quarter) sequence of courses.

We hope, too, that this book will be used by practitioners in engineering, basic science, and

industry, and our presentation style is intended to facilitate self-study. Since the book treats

a number of new algorithms and ideas that have not been described in earlier textbooks, we

hope that this book will also be a useful reference for optimization researchers.

Prerequisites for this book include some knowledge of linear algebra (including nu-

merical linear algebra) and the standard sequence of calculus courses. To make the book as

self-contained as possible, we have summarized much of the relevant material from these ar-

eas in the Appendix. Our experience in teaching engineering students has shown us that the

material is best assimilated when combined with computer programming projects in which

the student gains a good feeling for the algorithms—their complexity, memory demands,

and elegance—and for the applications. In most chapters we provide simple computer

exercises that require only minimal programming proﬁciency.

Emphasis and Writing Style

We have used a conversational style to motivate the ideas and present the numerical

algorithms. Rather than being as concise as possible, our aim is to make the discussion ﬂow

in a natural way. As a result, the book is comparatively long, but we believe that it can be

read relatively rapidly. The instructor can assign substantial reading assignments from the

text and focus in class only on the main ideas.

A typical chapter begins with a nonrigorous discussion of the topic at hand, including

ﬁgures and diagrams and excluding technical details as far as possible. In subsequent sections,

P REFACE xix

the algorithms are motivated and discussed, and then stated explicitly. The major theoretical

results are stated, and in many cases proved, in a rigorous fashion. These proofs can be

skipped by readers who wish to avoid technical details.

The practice of optimization depends not only on efﬁcient and robust algorithms,

but also on good modeling techniques, careful interpretation of results, and user-friendly

software. In this book we discuss the various aspects of the optimization process—modeling,

optimality conditions, algorithms, implementation, and interpretation of results—but not

with equal weight. Examples throughout the book show how practical problems are formu-

lated as optimization problems, but our treatment of modeling is light and serves mainly

to set the stage for algorithmic developments. We refer the reader to Dantzig [86] and

Fourer, Gay, and Kernighan [112] for more comprehensive discussion of this issue. Our

treatment of optimality conditions is thorough but not exhaustive; some concepts are dis-

cussed more extensively in Mangasarian [198] and Clarke [62]. As mentioned above, we are

quite comprehensive in discussing optimization algorithms.

Topics Not Covered

We omit some important topics, such as network optimization, integer programming,

stochastic programming, nonsmooth optimization, and global optimization. Network and

integer optimization aredescribed in some excellent texts: for instance,Ahuja, Magnanti, and

Orlin [1] in the case of network optimization and Nemhauser and Wolsey [224], Papadim-

itriou and Steiglitz [235], and Wolsey [312] in the case of integer programming. Books on

stochastic optimization are only now appearing; we mention those of Kall and Wallace [174],

Birge and Louveaux [22]. Nonsmooth optimization comes in many ﬂavors. The relatively

simple structures that arise in robust data ﬁtting (which is sometimes based on the 

norm)

are treated by Osborne [232] and Fletcher [101]. The latter book also discusses algorithms

for nonsmooth penalty functions that arise in constrained optimization; we discuss these

brieﬂy, too, in Chapter 18. A more analytical treatment of nonsmooth optimization is given

by Hiriart-Urruty and Lemar

echal [170]. We omit detailed treatment of some important

topics that are the focus of intense current research, including interior-point methods for

nonlinear programming and algorithms for complementarity problems.

Additional Resource

The material in the book is complemented by an online resource called the NEOS

Guide, which can be found on the World-Wide Web at

http://www.mcs.anl.gov/otc/Guide/

The Guide contains information about most areas of optimization, and presents a number

of case studies that describe applications of various optimization algorithms to real-world

problems such as portfolio optimization and optimal dieting. Some of this material is

interactive in nature and has been used extensively for class exercises.

剩余685页未读，继续阅读

alss1923

粉丝: 22
资源: 66

优化方法：S.J.Wright的数值优化第二版解析

《Numerical Optimization》——J.Nocedal & S.Wright的优化方法解析

《Numerical Optimization》——非线性最优化经典著作

优化方法：Numerical Optimization

Numerical Optimization 2ed - Nocedal.pdf

Numerical Optimization 2ed - Nocedal

Numerical Optimazation 2nd Edition Nocedal-Wright 英文原版

Numerical-Optimization 数值优化 第2版

Jorge Nocedal与Stephen J. Wright合著《数值优化》第二版解析

Linux命令行操练场：实践numerical optimization与文件管理

YOLO算法-城市电杆数据集-496张图像带标签-电杆.zip

最新资源

Numerical-Optimization 数值优化第2版