无约束优化：Wolfram Mathematica 7.0教程

需积分: 13 84 浏览量更新于2024-07-17 收藏 18.82MB PDF 举报

《无约束优化》是一本针对Wolfram Mathematica® 7.0及更高版本的专业电子书，由罗伯·纳普（Rob Knapp）撰写，收录在Wolfram Research的教程集合中。本书深入探讨了无约束优化的相关理论与实践，适用于那些希望在数学建模、工程设计、经济分析等领域运用高级数学方法解决未受限制条件限制的最优化问题的学习者和专业人士。该书涵盖了无约束优化的基本概念，如梯度下降法、牛顿法、拟牛顿法等优化算法的原理与实现，以及如何在Wolfram Mathematica的环境下利用其强大的数值计算功能进行高效求解。书中还可能包含对多变量函数、非线性问题、局部与全局最优解、收敛性和性能评估等方面的详细讨论。为了获取最新的更新和更正，读者可以访问Wolfram官方文档网站reference.wolfram.com。对于获取额外的副本或技术支持，可以前往www.wolfram.com/services/customerservice网站，或者通过电子邮件联系CustomerService部门，地址是info@wolfram.com。同时，欢迎读者将对本书的评论发送至comments@wolfram.com，以便于作者和社区共同进步。版权方面，所有内容受Wolfram Research, Inc.保护，未经版权所有者事先书面许可，任何形式的复制或传播，无论是电子、机械、影印、录音或其他方式，都是被禁止的。Wolfram Research是Wolfram Mathematica软件系统的版权持有者，包括软件的代码、结构、组织方式、用户界面设计以及命令名称等元素。《无约束优化》是一本实用且权威的教材，不仅适合Wolfram Mathematica用户提升其在优化问题上的技能，也为从事相关领域研究的学者提供了宝贵的参考资源。通过阅读这本书，读者将能掌握无约束优化的核心概念，并能在实际项目中灵活应用这些技术。

When the gradient and Hessian are both computed using finite differences, the error in the

Hessian may be quite large and it may be better to use a different method. In this case,

FindMinimum

does find the minimum quite accurately, but cannot be sure because of inade-

quate derivative information. Also, the number of function and gradient evaluations is much

greater than in the example with the symbolic derivatives computed automatically because

extra evaluations are required to approximate the gradient and Hessian, respectively.

If it is possible to supply the gradient (or the function is such that it can be computed automati-

cally), the method will typically work much better. You can give the gradient using the

Gradient

option, which has several ways you can "specify derivatives".

This defines a function that returns the gradient for numerical values of

and

In[5]:=

g@x_ ?NumberQ, y_ ? NumberQD = Map@D@Cos@x ^ 2 - 3 yD + Sin@x ^ 2 + y ^ 2D, ÒD &, 8x, y<D

Out[5]=

92 x CosAx

+ y

E - 2 x SinAx

- 3 yE, 2 y CosAx

+ y

E + 3 SinAx

- 3 yE=

This tells

FindMinimum

to use the supplied gradient. The Hessian is computed using finite

differences of the gradient.

In[6]:=

FindMinimum@f@x, yD, 88x, 1<, 8y, 1<<, Gradient Ø g@x, yD, Method Ø "Newton"D

Out[6]=

8-2., 8x Ø 1.37638, y Ø 1.67868<<

If you can provide a program that gives the Hessian, you can provide this also. Because the

Hessian is only used by Newton's method, it is given as a method option of Newton.

This defines a function that returns the Hessian for numerical values of

and

In[7]:=

h@x_ ?NumberQ, y_ ? NumberQD =

Outer@D@Cos@x ^ 2 - 3 yD + Sin@x ^ 2 + y ^ 2D, ÒÒD &, 8x, y<, 8x, y<D

Out[7]=

99-4 x

CosAx

- 3 yE + 2 CosAx

+ y

E - 2 SinAx

- 3 yE - 4 x

SinAx

+ y

6 x CosAx

- 3 yE - 4 x y SinAx

+ y

E=,

96 x CosAx

- 3 yE - 4 x y SinAx

+ y

E, -9 CosAx

- 3 yE + 2 CosAx

+ y

E - 4 y

SinAx

+ y

E==

This tells

FindMinimum

to use the supplied gradient and Hessian.

In[8]:=

FindMinimum@f@x, yD, 88x, 1<, 8y, 1<<,

Gradient Ø g@x, yD, Method Ø 8"Newton", "Hessian" Ø h@x, yD<D

Out[8]=

8-2., 8x Ø 1.37638, y Ø 1.67868<<

In principle, Newton's method uses the Hessian computed either by evaluating the symbolic

derivative or by using finite differences. However, the convergence for the method computed

Unconstrained Optimization

modifies the Hessian by a diagonal matrix

with entries large enough so that

= “

f Hx

L + E

positive definite. Such methods are sometimes referred to as modified Newton methods. The

modification to

is done during the process of computing a Cholesky decomposition somewhat

along the lines described in [GMW81], both for dense and sparse Hessians. The modification is

only done if

“

f Hx

is not positive definite. This decomposition method is accessible through

LinearSolve

if you want to use it independently.

This computes the step using

= -“ f H x

, where

is determined as the Cholesky factors of

the Hessian are being computed.

In[12]:=

LinearSolve@h@1.2, .5D, - g@1.2, .5D,

Method Ø 8"Cholesky", "Modification" Ø "Minimal "<D

Out[12]=

80.00405502, 0.0196737<

The computed step is in a descent direction.

In[13]:=

%.g@1.2, .5D

Out[13]=

-0.00645255

Besides the robustness of the (modified) Newton method, another key aspect is its convergence

rate. Once a search is close enough to a local minimum, the convergence is said to be

quadratic, which means that if

is the local minimum point, then

° x

k+1

- x

¥ § b °x

- x

for some constant

b > 0

At machine precision, this does not always make a substantial difference since it is typical that

most of the steps are spent getting near to the local minimum. However, if you want a root to

extremely high precision, Newton's method is usually the best choice because of the rapid

convergence.

This computes a very high-precision solution using Newton's method. The precision is adap-

tively increased from machine precision (the precision of the starting point) to the maximal

working precision of 100000 digits.

Reap

is used with

Sow

to save the steps taken. Counters

are used to track and print the number of function evaluations and steps used.

In[14]:=

First@Timing@Block@8e = 0, s = 0<, 88min, minpoint<, 8points<< =

Reap@FindMinimum@Cos@x ^ 2 - 3 yD + Sin@x ^ 2 + y ^ 2D,

88x, 1.<, 8y, 1.<<, Method -> "Newton", WorkingPrecision Ø 100 000,

StepMonitor ß Hs ++; Sow@8x, y<DL, EvaluationMonitor ß e ++DD;

Print@s, " steps and ", e, " evaluations"DDDD

17 steps and 27 evaluations

Out[14]=

4.56134

Unconstrained Optimization

When the option

WorkingPrecision -> prec

is used, the default for the

AccuracyGoal

and

PrecisionGoal

prec ê 2

. Thus, this example should find the minimum to at least 50000 digits.

This computes a symbolic solution for the position of the minimum which the search approaches.

In[15]:=

exact = 8x, y< ê. Last@Solve@8x ^2 + y ^2 ã 3 Pi ê 2, x^ 2 - 3 y ã - Pi<, 8x, y<DD

Out[15]=

: -

- p +

9 + 10 p ,

-3 + 9 + 10 p >

This computes the norm of the distance from the search points at the end of each step to the

exact minimum.

In[16]:=

N@Map@Norm@exact - ÒD &, pointsDD

Out[16]=

90.140411, 0.0156607, 0.000236558, 6.09444 µ 10

-8

, 3.8255 µ 10

-15

, 1.59653 µ 10

-29

, 3.24619 µ 10

-58

4.8604 µ 10

-108

, 1.26122 µ 10

-212

, 5.865676867279906 µ 10

-406

, 1.755647053247051 µ 10

-791

4.345222958143836 µ 10

-1581

, 1.099183429735576 µ 10

-3141

, 1.614858677992596 µ 10

-6262

5.998002325828813 µ 10

-12 514

, 1.543301971989607 µ 10

-25 010

, 1.131416408748486 µ 10

-50 010

The reason that more function evaluations were required than the number of steps is that

Mathematica adaptively increases the precision from the precision of the initial value to the

requested maximum

WorkingPrecision

. The sequence of precisions used is chosen so that as

few computations are done at the most expensive final precision as possible under the assump-

tion that the points are converging to the minimum. Sometimes when Mathematica changes

precision, it is necessary to reevaluate the function at the higher precision.

This shows a table with the precision of each of the points with the norm of their errors.

In[17]:=

TableForm@Transpose@8Map@Precision, pointsD, N@Map@Norm@exact - ÒD &, pointsDD<DD

Out[17]//TableForm=

MachinePrecision 0.140411

MachinePrecision 0.0156607

MachinePrecision 0.000236558

MachinePrecision 6.09444 µ 10

-8

24.4141 3.8255 µ 10

-15

48.8283 1.59653 µ 10

-29

97.6565 3.24619 µ 10

-58

195.313 4.8604 µ 10

-108

390.626 1.26122 µ 10

-212

781.252 5.865676867279906 µ 10

-406

1562.5 1.755647053247051 µ 10

-791

3125.01 4.345222958143836 µ 10

-1581

6250.02 1.099183429735576 µ 10

-3141

12 500. 1.614858677992596 µ 10

-6262

25 000.1 5.998002325828813 µ 10

-12 514

50 000.2 1.543301971989607 µ 10

-25 010

100 000. 1.131416408748486 µ 10

-50 010

Unconstrained Optimization

剩余75页未读，继续阅读

J-10

粉丝: 18
资源: 483

无约束优化：Wolfram Mathematica 7.0教程

An Unconstrained Optimization Test Functions Collection Matlab代码

alpr（车牌识别）

alpr-unconstrained：不受约束的场景中的车牌检测和识别

生成一段惯性松弛算法代码

次梯度法matlab代码

对于没有目标函数的关于功率p的约束条件，如何使用python进行编写代码

无约束优化牛顿法python

拉格朗日乘数法matlab代码

BFGS在matlab中的实现

使用python编写牛顿法并求解无约束优化问题

最新资源