Levenberg–Marquardt 算法原理及 Matlab 实现代码

版权申诉

33 浏览量更新于2024-07-05 收藏 399KB PDF 举报

"Levenberg-Marquart 算法介绍及Matlab代码" Levenberg-Marquart 算法是一种广泛应用于非线性最小二乘法问题的数值优化算法。该算法的主要思想是将 Gauss-Newton 方法与梯度下降法相结合，以此来解决非线性最小二乘法问题。该算法的优点是收敛速度快、稳定性好、适用范围广泛。 Matlab 是一种广泛应用于科学计算、数据分析和可视化的编程语言。Matlab 提供了一个强大的开发环境，用户可以使用 Matlab 编写代码来实现 Levenberg-Marquart 算法。Matlab 的优点是易于使用、开发效率高、功能强大，因此广泛应用于科学研究、工程设计和数据分析等领域。在 Matlab 中实现 Levenberg-Marquart 算法需要使用到一些基本的数学运算和矩阵操作。Matlab 提供了一个强大的矩阵运算环境，用户可以使用 Matlab 的矩阵运算符来实现矩阵的加法、减法、乘法和除法等操作。此外，Matlab 还提供了一些特殊的函数来实现 Levenberg-Marquart 算法，例如 `fminsearch` 函数可以用来实现非线性最小二乘法问题的求解。 Levenberg-Marquart 算法的 Matlab 代码可以从 Matlab Central 上下载，网址为 http://www.mathworks.com/matlabcentral/fileexchange/authors/31545。该网址提供了三个版本的 Levenberg-Marquart 算法的 Matlab 代码，分别是 LMFsolve、LMFnlsq 和 LMFnlsq2。这些代码都是由 Miroslav Balda 编写的，用户可以免费下载并使用。 Miroslav Balda 是一位具有共享精神的开发者，他将自己编写的代码都放在了 Matlab Central 上，供大家免费下载和使用。他的代码包括了 Levenberg-Marquart 算法的多种实现方式，用户可以根据自己的需求选择合适的版本。在使用 Levenberg-Marquart 算法时，需要注意一些重要的参数设置，例如收敛阈值、最大迭代次数等。这些参数的设置将直接影响算法的收敛速度和稳定性。因此，用户需要根据自己的问题选择合适的参数设置。 Levenberg-Marquart 算法是一种广泛应用于非线性最小二乘法问题的数值优化算法。Matlab 是一种强大的开发环境，用户可以使用 Matlab 编写代码来实现 Levenberg-Marquart 算法。Levenberg-Marquart 算法的 Matlab 代码可以从 Matlab Central 上下载，用户可以根据自己的需求选择合适的版本。

72.DESCENT METHODS

Deﬁnition 2.6. Descent direction.

h is a descent direction for F at x if h

(x) < 0 .

If no such h exists, then F

(x)=0, showing that in this case x is stationary.

Otherwise, we have to choose α, ie how far we should go from x in the

direction given by h

, so that we get a decrease in the value of the objective

function. One way of doing this is to ﬁnd (an approximation to)

= argmin

α>0

{F (x+αh)} . (2.7)

The process is called line search, and is discussed in Section 2.3. First,

however, we shall introduce two methods for computing a descent direction.

2.1. The Steepest Descent method

From (2.5) we see that when we perform a step αh with positive α, then the

relative gain in function value satisﬁes

lim

α→0

F (x) − F (x+αh)

αkhk

= −

khk

(x)=−kF

(x)kcos θ,

where θ is the angle between the vectors h and F

(x). This shows that we

get the greatest gain rate if θ = π, ie if we use the steepest descent direction

given by

= −F

(x) . (2.8)

The method based on (2.8) (ie h

= h

in Algorithm 2.4) is called the steep-

est descent method or gradient method. The choice of descent direction is

“the best” (locally) and we could combine it with an exact line search (2.7).

A method like this converges, but the ﬁnal convergence is linear and often

very slow. Examples in Frandsen et al (2004) show how the steepest descent

method with exact line search and ﬁnite computer precision can fail to ﬁnd

the minimizer of a second degree polynomial. For many problems, however,

the method has quite good performance in the initial stage of the iterative

process.

2.2. Newton’s Method 8

Considerations like this has lead to the so-called hybrid methods, which – as

the name suggests – are based on two different methods. One which is good

in the initial stage, like the gradient method, and another method which is

good in the ﬁnal stage, like Newton’s method; see the next section. A major

problem with a hybrid method is the mechanism which switches between

the two methods when appropriate.

2.2. Newton’s Method

We can derive this method from the condition that x

∗

is a stationary point.

According to Deﬁnition 1.6 it satisﬁes F

∗

)=0. This is a nonlinear sys-

tem of equations, and from the Taylor expansion

(x+h)=F

(x)+F

(x)h + O(khk

)

' F

(x)+F

(x)h for khk sufﬁciently small

we derive Newton’s method: Find h

as the solutions to

=−F

(x) with H = F

(x) , (2.9a)

and compute the next iterate by

x := x + h

. (2.9b)

Suppose that H is positive deﬁnite, then it is nonsingular (implying that

(2.9a) has a unique solution), and u

Hu>0 for all nonzero u. Thus, by

multiplying with h

on both sides of (2.9a) we get

0 < h

=−h

(x), (2.10)

showing that h

is a descent direction: it satisﬁes the condition in Deﬁni-

tion 2.6.

Newton’s method is very good in the ﬁnal stage of the iteration, where x is

close to x

∗

. One can show (see Frandsen et al (2004)) that if the Hessian

at the solution is positive deﬁnite (the sufﬁcient condition in Theorem 1.8

is satisﬁed) and if we are at a position inside the region around x

∗

where

92.DESCENT METHODS

(x) is positive deﬁnite, then we get quadratic convergence (deﬁned in

(2.3)). On the other hand, if x is in a region where F

(x) is negative deﬁnite

everywhere, and where there is a stationary point, the basic Newton method

(2.9) would converge (quadratically) towards this stationary point, which

is a maximizer. We can avoid this by requiring that all steps taken are in

descent directions.

We can build a hybrid method, based on Newton’s method and the steepest

descent method. According to (2.10) the Newton step is guaranteed to be

downhill if F

(x) is positive deﬁnite, so a sketch of the central section of

this hybrid algorithm could be

if F

(x) is positive deﬁnite

h := h

else

h := h

x := x + αh

(2.11)

Here, h

is the steepest descent direction and α is found by line search; see

Section 2.3. A good tool for checking a matrix for positive deﬁniteness is

Cholesky’s method (see Appendix A) which, when successful, is also used

for solving the linear system in question. Thus, the check for deﬁniteness is

almost for free.

In Section 2.4 we introduce some methods, where the computation of the

search direction h

and step length α is done simultaneously, and give a

version of (2.11) without line search. Such hybrid methods can be very

efﬁcient, but they are hardly ever used. The reason is that they need an im-

plementation of F

(x), and for complicated application problems this is not

available. Instead we can use a so-called Quasi-Newton method, based on

series of matrices which gradually approach H

∗

= F

∗

). In Section 3.4

we present such a method. Also see Chapter 5 in Frandsen et al (2004).

2.3. Line Search

Given a point x and a descent direction h. The next iteration step is a move

from x in direction h. To ﬁnd out, how far to move, we study the variation

of the given function along the half line from x in the direction h,

2.3. Line Search 10

ϕ(α)=F(x+αh), xand h ﬁxed,α≥0. (2.12)

An example of the behaviour of ϕ(α) is shown in Figure 2.1.

y = φ(0)

y = φ(α)

Figure 2.1. Variation of the cost

function along the search line.

Our h being a descent direction ensures that

(0) = h

(x) < 0 ,

indicating that if α is sufﬁciently small, we satisfy the descending condition

(2.1), which is equivalent to

ϕ(α) <ϕ(0) .

Often, we are given an initial guess on α,egα=1with Newton’s method.

Figure 2.1 illustrates that three different situations can arise

◦

α is so small that the gain in value of the objective function is very

small. α should be increased.

◦

α is too large: ϕ(α) ≥ϕ(0). Decrease α in order to satisfy the descent

condition (2.1).

◦

α is close to the minimizer

of ϕ(α). Accept this α-value.

More precisely: the smallest local minimizer of ϕ. If we increase α beyond the interval

shown in Figure 2.1, it may well happen that we get close to another local minimum

for F .

剩余30页未读，继续阅读

老帽爬新坡

粉丝: 98
资源: 2万+

Levenberg–Marquardt 算法原理及 Matlab 实现代码

Levenberg–Marquardt Matlab代码

Levenberg-Marquardt算法

Levenberg-Marquardt 工具箱：具有 Broyden 更新、框约束和参数传递的 Levenberg-Marquardt 算法。-matlab开发

BP算法的改进在Matlab的实现研究.pdf

Matlab环境下基于遗传算法的正弦波信号参数提取.pdf

优化教程：用于凸优化的函数集，包括不同的优化和线搜索算法-matlab开发

MATLAB在建筑抗震仿真实用教程：结构动力特性与神经网络应用

用matlab写bp神经网络，激活函数用sigmoid,训练算法用levenberg-marquart,隐藏层数目为1，隐藏元数目为25

carsim，simulink联合仿真，自动驾驶基于mpc自定义期望速度跟踪控制，可以在外部自定义期望速度传入sfunction函数，设置了两个不同状态方程，控制量为加速度，加速度变化量提供进行对比

matlab实现阿基米德算法AOA求解零空闲流水车间调度问题NIFSP-阿基米德算法-流水车间调度-NIFSP-matlab

最新资源