L-BFGS的matlab代码示例

以下是一个使用L-BFGS优化方法的matlab代码示例：

function [x_opt, f_opt] = lbfgs(fun, x0, opts)
% LBFGS - Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS)
%
%   MINIMIZE:
%       f(x) = fun(x)
%   where fun: R^n -> R.
%
%   INPUTS:
%       fun: function handle to the objective function
%           [f, g] = fun(x) returns the function value and gradient at x
%           f: scalar, function value at x
%           g: column vector, gradient at x
%       x0: column vector, initial point
%       opts: struct, optional inputs
%           opts.max_iter: scalar, maximum number of iterations (default: 100)
%           opts.tol_fun: scalar, tolerance for the function value (default: 1e-6)
%           opts.tol_grad: scalar, tolerance for the gradient (default: 1e-6)
%           opts.m: scalar, number of previous iterations to remember (default: 10)
%
%   OUTPUTS:
%       x_opt: column vector, optimal point
%       f_opt: scalar, optimal function value
%
%   REFERENCES:
%       [1] Nocedal, J., & Wright, S. J. (2006). Numerical optimization (2nd ed.).
%           Springer New York. https://doi.org/10.1007/978-0-387-40065-5

% default options
if nargin < 3
    opts = struct();
end
if ~isfield(opts, 'max_iter')
    opts.max_iter = 100;
end
if ~isfield(opts, 'tol_fun')
    opts.tol_fun = 1e-6;
end
if ~isfield(opts, 'tol_grad')
    opts.tol_grad = 1e-6;
end
if ~isfield(opts, 'm')
    opts.m = 10;
end

% initialization
x = x0;
[f, g] = fun(x);
p = -g;
H = eye(length(x));
s = [];
y = [];
alpha = zeros(opts.m, 1);
beta = zeros(opts.m, 1);

% main loop
for iter = 1 : opts.max_iter
    % line search
    t = 1;
    while true
        x_new = x + t * p;
        [f_new, g_new] = fun(x_new);
        if f_new < f + opts.tol_fun * t * g' * p
            break;
        end
        t = t / 2;
        if t < eps
            break;
        end
    end

    % convergence check
    if norm(g_new) < opts.tol_grad
        break;
    end

    % update s, y, and H
    if size(s, 2) == opts.m
        s(:, 1:end-1) = s(:, 2:end);
        y(:, 1:end-1) = y(:, 2:end);
        alpha(1:end-1) = alpha(2:end);
        beta(1:end-1) = beta(2:end);
    end
    s(:, end+1) = x_new - x;
    y(:, end+1) = g_new - g;
    alpha(end+1) = s(:, end)' * y(:, end) / (y(:, end)' * y(:, end));
    q = g_new;
    for i = size(s, 2) : -1 : max(1, size(s, 2) - opts.m + 1)
        rho = 1 / (y(:, i)' * s(:, i));
        alpha(i) = rho * s(:, i)' * q;
        q = q - alpha(i) * y(:, i);
    end
    if size(s, 2) > 0
        gamma = (s(:, end)' * y(:, end)) / (y(:, end)' * y(:, end));
        q = gamma * q;
        for i = max(1, size(s, 2) - opts.m + 1) : size(s, 2)
            rho = 1 / (y(:, i)' * s(:, i));
            beta(i) = rho * y(:, i)' * q;
            q = q + s(:, i) * (alpha(i) - beta(i));
        end
    end
    H = q * p' + p * q';
    H = (H + H') / 2;

    % update x, f, g, and p
    x = x_new;
    f = f_new;
    g = g_new;
    p = -H \ g;
end

x_opt = x;
f_opt = f;

end

该函数接受一个函数句柄 `fun`，一个初始点 `x0`，以及一些可选参数 `opts`。函数 `fun` 应该返回函数值和梯度，以便 L-BFGS 方法可以优化该函数。该函数还可以接受其他可选参数，例如最大迭代次数、目标函数和梯度的容差等。

该代码使用了标准的 L-BFGS 更新公式，包括对 S、Y 和 alpha、beta 的更新以及 Hessian 矩阵的逆的计算。该代码还包括一个线搜索步骤，以确保每次更新都是朝着下降方向。