LQR与神经网络结合的MPC：高效稳定控制新策略

版权申诉

5星 · 超过95%的资源 181 浏览量更新于2024-07-07 3 收藏 1.14MB PDF 举报

在现代工业自动化和控制系统中，模型预测控制（Model Predictive Control, MPC）因其处理动态系统约束的能力而受到广泛关注。然而，MPC在实时应用中的挑战主要体现在三个方面：对于低延迟需求的系统、计算资源有限的系统以及具有快速复杂动态的系统，解决迭代优化问题的速度成为关键瓶颈。为了克服这些挑战，研究人员提出了将线性二次 regulator (Linear Quadratic Regulator, LQR) 和神经网络结合的方法，以提高MPC的效率和稳定性。LQR，作为一种经典的控制理论，适用于低维线性系统，通过设计最优控制器来最小化成本函数，其计算过程相对简单且易于实施。然而，对于高维或非线性系统，传统的LQR方法就显得力不从心，因为它们无法适应复杂的系统动态和约束条件。神经网络作为函数逼近工具，可以用于解决这些问题。通过训练神经网络模型，我们可以近似非线性的系统动态，并将其作为MPC的控制策略。这种方法的优势在于能够处理高维度和非线性系统的复杂性，而且理论上可以通过增加网络容量和深度来逼近精确的系统行为。然而，神经网络的局限性在于可能存在过拟合风险，且训练过程可能耗时且对初始参数敏感。组合LQR和神经网络的MPC策略试图克服这些不足。首先，LQR用于提供一个初始的稳定控制基础，确保基本的系统稳定性。然后，神经网络被用来优化这个基础，通过学习在线调整，以适应环境变化和不确定性。这种方法可以在保持系统稳定性的同时，显著减少计算负担，使得MPC能够在实时环境中更有效地运行。然而，这种方法也存在一些挑战。例如，如何选择合适的神经网络结构和训练策略，以确保模型既能捕捉复杂动态又不会过度拟合？此外，如何在有限的时间窗口内平衡模型预测的准确性与实时决策的效率？为了解决这些问题，研究者可能需要开发新颖的算法和优化技术，如使用深度强化学习或者在线学习策略来调整神经网络。总结来说，将LQR与神经网络融合在MPC中的方法展示了巨大的潜力，特别是在处理高维度和非线性控制问题上。尽管如此，进一步的研究和发展仍然是必要的，以确保这种混合策略在实际应用中的性能和鲁棒性。随着计算能力和数据处理技术的进步，我们期待看到更多此类创新方法在工业自动化和控制系统中的广泛应用。

COMPOSING MPC WITH LQR AND NEURAL NETWORKS 5

4. Composing MPC, LQR, and NN. Provided with a computationally costly

implicit MPC u

MPC

that satisﬁes Theorem (3.1), we propose a hybrid control scheme,

namely, Memory-Augmented Model Predictive Control (MAMPC), u

MAMPC

: X → U,

by augmenting u

MPC

with a LQR controller u

LQR

and a NN controller u

, where the

LQR controller and the NN controller are independently developed from the implicit

MPC. In addition, we present two variants of MAMPC speciﬁcally tailored for chaotic

systems and slow systems, respectively.

4.1. LQR from MPC. Provided with a MPC, we derive an inﬁnite-time LQR

controller by 1) linearizing the system dynamics around the equilibrium, 2) removing

stage constraints, terminal constraint, and terminal cost, 3) taking planning and

control horizon N to ∞, and 4) replacing stage cost c(·) with a positive deﬁnite

quadratic cost, if it is not already so in the original MPC.

Formally, the MPC-induced LQR problem is deﬁned as follows

(4.1)

min

1:∞

0:∞

∞

k=0

[k|i]Qx[k|i] + u

[k|i]Ru[k|i]

s.t.: x[0|i] = x[i],

x[k + 1|i] = Ax[k|i] + Bu[k|i],

k ∈ Z

where A

∂f

∂x



(0,0)

∈ R

n×n

, B

∂f

∂u



(0,0)

∈ R

n×m

are linearized system dynamics

around the equilibrium, and Q ∈ {N ∈ R

n×n

| N  0}, R ∈ {M ∈ R

m×m

| M  0}

are weighting matrices that measure signiﬁcance of state deviations and control costs,

respectively.

It can be shown that the optimal LQR control law for an initial state x is a linear

map

(4.2) u

LQR

(x) = −Kx

for some K ∈ R

m×n

[2]. Furthermore, if the linearized system (A, B) is stabilizable

and if A − BK is Hurwitz, then the closed-loop system

(4.3) x[i + 1] = f (x[i], u

LQR

(x[i])),

is locally asymptotically stable near the equilibrium with a positively invariant region

of attraction R

LQR

, as shown in Theorem 4.7 in [13]. Speciﬁcally, R

LQR

is a set that

if x

∈ R

LQR

and x[0] = x

, then x[i] ∈ R

LQR

, ∀i ∈ Z

and lim

i→∞

x[i] = 0. As

implied by local asymptotic stability, the largest possible R

LQR

is nonempty.

Because the LQR controller only requires one matrix multiplication within a

single controller step, it is usually signiﬁcantly more eﬃcient than the implicit MPC

in terms of per-step computation. Hence, it is usually economical to switch to the

LQR controller when the state is in the region of attraction of the LQR, that is,

x[i] ∈ R

LQR

4.2. NN from MPC. Provided with a MPC, we derive a NN controller 1) by

imitating the MPC policy u

MPC

(x) through supervised learning and 2) optionally

by interacting with the environment through reinforcement learning, as illustrated in

Figure 1.

6 F. WU, G. WANG, S. ZHUANG, K. WANG, A. KEIMER, I. STOICA, AND A. BAYEN

MPC

(x)

(a) Imitation phase

f(x, u

(x))

(x)

(b) Adaptation phase

Fig. 1. Schematics of the learning processes. (a) Imitation phase: (x, u

MPC

(x)) provides data

to train the NN controller u

in a supervised learning setting. (b) Adaptation phase (optional):

interacts with the environment f(·, ·) to enhance control eﬀectiveness in a reinforcement learn-

ing setting.

In the imitation phase, the NN training is a supervised learning problem usually

of the following mean squared form

(4.4) min

x∈D

MPC

(x) − φ(x | W)k

where φ : R

→ R

is a NN model with weights W and D is a set of M states

randomly sampled from X

Therefore, the resulting NN control law is

(4.5) u

(x)

= φ(x | W),

where W is a suboptimal solution to problem (4.4), generally estimated via a gradient

method such as Adam [14].

In the adaptation phase, it is possible to ﬁne tune the NN further through a

reinforcement learning (RL) algorithm. The training setup varies based on the speciﬁc

choice of the RL algorithm. A comprehensive review of reinforcement learning is

clearly beyond the scope of this work. Interested readers may refer to the standard

text on this topic [28]. Lastly, we remark that this phase may be skipped if the NN

trained in the imitation phase is suﬃciently eﬀective for the target control application

or if the RL algorithm does not bring any improvement to the trained NN.

In practice, people ﬁnd that implicit MPC controllers can be well approximated

by a more eﬃcient NN controller [23]. However, it is not easy to certify stability of the

closed-loop system without bounding approximation errors between the NN controller

and the original MPC controller.

(4.6) x[i + 1] = f (x[i], u

(x[i])).

To solve the problem, we show that by composing the NN controller with the MPC

and the LQR in a hybrid control scheme, we can prove the local asymptotic stability

of the closed-loop system at an acceptable cost of computational eﬃciency, even if

the NN is random. Next, we present the basic form of our method, i.e., the standard

MAMPC.

4.3. Standard Memory-Augmented MPC. The hybrid control scheme com-

bines the LQR controller and the NN controller with the original MPC controller. At

every control step, if the state is in the region of attraction of the LQR, apply the

LQR controller; else, we simulate the closed-loop system (4.6) for up to N

LQR

steps:

剩余25页未读，继续阅读

易小侠

粉丝: 6592
资源: 9万+

LQR与神经网络结合的MPC：高效稳定控制新策略

MPC:神经网络动力学，用于基于模型的深度强化学习，且无模型精调

神经网络模型预测控制器

MPC.rar_MPC_mpc 约束_mpc控制_模型预测_预测 matlab

LQR控制.rar_LQR最优控制_LQR稳定性控制_lqr() simulink_基于LQR算法的_汽车控制

mpcsetup_MPCstatespace_模型预测控制_MPC预测控制_MPCmatlab_MPC模型预测_

神经网络的分类 GA_LQR_run.zip_ga lqr_run_分类_神经网络的分类 GA_LQR_run

dlb_LQR.rar_LQR控制_lqr_倒立摆LQR_状态反馈_状态反馈控制

LQR_someonexw7_LQR控制_lqrmatlab_LQR编程_lqr_源码

chapter5基于遗传算法的LQR控制器优化设计.rar_LQR 遗传_LQR控制_LQR最优控制_lqr优化_最优控制

ejidaolib.rar_LQR控制_LQR控制 simulink_ejidaolib_lqr仿真_二级控制

最新资源