Mean Field Team Decision Problems for Makov Jump
Multiagent Systems
Bing-Chang Wang
School of Control Science and Engineering, Shandong University, Jinan, 250061, P. R. China
E-mail: bcwang@sdu.edu.cn
Abstract: This paper studies the mean field team decision problem for multiagent systems with Markov jump parameters and
coupled indices. By analyzing the centralized strategy of the team problem, we get a parameterized equation. Then by solving
an optimal control problem in the augmented state space, we obtain the consistency equation, from which a set of distributed
strategies is designed. By constructing the Lypunov function, we show that the closed-loop system is uniformly stable, and the
set of distributed strategies is team-optimal.
Key Words: Team decision problem, mean field control, distributed strategy, Markov jump system
1 Introduction
In recent years, the study of mean field games and con-
trol has become a hot topic in the community of systems and
control [1]. Mean field models have wide application back-
grounds in many areas including economics, finance, com-
munication engineering, biology and medicine [2–4]. Such
models have been investigated by researchers in diverse ar-
eas from a variety of perspectives [5–14]. In mean field
models, each agent is affected by the average interaction of
all the other agents, while the individual influence of each
agent is negligible. From the relationship between popula-
tion macroscopic behavior and individual behavior, one can
get that the population aggregate effect satisfies a fixed-point
equation. Then by solving the fixed-point equation and the
single-agent optimal control problem, decentralized asymp-
totical Nash equilibria are obtained [6, 7, 13].
In practical financial markets, ecological systems, and so-
cial systems, surrounding environment is constantly chang-
ing. For instance, the change rates of prices in a finan-
cial market in different time slots may be very different.
A powerful tool depicting abrupt environmental changes
is the Markov jump model [15, 16]. Wang and Zhang
[13, 17, 18] investigated mean field game and control prob-
lems for Markov jump multigagent systems, and gave dis-
tributed asymptotical Nash equilibrium strategies.
With wide applications, team decision problems have a
long history [19–21]. In team decision problems, all the
agents have a common objective function, which is regarded
as the social index. Agents have different measurements or
information structures [21]. The team-optimal strategy is
globally optimal, hence it is not only person-by-person op-
timal, but also Pareto optimal. Under some convexity con-
ditions, the person-by-person optimal strategy is also team-
optimal [22]. Huang et al. [23] investigated social optima
for mean field LQG control models, and gave centralized
and decentralized team-optimal solutions.
This paper considers the team decision problem of mean
field models with Markov jump parameters and coupled in-
dices. Different from previous work [13, 23], the dynam-
This work is supported by National Natural Science Foundation of
China under Grant 61403233, and the Fundamental Research Funds of
Shandong University under Grant 2014TB007.
ics of all the agents are driven by the same continuous-time
Markov chain. Due to the impact of random parameters, the
population aggregate effect is a stochastic process depend-
ing on Markov jump parameters, instead of a deterministic
function. Thus, the population aggregate effect is no more
obtained by dealing with a fixed-point equation as in pre-
vious work [23]. We achieve the control synthesis by the
parametric approach and the state space augmentation. By
analyzing the centralized strategy of the team problem, we
get a parameterized equation. Then by solving an optimal
control problem in the augmented state space, we obtained
the consistency equations, from which a set of distributed
strategies is designed. By constructing the Lypunov function
and using the probability limitation theory, we show that the
closed-loop system is uniformly stable, and the set of dis-
tributed strategies is team-optimal.
The following notations will be used in the paper. ∥⋅∥
denotes the Euclidean vector norm or matrix norm induced
by Euclidean vector norm; 𝐼
𝑛
denotes an 𝑛-dimensional
identity matrix. For any vector 𝑥 with proper dimensions
and symmetric matrix 𝑄 ≥ 0, ∥𝑥∥
𝑄
=(𝑥
𝑇
𝑄𝑥)
1/2
.
𝐶
𝑏
([0, ∞), ℝ
𝑛
) denotes the class of 𝑛-dimensional bounded
continuous functions in [0, ∞).
2 Problem Formulation
Consider the multiagent system evolving by the following
dynamics:
𝑑𝑥
𝑖
(𝑡)=𝐴
𝜃(𝑡)
𝑥
𝑖
(𝑡)𝑑𝑡 + 𝐵
𝜃(𝑡)
𝑢
𝑖
(𝑡)𝑑𝑡 + ℎ(𝑡)𝑑𝑡
+𝐷
𝜃(𝑡)
𝑑𝑊
𝑖
(𝑡), 1 ≤ 𝑖 ≤ 𝑁, (1)
where 𝑥
𝑖
∈ ℝ
𝑛
and 𝑢
𝑖
∈ ℝ
𝑟
are the state and input of the 𝑖th
agent, and {𝑊
𝑖
(𝑡), 1 ≤ 𝑖 ≤ 𝑁} is a family of independent
𝑑-dimensional Brownian motions. ℎ ∈ 𝐶
𝑏
([0, ∞), ℝ
𝑛
) is
an external signal, reflecting the impact on the 𝑖th agent by
the environment. {𝜃(𝑡)} is a continuous-time Markov chain
taking value in 𝑆 = {1, 2,...,𝑚} with the transition rate
matrix (infinitesimal generator) Λ={𝜆
𝑖𝑗
,𝑖,𝑗 =1,...,𝑚}.
The index of the 𝑖th agent is
𝐽
𝑖
(𝑢) = lim sup
𝑇 →∞
1
𝑇
𝐸
𝑇
0
𝑥
𝑖
(𝑡) − Φ[𝑥
(𝑁)
(𝑡)]
2
𝑄
𝜃(𝑡)
+∥𝑢
𝑖
(𝑡)∥
2
𝑅
𝜃(𝑡)
𝑑𝑡, (2)
Proceedings of the 34th Chinese Control Conference
Jul
28-30, 2015, Han
zhou, China
1845