多机器人辅助的可扩展操作员分配：不宁Bandit方法
版权申诉
126 浏览量
更新于2024-07-06
收藏 1.73MB PDF 举报
"这篇论文‘用于多机器人协助的可扩展操作员分配一种不宁Bandit方法_Scalable Operator Allocation for Multi-Robot Assistance: A Restless Bandit Approach’是发表在IEEE Transactions on Control of Network Systems 2021年的一篇文章，作者包括Abhinav Dahiya, Nima Akbarzadeh, Aditya Mahajan和Stephen L. Smith。"
本文关注的是在多机器人系统中的人工操作员分配问题。在这种系统中，多个半自主机器人各自需要执行一系列独立的任务，每个任务都有可能失败并进入故障状态。当需要时，人类操作员可以协助或远程操作机器人。传统的马尔科夫决策过程（Markov Decision Process, MDP）技术在解决此类问题时面临可扩展性问题，因为随着机器人和操作员数量的增加，状态和动作空间会呈指数增长。
为了克服这个问题，论文提出了一种不宁Bandit（Restless Bandit）的方法。在Bandit理论中，"不宁"意味着各个臂（即不同的选择）的状态在没有被选择时也会发生变化，这很好地模拟了机器人即使未被操作员干预也可能发生故障的情况。作者推导出了操作员分配问题满足可指数化条件的情况，这使得可以应用Whittle指数启发式策略。这种可指数化条件易于检查，并且论文表明这些条件适用于广泛的问题场景。
关键洞察在于利用单个机器人价值函数的结构，从而得出可以分别验证的条件。这种方法使得问题的复杂度不再随机器人数量的增加而线性增长，提高了算法的可扩展性。通过这种方法，操作员可以更有效地被分配到需要的机器人上，优化系统的整体性能。
这篇论文为大规模多机器人系统的人工操作员分配提供了一种创新的、基于不宁Bandit的解决方案，解决了传统MDP方法的可扩展性挑战，有助于提升多机器人协作任务的效率和可靠性。
DAHIYA et al.: OPERATOR ALLOCATION FOR MULTI-ROBOT ASSISTANCE 3
B. Organization
The contents of this paper are organized as follows: The
multi-robot assistance problem is presented in Section II.
We discuss the general Restless Bandit Problem and define
property of indexability in Section III. In Section V, we present
two practical classes of transition functions and establish
conditions under which problem indexability is ensured. In
Section VI, we cover the calculation of Whittle index heuristic
and present an efficient policy for the problem. Next, we
present simulations of the problem in Section VII to examine
validity and performance of the presented policy. The paper
ends with a brief discussion and conclusion.
II. MULTI-ROBOT ASSISTANCE PROBLEM
Consider a decision support system (DSS), consisting of
a team of M human operators supervising a fleet of K
semi-autonomous robots. Each robot k ∈ K
:
= {1, . . . , K}
is required to complete a sequence of N
k
tasks to reach
its goal. We will use a fleet of robots delivering packages
in a city as a running example but similar interpretations
hold for other applications mentioned in previous sections
(e.g., robots reaching a sequence of configurations [12]). In
this case, the robot’s trajectory would correspond to a series
of waypoints that a robot needs to navigate to reach its
destination (goal location). At each waypoint, a robot can
either operate autonomously or be teleoperated by one of the
human operators. We assume that all human operators are
identical in the way they operate the robots and that a human
operator can operate at most one robot at a time. We now
provide a mathematical model for different components of the
system
1
.
A. Model of the robots
It is assumed that when operating autonomously, each robot
uses a pre-specified control algorithm to complete its task.
For the delivery robot example, this could be, for instance,
a SLAM-based local path planner that the robot uses for
navigating between the waypoints. We will not model the
details of this control algorithm but simply assume that this
control is imperfect and occasionally causes the robot to enter
a fault state while doing a task (e.g., delivery robot getting
stuck in a pothole or losing its localization). We model this
behaviour by assuming that while completing each task, the
robot may be in one of the two internal states: a normal state
(denoted by s = 0) or a fault state (denoted by s = 1). When
a robot is being teleoperated, it may still be possible for it to
enter into a fault state.
The operating state of robot k ∈ K at time t, denoted by
x
k
t
= (n
k
t
, s
k
t
), is tuple of its current task and internal state.
The state space for robot k is given by
X
k
:
=
N
k
[
n=1
{(n, 0), (n, 1)} ∪ {(G, 0)},
1
Remark on notation: Throughout this paper, we use calligraphic font to
denote sets and roman font to denote variables. Uppercase letters are used to
represent random variables and the corresponding lowercase letters represent
their realizations. Bold letters are used for variables pertaining to multi-robot
system while light letters represent corresponding single-robot variables.
where the terminal state (G, 0) indicates that all tasks have
been completed. The state space for all robots is denoted by
X
X
X = X
1
× · · · × X
K
.
The state of a robot evolves differently depending on
whether it is operating autonomously (denoted by mode a
k
=
0) or teleoperated (denoted by a
k
= 1). Given robot k ∈ K
in state (n, s) ∈ X
k
operating in mode a ∈ {0, 1}, let p
ka
ns
denote the probability of successfully completing the current
task at the current time step and let q
ka
ns
denote the probability
of toggling the current internal state (i.e. going from normal
to fault state and vice-versa). A diagram describing these
transitions is shown in Fig. 2. Note that the terminal state
(G, 0) is an absorbing state, so p
ka
G0
= 0 and q
ka
G0
= 0.
Fig. 2. State-transition diagram for robot k working on task n, where in
(n, 0) the robot is in the normal state s = 0, and in (n, 1) the robot is
in the fault state s = 1. Transitions can occur between (n, 0), (n, 1)
and (n + 1, 0), and the probabilities change with operating mode a.
There is a per-step cost C
k
: X
k
× {0, 1} → R
≥0
, where
C
k
((n
k
, s
k
), a
k
) denotes the cost of operating robot k ∈ K in
mode a
k
when the robot is in state (n
k
, s
k
). Note that the per-
step cost is zero in the terminal state, i.e, C
k
((G, 0), a) = 0.
B. Model of the decision support system (DSS)
There is a decision support system that helps to allocate
operators to the robots. At each time the decision support
system observes the operating state X
t
:
= (X
1
k
, . . . , X
K
t
)
of all robots and picks at most M robots to teleoperate. We
capture this by the allocation A
t
= (A
1
t
, . . . , A
K
t
) ∈ A, where
A =
a
:
= (a
1
, . . . , a
K
) ∈ {0, 1}
K
:
K
X
k=1
a
k
≤ M
. (1)
The allocation is selected according to a time-homogeneous
Markov policy π : X
X
X → A. The expected total cost incurred
by any policy π is given by
J(π) = E
π
∞
X
t=0
γ
t
K
X
k=1
C
k
(X
k
t
, A
k
t
)
X
0
= x
0
, (2)
where γ ∈ (0, 1) is the discount factor and x
0
= (x
1
0
, . . . , x
K
0
)
is the initial state with x
k
0
= (1, 0) for every k ∈ K.
C. Problem objective
We impose the following assumptions on the model:
(A1) Given an allocation a = (a
1
, . . . , a
K
) by the DSS, the
operating states of the robots evolve independently of
each other.
剩余14页未读，继续阅读
2022-07-14 上传
2023-03-11 上传
2023-09-19 上传
2023-03-25 上传
2023-03-16 上传
2023-04-23 上传
2023-07-11 上传
2024-05-25 上传
易小侠
- 粉丝: 6547
- 资源: 9万+
我的内容管理
展开
最新资源
- 构建Cadence PSpice仿真模型库教程
- VMware 10.0安装指南：步骤详解与网络、文件共享解决方案
- 中国互联网20周年必读：影响行业的100本经典书籍
- SQL Server 2000 Analysis Services的经典MDX查询示例
- VC6.0 MFC操作Excel教程：亲测Win7下的应用与保存技巧
- 使用Python NetworkX处理网络图
- 科技驱动：计算机控制技术的革新与应用
- MF-1型机器人硬件与robobasic编程详解
- ADC性能指标解析：超越位数、SNR和谐波
- 通用示波器改造为逻辑分析仪：0-1字符显示与电路设计
- C++实现TCP控制台客户端
- SOA架构下ESB在卷烟厂的信息整合与决策支持
- 三维人脸识别：技术进展与应用解析
- 单张人脸图像的眼镜边框自动去除方法
- C语言绘制图形：余弦曲线与正弦函数示例
- Matlab 文件操作入门：fopen、fclose、fprintf、fscanf 等函数使用详解
