![](https://csdnimg.cn/release/download_crawler_static/78086625/bg3.jpg)
DAHIYA et al.: OPERATOR ALLOCATION FOR MULTI-ROBOT ASSISTANCE 3
B. Organization
The contents of this paper are organized as follows: The
multi-robot assistance problem is presented in Section II.
We discuss the general Restless Bandit Problem and define
property of indexability in Section III. In Section V, we present
two practical classes of transition functions and establish
conditions under which problem indexability is ensured. In
Section VI, we cover the calculation of Whittle index heuristic
and present an efficient policy for the problem. Next, we
present simulations of the problem in Section VII to examine
validity and performance of the presented policy. The paper
ends with a brief discussion and conclusion.
II. MULTI-ROBOT ASSISTANCE PROBLEM
Consider a decision support system (DSS), consisting of
a team of M human operators supervising a fleet of K
semi-autonomous robots. Each robot k ∈ K
:
= {1, . . . , K}
is required to complete a sequence of N
k
tasks to reach
its goal. We will use a fleet of robots delivering packages
in a city as a running example but similar interpretations
hold for other applications mentioned in previous sections
(e.g., robots reaching a sequence of configurations [12]). In
this case, the robot’s trajectory would correspond to a series
of waypoints that a robot needs to navigate to reach its
destination (goal location). At each waypoint, a robot can
either operate autonomously or be teleoperated by one of the
human operators. We assume that all human operators are
identical in the way they operate the robots and that a human
operator can operate at most one robot at a time. We now
provide a mathematical model for different components of the
system
1
.
A. Model of the robots
It is assumed that when operating autonomously, each robot
uses a pre-specified control algorithm to complete its task.
For the delivery robot example, this could be, for instance,
a SLAM-based local path planner that the robot uses for
navigating between the waypoints. We will not model the
details of this control algorithm but simply assume that this
control is imperfect and occasionally causes the robot to enter
a fault state while doing a task (e.g., delivery robot getting
stuck in a pothole or losing its localization). We model this
behaviour by assuming that while completing each task, the
robot may be in one of the two internal states: a normal state
(denoted by s = 0) or a fault state (denoted by s = 1). When
a robot is being teleoperated, it may still be possible for it to
enter into a fault state.
The operating state of robot k ∈ K at time t, denoted by
x
k
t
= (n
k
t
, s
k
t
), is tuple of its current task and internal state.
The state space for robot k is given by
X
k
:
=
N
k
[
n=1
{(n, 0), (n, 1)} ∪ {(G, 0)},
1
Remark on notation: Throughout this paper, we use calligraphic font to
denote sets and roman font to denote variables. Uppercase letters are used to
represent random variables and the corresponding lowercase letters represent
their realizations. Bold letters are used for variables pertaining to multi-robot
system while light letters represent corresponding single-robot variables.
where the terminal state (G, 0) indicates that all tasks have
been completed. The state space for all robots is denoted by
X
X
X = X
1
× · · · × X
K
.
The state of a robot evolves differently depending on
whether it is operating autonomously (denoted by mode a
k
=
0) or teleoperated (denoted by a
k
= 1). Given robot k ∈ K
in state (n, s) ∈ X
k
operating in mode a ∈ {0, 1}, let p
ka
ns
denote the probability of successfully completing the current
task at the current time step and let q
ka
ns
denote the probability
of toggling the current internal state (i.e. going from normal
to fault state and vice-versa). A diagram describing these
transitions is shown in Fig. 2. Note that the terminal state
(G, 0) is an absorbing state, so p
ka
G0
= 0 and q
ka
G0
= 0.
Fig. 2. State-transition diagram for robot k working on task n, where in
(n, 0) the robot is in the normal state s = 0, and in (n, 1) the robot is
in the fault state s = 1. Transitions can occur between (n, 0), (n, 1)
and (n + 1, 0), and the probabilities change with operating mode a.
There is a per-step cost C
k
: X
k
× {0, 1} → R
≥0
, where
C
k
((n
k
, s
k
), a
k
) denotes the cost of operating robot k ∈ K in
mode a
k
when the robot is in state (n
k
, s
k
). Note that the per-
step cost is zero in the terminal state, i.e, C
k
((G, 0), a) = 0.
B. Model of the decision support system (DSS)
There is a decision support system that helps to allocate
operators to the robots. At each time the decision support
system observes the operating state X
t
:
= (X
1
k
, . . . , X
K
t
)
of all robots and picks at most M robots to teleoperate. We
capture this by the allocation A
t
= (A
1
t
, . . . , A
K
t
) ∈ A, where
A =
a
:
= (a
1
, . . . , a
K
) ∈ {0, 1}
K
:
K
X
k=1
a
k
≤ M
. (1)
The allocation is selected according to a time-homogeneous
Markov policy π : X
X
X → A. The expected total cost incurred
by any policy π is given by
J(π) = E
π
∞
X
t=0
γ
t
K
X
k=1
C
k
(X
k
t
, A
k
t
)
X
0
= x
0
, (2)
where γ ∈ (0, 1) is the discount factor and x
0
= (x
1
0
, . . . , x
K
0
)
is the initial state with x
k
0
= (1, 0) for every k ∈ K.
C. Problem objective
We impose the following assumptions on the model:
(A1) Given an allocation a = (a
1
, . . . , a
K
) by the DSS, the
operating states of the robots evolve independently of
each other.