Int J of Soc Robotics
as a motion capture system to acquire the coordinate data
information of the human actor. Furthermore, we proposed
an optimization technique to solve the inverse kinematics
(IK) problem, deriving the robot joint angles iteratively
through the Levenberg–Marquardt algorithm (LMA). The
pre-processed data were then transferred to NAO through the
wireless fidelity (Wi-Fi) connection and further processed for
the final control to follow human motions.
The remainder of the paper is organized as follows:
Sect. 2 summarizes the related work, where a number of
previous articles, which greatly inspire the control strate-
gies, motion capturing and balance control methods in this
study are described. The optimization approach based on
the LMA is proposed to solve IK in Sect. 3. Section 4
introduces the low-level control of the humanoid robot. In
Sect. 5 the balance control problem is discussed. The self-
collision avoidance technique is briefly described in Sect. 6.
Section 7 presents several simulations and experiments we
performed on our imitation system. Finally, Sect. 8 concludes
the paper.
2 Related Work
Baker and Kuniyoshi reviewed the early robot imitation
research and defined the notion of robot imitation as “an
agent learns a behavior from observing the execution of that
behavior by a teacher” [2]. A human imitation system for
robots provides a natural way for learning by directly extract-
ing information from the way human beings control their
bodies. In recent years, several exciting projects have been
presented in this area, but imitation of humans by robots
remains a difficult issue and several challenges need to be
faced.
The first step of robot imitation, as we have mentioned, is
to use some motion capture system to observe the postures
and motions of the human actor. To capture accurate human
motions and make the robot perform Chinese Kungfu “Taiji”
[17], Zhao et al. employed a motion capture system consist-
ing of six optical cameras and 38 markers attached to body
of a human actor. Koenemann and Bennewitz worked out an
imitation system based on the NAO robot which could exe-
cute challenging human actions [5,6]. The motion capture
phase of the system was realized through the Xsens MVN
System. Both of the two motion capture systems turned out
to be efficient, but the former was not straightforward to use
while the popular use of the latter system was limited because
of its high cost.
Therefore, a real-time easy-to-use motion capture system
with a lower cost is needed. The Kinect motion system not
only provides a fast and user-friendly approach in capturing
human motion, but also has the advantages of price, making
it attractive to many robot study groups. Zuher and Romero
used Kinect to recognize whole-body human motions for
robot imitation and control [18], but self-collision avoidance
(defined as when links are blocked by other parts of the body)
was not considered. The output information of Kinect makes
it necessary to apply some self-collision avoidance strategy
to ensure the motions can meet the mechanical limits.
Kinematics mapping is an important technique because
the differences in body size and DOF between the human and
robot need to be seriously considered. The measurement of
the similarity is always realized by optimizing a “similarity
function” describing the similarity (or difference) between
the imitator and the actor. Zhao et al. proposed a similarity
function based on the difference between the joint angles of
the robot and those of the human actor acquired by IK [12].
Do et al. develop a similarity function based on MMM by
adding a term that depicted the difference in the positions
of end effectors between the human and robot [4]. How-
ever, for actors and imitators of quite different sizes (NAO
is only a third of the size of a human) and structures, the
comparison between the end effector positions is no longer
reasonable. Besides, the IK process relies on the model of
a human body thus a modeling error is introduced into the
IK process, and this error is not taken into account in the
similarity function calculated from joint angles. Therefore,
developing a more reasonable measurement technique is a
pressing requirement.
Imitation studies can be roughly divided into two main cat-
egories: whole-body (or f ull-body) imitations and half-body
imitations. Some researchers focus their interest on arm ges-
ture imitations. Cheng and Kuniyoshi developed a human
imitation system based on the ETL-humanoid robot [3]. The
head and two arms of the actor were detected by a real-time
stereo vision system and then tracked by the robot. Thobbi
and Sheng proposed two approaches to solve the missing data
problem when learning arm gestures by imitation [15]. The
application and development of an imitation system has also
been reported by Shon et al. [13], where an arm mimicking
system was built followed by a Bayesian action selection to
determine the best action to take in a particular environment.
These studies can be classified into half-body imitations.
Because only the motions of the human upper body were
studied, the problem of falling down was not included in the
above research.
By contrast, what we are interested in is the full-body
imitation problem for humanoid robots. This study aims to
solve the actors full-body postures to find the joint angles
describing the full-body profile. This means the problem of
maintaining balance becomes quite critical and should no
longer be ignored. Therefore, it seems important to develop
effective methods to solve the IK problem to achieve highly
similar imitations.
The robot NAO used in Zuher and Romero’s experiment
could walk, sit and produce a hello gesture, but control of bal-
123