220 IEEE TRANSACTIONS ON ROBOTICS, VOL. 33, NO. 1, FEBRUARY 2017
Automatic Sample Alignment Under Microscopy for 360° Imaging Based on the
Nanorobotic Manipulation System
Yajing Shen, Member, IEEE,WenfengWan, Member, IEEE, Haojian Lu, Member, IEEE,
Toshio Fukuda, Fellow, IEEE, and Wanfeng Shang
Abstract—Microscopy has been an indispensable tool for micro/
nanosample imaging, manipulation, and characterization. However, view-
ing the micro/nanosample from multidirection is still a big challenge for
current microscopy. To address the above issue, this paper proposes a
novel nanorobotic manipulation system for the automatic alignment and
multidirectional imaging under microscopes. First, a miniature rotation
robot with three degrees of freedom is designed and integrated with a mi-
croscope. Then, a forward-backward alignment strategy containing three
loops, i.e., position shift loop, angle loop, and magnification loop, is proposed
to align the sample to the rotation axis of the robot automatically. After that,
the sample is imaged from multidirection by rotating the robot with one
revolution (360°). Finally, the alignment accuracy is evaluated and multi-
directional images of various samples are implemented. This study provides
a new way for the microscopic imaging, which is expected to exert a signifi-
cant impact in multiple fields on a small scale, including microscopy imag-
ing, microdefect detection, micromanipulation, in situ characterization,
and so on.
Index Terms—Microscopy multidirectional imaging, nanoautomation,
nanomanipulation, nanorobotic manipulation system, rotation alignment.
I. INTRODUCTION
Viewing the micro/nanoobject with high magnification and large
view area at small scale is essentially required in many fields, such as
for microdefect detection, precise manipulation, and in situ material
characterization [1], [2]. However, viewing samples from multidirec-
tion under microscopy is still very challenge until now limited to the
low manipulation flexibility of the current system. The image infor-
mation obtained from the sole direction cannot fully reflect the overall
properties of the sample, resulting in some research works become less
complete and reliable. For instance, lack of image information from
Manuscript received April 23, 2016; revised August 10, 2016; accepted Oc-
tober 5, 2016. Date of publication December 1, 2016; date of current version
February 3, 2017. This paper was recommended for publication by Associate
Editor J. Abbott and Editor P. Dupont upon evaluation of the reviewers’ com-
ments. This work was supported in part by the National Natural Science Foun-
dation of China under Grant 61403323 and 51305338, in part by the General
Research Fund of Hong Kong under Grant 21201314, and in part by the Shen-
Zhen (China) Basic Research Project under Grant JCYJ20160329150236426.
Y. Shen is with the Department of Mechanical and Biomedical Engineering,
City University of Hong Kong, Kowloon Tong, Hong Kong, and also with
City University of Hong Kong Shenzhen Research Institute, Shenzhen 518057,
China (e-mail: yajishen@cityu.edu.hk).
W. Wan and H. Lu are with the Department of Mechanical and Biomedi-
cal Engineering, City University of Hong Kong, Kowloon Tong Hong Kong
(e-mail: wfwan2-c@my.cityu.edu.hk; haojianlu2-c@my.cityu.edu.hk).
T. Fukuda is with the School of Mechatronic Engineering, Beijing Institute
of Technology, Beijing100081, China (e-mail: tofukuda@meijo-u.ac.jp).
W. Shang is with the Department of Mechanical Engineering, Xi’an Univer-
sity of Science and Technology, Xi’an710000, China (e-mail: shangwanfeng@
gmail.com).
This paper has supplementary downloadable material available at http://
ieeexplore.ieee.org provided by the author. This video illustrates the align-
ment principle, the alignment process, and compares the result before and after
alignment.
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org
Digital Object Identifier 10.1109/TRO.2016.2623337
multidirection may lead to faulty judgement in the defect detection, and
may also affect the validity of sample investigation and characterization
[3, 4].
To address the above issue, microscopy with multiple lenses has
been proposed, such as mounting optical microscope (OM) on the
side of scanning electron microscope (SEM) chamber [5], assembling
a miniature charge-coupled device camera [6], or using three OMs
together [7]. Although the above system could provide multidirectional
images as many as the number of microscope lens, sample imaging
from 360° is still not available since there is no space to install so many
lenses. Besides, the microscopy lenses are fixed without movability,
and thereby the controllability and flexibility of this method is low.
Tilting stage is another widely used solution for multidirectional
imaging, which is able to tilt the sample with a certain angle (<
±90°) under microscopy [8]–[10]. C ompared with the former one, this
approach does not require the installation of multiple lenses, which
makes it more acceptable and practicable for general applications. For
instance, this kind of platform has been successfully used to observe
the microsample under OM [11], to construct three-dimensional recon-
struction under dual-beam SEM [12], to investigate the nanomaterials
under transmission electron microscope (TEM) [13], and so on. How-
ever, in this method, there i s still more than half of the sample that is out
of view due to the limited tilting angle caused by the inherent mechani-
cal structure. Other disadvantages of existing tilting stages include that
they are operated manually without feedback control and samples are
subject to moving out of microscope’s field of view (FOV). Therefore,
the application of the above technique is still restricted in some specific
research areas, and not suitable for the wide applications in practice.
The rapid progress of robotic technique provides new opportunities
for the precise manipulation of micro/nanoobject. For instance,
scientists have proposed various types of nanorobots and realized
the nanomaterial manipulation, electrical property characterization,
DNA manipulation, single-cell characterization, and so on [14]–[19].
However, at current stage, few nanorobotic manipulation systems have
the rotation degree to allow sample’s rotation and multidirectional
imaging under microscopes.
Herein, we develop a nanorobotic manipulation s ystem and the rel-
evant control strategy to realize the automatic alignment and multidi-
rectional imaging under microscopy. This paper is organized as fol-
lows. Section II illustrates the robot development, alignment principle,
alignment procedure optimization, and the control system. Section III
implements the alignment, discusses the accuracy and demonstrates
the multidirectional imaging. Lastly, we draw the conclusion to this
paper and discuss its potential applications and long-term impacts.
II. AUTOMATIC SAMPLE ALIGNMENT
A. Nanorobotic Manipulation System
As illustrated in Fig. 1, the nanorobotic manipulation system is
designed with three degrees of freedom (DOFs), i.e., two mutually
1552-3098 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications
standards/publications/rights/index.html for more information.