A 3D Virtual Reality Simulator for Training of Minimally Invasive
Surgery
Shao-Hua Mi, Zeng-Gunag Hou, Fan Yang, Xiao-Liang Xie and Gui-Bin Bian
Abstract— For the last decade, remarkable progress has been
made in the field of cardiovascular disease treatment. However,
these complex medical procedures require a combination of
rich experience and technical skills. In this paper, a 3D virtual
reality simulator for core skills training in minimally invasive
surgery is presented. The system can generate realistic 3D
vascular models segmented from patient datasets, including a
beating heart, and provide a real-time computation of force
and force feedback module for surgical simulation. Instruments,
such as a catheter or guide wire, are represented by a multi-
body mass-spring model. In addition, a realistic user interface
with multiple windows and real-time 3D views are developed.
Moreover, the simulator is also provided with a human-machine
interaction module that gives doctors the sense of touch during
the surgery training, enables them to control the motion of a
virtual catheter/guide wire inside a complex vascular model.
Experimental results show that the simulator is suitable for
minimally invasive surgery training.
I. INTRODUCTION
Minimal invasive surgery is a superior treatment paradigm
for treating cardiovascular disease and stroke [1]. In a real
treatment paradigm, by pushing, pulling and twisting at the
proximal end of a catheter/guide wire, the catheter/guide
wire is manipulated under X-ray guidance through the ab-
dominal aorta system then into the aortic arch. Comparing
with the open surgery, the main advantages of this surgical
technique are smaller incisions and less painful. However,
these complex medical procedures require a combination of
rich experience and technical skills. It is difficult for doctors
to control a flexible catheter/guide wire by pushing, pulling
and twisting at the proximal end of the catheter/guide wire
only under the X-ray projection images. Therefore, in order
to complete a minimally invasive surgery, extensive training
is required [2-4].
Traditionally, the best training environment on which
the hospitals train the learners is mainly based on living
animals or actual patients. Developing a real-time interactive
3D computer-aided surgical simulation is desirable and can
provide a alternative to traditional training environment. In
this way, the virtual reality simulator using a haptic device
can help the doctors obtain the core skills of controlling a
catheter/guide wire and decrease the risk of errors.
Several virtual reality simulators have been developed or
commercialized in projects [5], [6]. However, these systems
do not simulate the catheter/guide wire insertion procedures
The authors are with the State Key Laboratory of Management and
Control for Complex Systems the Institute of Automation, Chinese Academy
of Science, Beijing shaoHua.mi@ia.ac.cn
with a complex vascular structure and do not deal with
the real-time interactions between a catheter/guide wire
and the vascular model or between a haptic device and
the catheter/guide wire. In this paper, a 3D virtual reality
simulator for training in minimally invasive surgery (in real-
time or near real-time) is presented. Comparing with the
existing system, this simulator can provide a realistic 3D
virtual training environment and support real-time human-
machine interactions. The system has the following charac-
teristics: provides realistic and complex 3D vascular models,
which is generated from computed tomography angiography
(CTA) series in DICOM datasets captured in real patients;
provides a real-time virtual catheter/guide wire based on
multi-body mass-spring physical model; provides a real-time
force computation and force feedback computation module;
provides a real-time 3D rendering module and a real-time or
near real-time human-machine interaction module [7-9].
Organization of the paper is as follows. Section II outlines
an overview of the system. Section III describes the exper-
iments and presents the results. Section IV introduces the
future works and concludes this paper.
II. SYSTEM OVERVIEW
Fig. 1 overviews the real-time interactive 3D virtual reality
simulator. The system consists of six major components:
kernel module, device controller module, 3D vascular model
module, catheter/guide wire model module, user interface
module and human-machine interaction module.
Device Controller
Force Computation
Feedback Computation
3D Vascular Model
Geometric Model
Physical Model
Heart-beat Model
CTA Datasets
Kernel
3D Rendering
Math Library
Collision Detection
Collision Response
User Interface
Scene Stats
GUI
Human-machine Interaction
Keyboard Mode
Haptic Device Mode
Catheter/Guide wire
Model
Geometric Model
Physical Model
Fig. 1. Overview of the system
The prototype for the simulator is shown in Fig. 2.
A. 3D Vascular Model
A realistic and complex 3D vascular geometric model is
imported into the virtual environment of the system, which
is generated from computed tomography angiography (CTA)
series in DICOM datasets captured in actual patients, as
978-1-4244-7929-0/14/$26.00 ©2014 IEEE 349