See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/12763038

Virtual Reality Based Surgery Simulation for Endoscopic Gynaecology

Article  in  Studies in Health Technology and Informatics · February 1999

DOI: 10.3233/978-1-60750-906-6-351 · Source: PubMed

CITATIONS READS

40 133

18 authors, including:

Gábor Székely Michael Bajka

ETH Zurich University of Zurich
331 PUBLICATIONS   12,620 CITATIONS    99 PUBLICATIONS   1,839 CITATIONS   

SEE PROFILE SEE PROFILE

Jürg Dual
ETH Zurich
281 PUBLICATIONS   3,831 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

UltraVR View project

Microrobots for biomedical applications View project

All content following this page was uploaded by Michael Bajka on 27 March 2016.

The user has requested enhancement of the downloaded file.

 Virtual Reality Based Surgery Simulation for
Endoscopic Gynaecology
G. Székely PhD∗, M. Bajka MD‡, Ch. Brechbühler PhD∗, J. Dual PhD∗,
R. Enzler MSc∗, U. Haller MD†, J. Hug MSc∗, R. Hutter MSc∗,
N. Ironmonger MSc∗, M. Kauer MSc∗, V. Meier MSc∗, P. Niederer PhD∗,
A. Rhomberg MSc∗, P. Schmid PhD∗, G. Schweitzer PhD∗, M.Thaler PhD∗,
V. Vuskovic MSc∗and G. Tröster PhD∗
∗
Swiss Federal Institute of Technology, CH-8092 Zürich, Switzerland
†
Department of Gynaecology, University Hospital, CH-8091 Zürich
‡
Clinic of Gynaecology and Obstetrics, Hospital Uster, CH-8610 Uster

Abstract
Virtual reality (VR) based surgical simulator systems offer very elegant pos-
sibilities to both enrich and enhance traditional education in endoscopic surgery.
However, while a wide range of VR simulator systems have been proposed and
realized in the past few years, most of these systems are far from able to provide a
reasonably realistic surgical environment. We explore the basic approaches to the
current limits of realism and ultimately seek to extend these based on our descrip-
tion and analysis of the most important components of a VR-based endoscopic
simulator. The feasibility of the proposed techniques is demonstrated on a first
modular prototype system implementing the basic algorithms for VR-training in
gynaecologic laparoscopy.

1 Introduction
Endoscopic operations have recently become a very popular technique to minimize the
damage of the surrounding healthy tissue normally caused by the process of reaching
the more inaccessible internal organs during interventions. The relatively large cuts in
open surgery can be replaced by small perforation holes, serving as entry points for
optical and surgical instruments. The small spatial extent of the tissue injury and the
careful selection of the entry points result in a major gain in patient recovery after
operation.
The price for these advantages is paid by the surgeon who loses direct contact with
the operation site. The necessary visual information is mediated by a (usually mono-
scopic) specialized camera (the endoscope) and is presented on a screen, distracting
normal hand-eye coordination. Due to geometrical constraints posed by the external
control of the surgical instruments through the trocar hull, the surgeon loses much of
the manipulative freedom usually available in open surgery.
Performing operations under these conditions demands very specific capabilities of
the surgeon, which can only be gained with extensive training. Virtual reality based
 (a) (b) (c) (d)
Figure 1: Texture analysis/synthesis: (a,b) original and artificial uterus, (c,d) original
and artificial ovary.

surgical simulator systems offer a very elegant solution to this training problem. A
wide range of VR simulator systems have been proposed and implemented in the past
few years. Some of them are restricted to purely diagnostic endoscopic investigations
[3], while others, for example, allow the training of surgical procedures for laparoscopic
[2, 4], arthroscopic [1], or radiological [10] interventions.
The basic advantage of VR-based simulators would be that they can theoretically
provide a realistic and configurable training environment bridging the gap between basic
training and the performing of actual interventions on patients. However, the simulator
systems proposed up to now are some way from providing the necessary level of realism
in training. While this goal cannot be reached by today’s technology, it is of major
importance to explore the current limits of realism in endoscopic surgery simulation
and to analyze the possibilities for further development. This paper describes a first
attempt to meet these long-term research objectives. The various components of the
project will be described in detail in the following sections.
Besides theoretical and algorithmic results, a first working prototype of the planned
system with modular system architecture running on an SGI Onyx2/InfiniteReality
hardware is described in Section 8. This system allows flexible adaptation to different
hardware components with widely varying performance capabilities and serves as a
platform for ongoing and future developments.

2 Anatomical model building

Realistic simulation of the elastic deformation of abdominal organs and the resulting
forces is only possible on the basis of a detailed anatomical model. The motion and
deformation of organs is in many cases determined by morphological structures of small
spatial extent (e.g., ligaments), posing a major challenge for the generation of the model.
Customary radiological imaging procedures cannot provide the necessary information
due to serious constraints in image contrast and resolution. The Visible Human Female
data set of the National Library of Medicine [5], however, offers a consistent source of
anatomical and morphological information providing very high resolution and excellent
tissue contrast at the same time. Consequently, this dataset has been selected to provide
the basis for the construction of the anatomical model.
While organ definition is inherently a 3D task, we relied on the traditional slice-
by-slice technique using the familiar two-dimensional outlining methodology. This,
mostly due to the possibility of applying simple and intuitive interfaces for interaction
and visualization. To establish a powerful segmentation environment, we implemented
a complete multi-user segmentation system with an underlying anatomical database.
Special attention was paid to an intuitive user interface and very fast data access.
All abdominal organs influencing the elastomechanical behavior and visual appear-
ance of the operation site during gynecological laparoscopic interventions were seg-
mented within the aforementioned environment. In addition to these, the inner surface
of the abdominal cavity has been included. This multi-organ surface is very helpful
during simulation as it surrounds the potential space where the gas is insufflated at the
beginning of the surgery, and the confines to which the surgeon places the instruments.

3 Graphical modeling of organ appearance

Providing correct visual information is indispensable in laparoscopy simulation if re-
alistic training environment is required. Even though, visual feedback is the widest
information channel available to the surgeon, current surgical simulation systems often
treat the graphics aspect in an ad-hoc manner. Whereas visualization for laparoscopy
simulation involves the treatment of a fairly wide range of topics, we will restrict the
following discussion to the computation of organ specific textures.
All organ surfaces are covered by some micro-structure, which gives us information
about the type of the tissue and its relative smoothness or coarseness. Methods to
simulate such textures are provided by almost all visualization packages. Texturing,
however, is not only useful because it increases object realism, but also provides a
cue to space perception. In addition, texturing is indispensable in order to represent
pathological tissue allowing the improvement of diagnostic skills during training.
Geometry independent, organ specific base textures, i.e. textures without blood
vessels, can be generated by means of an automatic texture analysis/synthesis proce-
dure [7]. Thereby a statistical description of a small texture sample taken from real
laparoscopic imagery is computed in an analysis phase. In a subsequent optimization
step a 3D texture block, initially consisting of white noise, is modified until its second
order statistics is close enough to the description of the sample texture. Organs of
arbitrary shape can then be textured by carving them out of the solid texture block
(Figure 1 a,b). Procedural textures [8] can be used to modulate the resulting texture
patterns by both low frequency variations, and sparsely distributed features, which
cannot be efficiently handled by statistical techniques.
Whwrwas, geometry and texture have so far, been completely decoupled, this is not
possible in some cases. Figure 1 (c) shows an ovary covered by follicles distinctive in
both texture and geometry. The framework of procedural textures can also be used
to combine an automatically generated ovary texture with randomly selected follicle
images, taking into account their positions and sizes (Figure 1 c,d).
In order to generate organ specific blood vessels we have developed a method based
on L-systems [6]. They are widely used in computer graphics to generate artificial
plants. Basically, simple structures become more and more complex by simultaneously
deriving their individual components according to given stochastic rules. Although L-
systems are very suitable for generating trees, they have their disadvantages in the case
of net-like structures. Current research is therefore directed towards models that simu-
late the biological growing process of vessels. The appropriate texture map combining
the organ’s base texture with generated blood vessels along its surface can be computed
by means of a distributed ray-tracing approach.
Figure 2: Left: Finite Element model of the uterus containing about 2000 elements.
Middle and right: Simulated deformation of the corpus and the fallopian tube of the
uterus

4 Tissue Deformation Modeling

The Finite Element Method (FEM) is a very common and accurate way to solve
continuum-mechanical boundary-value problems. Difficulties arising in application to
biological tissue occur from our need to deal with large deformations, and anisotropic,
inhomogeneous, and nonlinear materials. Furthermore, organs and surgical instruments
interact, leading to numerical contact problems. Nonetheless, provided an adequate for-
mulation is chosen, even in these cases the FEM is a very powerful tool.
Within the FEM a body is subdivided into a finite number of well defined elements
(e.g., hexahedrons, tetrahedrons, quadrilaterals). Displacements and positions in the
element are interpolated from discrete nodal values. For every element the partial
differential equations governing the motion of material points of a continuum can be
formulated resulting in a discrete system of differential equations.
This dynamic equation has to be integrated with respect to time. Explicit time
integration can be performed without iteration and without solving a system of linear
algebraic equations. This integration scheme is only conditionally stable; that is, only
very small time steps lead to a stable solution. In the case of the model of an uterus
shown on the left of Figure 2, 10’000 time steps per second are required to maintain
stability.
A reduced volume integration scheme based on absolute strain formulation using a
hyper-elastic material law has been developed for modeling uterus deformation. The
method has been extensively tested by off-line simulations of deformations using a first
model of the uterus. Figure 2 left shows the generated FEM mesh, containing about
2000 elements. The abdominal cavity has been modeled as a rigid surface. Simulated
deformations of the uterus are illustrated on the middle and right image of Figure 2.

5 In vivo measurement of tissue elasticity

Realistic tissue deformation modeling cannot be performed without the knowledge of
the elastic properties of living tissue. Even the best description of the mechanical
behavior of tissue is useless, if its parameters cannot be determined. The selection of
the constitutive equation for modeling a specific organic material should therefore be
followed by the determination of the actual numerical values of material parameters.
Since significant differences are expected between the mechanical properties of dead
and living human and animal tissue, an additional requirement is that we perform the
measurements in vivo on patients during interventions.
One technique capable to provide accurate data is tissue aspiration in conjunction
with the inverse Finite Element method. The inverse Finite Element method allows the
greatest freedom concerning the boundary conditions and the geometry of the problem,
and due to the complexity encountered in in-vivo experiments, this freedom seems
necessary to allow apt realization of the material parameter estimation.
In the aspiration experiment a weak vacuum is applied and carefully increased while
the deformation of the surface is tracked over the whole loading time. The main ad-
vantage of this method is that the boundary conditions of the experimental setup are
well defined by the contact between the aspiration tube and the tissue surface.
The components of the measuring instrument are basically the aspiration tube, the
pneumatic system, the part measuring the surface deformation and the data acquisition
system. Assuming axisymmetry and homogeneous tissue in the portion covered by the
aspiration tube, it suffices to measure a profile of the deformed tissue. Using a small
inclined mirror beside the aspiration hole, it is possible to track the profile of the
deformed tissue with a camera placed on the other end of the tube.

optic graphic

>= 24 Hz
position 24 Hz

4 Reals
Global
Network
ix 2000 * 6 Reals

4 Reals
node coordinates
and normals
force

instrument mechanic
Processing Element
1 kHz 10 kHz

Figure 3: Left: 3D communication architecture of the multiprocessor network. Right:

the modular structure of the prototype simulator.

6 Design of a real-time FEM Computation Engine

The only way to provide the necessary computational power for the real-time solution
of complex Finite Element systems is to build a parallel computer which supports
fully parallel algorithms for the explicit time integration scheme. Algorithmic design
as described below allows to scale the computation to the necessary speed with the
selection of an appropriate number of processor units or processing elements (PE).
Analysis of optimized explicit Finite Element algorithms shows that approximately
700 floating point operations per element are needed in each time step. Additional
computation time has to be reserved for collision detection and handling. This leads
to 10 MFLOPS (Million Floating Point Operations Per Second) per finite element for
the chosen time step of 100 µs. For 2000 elements, a total of 20 GFLOPS sustained is
needed. This performance can only be reached by a high performance parallel machine.
A 3D lattice of processors, where every processing element (PE) is connected to its
six neighbors is an optimal configuration for the organization of local communication
(Figure 3 left).
 Explicit FE computation suggests element-wise parallelism, resulting in a spatial
decomposition of the problem. Each element is mapped to one processor, and a pro-
cessor computes the internal forces of several elements. Whenever elements residing
on different processors share common nodes, these nodes must be represented on all
involved processors. The resulting force acting on such a distributed node emerges from
the communication among the contributing processors, which treat the foreign forces
like external forces, adding them to their own.
The mapping of elements to processors should balance the computational load and
keep the necessary communication bandwidth within manageable limits. We use a
heuristic solution for this discrete optimization problem minimizing the overall compu-
tation time.

7 Force feedback manipulator

Although the tactile information mediated by the surgical instruments during laparo-
scopic surgery is strongly limited, force feedback is an indispensable component of any
realistic simulation environment. Until recently, no really satisfactory technical solu-
tions have been presented for providing tactile and force feedback for the simulation
of open surgery. However, during minimal invasive operations, haptic information is
provided exclusively by mechanical manipulators, constraining the implementation re-
quirements of simulated surgical instruments, thereby allowing realistic force feedback
based on the technology available today. In our system we plan to build the simulated
surgical instruments on the basis of the commercially available PHANToM device [9],
which conforms to these needs.

8 Prototype simulator for laparoscopic gynaecology

Although the dedicated hardware is not yet assembled and functional, we have im-
plemented the algorithms both for the preparatory steps of partitioning the mesh and
for the actual parallel computation, as well as a complete system model, simulating
not-yet-existing devices.
We can identify the following five units that work together in the simulator, each
being a more or less independent entity. The instrument device encodes the position
of the virtual instrument controlled by the user and feeds back the resulting contact
forces. The optic device encodes the position of the virtual endoscope and hence the
viewpoint of the camera. The mechanic engine performs the computationally expensive
FEM calculations. The graphics engine renders the scene as seen through the endo-
scope. These four units are connected through a link (ix) which establishes all links
of communication. A corresponding modular organization of the simulator (Figure 3
right) makes it simple to reconfigure, allowing fast adaptation to user-specific needs
and more efficient prototyping.
Partitioning is a preliminary step and can be done without the need of any particular
hardware. It effectively schedules in all details the steps of the later computation and the
exchange of partial results between the PEs of the mechanic engine. Today, the parallel
computation itself is carried out by several UNIX processes, each process simulating one
PE capable of handling time steps of about 2ms on a very simplistic model consisting
of 21 elements. Calculations are performed on an 8-processor (MIPS R10000 175MHz)
SGI Onyx2.
 9 Conclusion
The presented work summarizes the first steps for the construction of realistic virtual
reality based endoscopic surgery simulators. Experience gained during this project
phase has shown, that while a lot of theoretical and practical difficulties remain to
be overcome, sufficiently realistic simulator components mediating visual and haptic
information to the user are in principle within reach and can be expected to provide
satisfactory results in the foreseeable future.
Fundamental difficulties are still to be expected in the field of physically based
simulation of soft tissue behavior. Despite the initial promising results shown for the
simulation of diagnostic laparoscopic procedures using FEM techniques, even the case
of “simple” deformation modeling warrants significant basic research to achieve a de-
sirable level of realism. In the field of elastic tissue property measurements, results
are even more preliminary, and longterm research and development efforts, combined
with the establishment of complex anatomical databases containing integrated geo-
metrical, anatomical, physiological and elastomechanical data (including information
about pathological changes) are needed before VR-based simulators can comply with
the expectations of the surgeon.

References
[1] R. Ziegler, W. Mueller, G. Fischer and M. Goebel: A Virtual Reality Medical Training
System, Proc. 1st In. Conf. on Comp. Vision, Virtual Reality and Robotics in Medicine,
CVRMed’95, Nice, Lecture Notes in Comp. Sci., 905:282–286, Springer-Verlag, 1995
[2] U.G. Kühnapfel, H.G. Krumm, C. Kuhn, M. Hübner and B. Neisius: Endosurgery Sim-
ulations with KISMET: A flexible tool for Surgical Instrument Design, Operation Room
Planning and VR Technology based Abdominal Surgery Training, Proc. Virtual reality
World’95, Stuttgart, 165–171, 1995
[3] A.M. Alyassin, W.E. Lorensen: Virtual Endoscopy Software Application on a PC, Proc.
MMVR’98: 84–89, IOS Press, 1998
[4] Ch. Baur, D. Guzzoni and O. Georg: Virgy: A Virtual Reality and Force Feedback Based
Endoscopy Surgery Simulator, Proc. MMVR’98:110–116, IOS Press, 1998
[5] The Visible Human Project, National Library of Medicine,
http://www.nlm.nih.gov/research/visible/visible human.html
[6] P. Prusinkiewicz, A. Lindenmayer: The algorithmic beauty of plants, Springer-Verlag,
New York, 1990
[7] A. Gagalowicz, S.D. Ma: Sequential Synthesis of Natural Textures, Computer Vision,
Graphics, and Image Processing, pages 289-315, 1985
[8] K. Perlin: An Image Synthesizer, Proc. SIGGRAPH’85, pages 287-296, 1985
[9] T.H. Massie and J.K. Salisbury: The PHANToM Haptic Interface: A Device for Probing
Virtual Objects, in Dynamic Systems and Control, C.J. Radcliffe (Ed.), ASME DSC-55
1 pp.295-301, 1994
[10] J.K. Hahn, R. Kaufman, A.B. Winick, Th. Carleton, Y. Park, R. Lindeman, K-M Oh,
N. Al-Ghreimil, R.J. Walsh, M. Loew, J. Gerber and S. Sankar: Training Environment
for Inferior Vena Caval Filter Placement, Proc. MMVR’98:291–297, IOS Press, 1998

View publication stats