Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

accurate calibration and registration, and convenient visualization methods, AR could greatly aid
in the clinic for surgery.
In this section, we provide detailed technical descriptions of a surgical microscope, including
its optical system, illumination, mechanical system, and visualization. Advanced technologies
employed with surgical microscopes for image-guided surgery will be explained, namely the
AR, intraoperative fluorescence imaging, microscope-mounted OCT, HSI, and photoacoustic
imaging. The purpose of this section is to provide a comprehensive explanation of the principle
of the surgical microscope and how advanced technologies are adopted. It provides references
for microscope selection and system development.

3.1 Optical System

The optical system of the microscope is the main determinant of the imaging quality that a sys-
tem can achieve. It is basically a binocular (with eyepieces on top) with a close-up lens, namely
the optical components including the objective lens and the magnification changer (or zoom
changer).26,115 The focal length of the objective lens fully determines the value of working dis-
tance, which is the distance from the objective lens to the point of focus of the optical system.
The zoom changer is either a series of lenses moving in and out of the viewing axis or a system
that changes the relative positions of lens elements.115 The binocular is equivalent to two tele-
scopes hinged together, wherein prisms are used for a compact size of the unit. Stereopsis, which
introduces the depth information into the surgeon’s vision, is an important feature brought by
binocular and will be discussed in the visualization section.

3.1.1 Magnification
Clinicians from different fields have recognized the usefulness of
magnification.10,25,62,72,82,103,114,121
The total magnification (M total ) of a surgical microscope is determined by all the four optical
components in the microscope, namely the focal length of the objective lens (f OBJ ), zoom value
(MZOOM ), the focal length of binocular (f TUBE ), and the magnifying power of eyepieces
(MEP ),115 as Eq. (1)
f TUBE
Mtotal ¼ × MEP × M ZOOM : (1)
f OBJ
EQ-TARGET;temp:intralink-;e001;116;362

Magnification of modern surgical microscopes varies from 4× to 40× 10,73,122 and is usually
selected through a manual or motorized magnification changer. The zoom value is usually 6:1
but can be as high as 8:1.123 For some microscopes, an additional magnification multiplier
is applicable, which provides 40% more magnification.124 Resolution measures the acuity
improved by magnification. It is the ability of an optical system to distinguish two separate
entities.74 Human eyes have an inherent resolution of 0.2 mm125 but with 20× magnification,
it can be increased to 0.01 mm.126 This can add more confidence to surgeons, enhance the
advanced surgical skills, and enable the use of many fine surgical instrumentations when they
operate on fine anatomical structures.121

3.1.2 Optics
The design of optics is vital to the image quality of a surgical microscope. Aberration is an
inherent property of optical systems, and it causes the blur or distortion of images, which is
adverse to the desire for a clear view. Monochromatic aberrations such as spherical aberration,
coma, and astigmatism can be corrected but usually only for one color.127 Chromatic aberration
is a failure of a lens to focus all colors to the same point, because of which images show color
fringes and lose sharpness. Chromatic aberration correction is necessary for optics in a surgical
microscope not only because of the wide-band light source used but also due to the image
enhancement in cameras, such as sharpening and edge enhancement, which enhances the image
edge as well as the color fringes.128 Achromatic lens, which is a combination of converging and

Journal of Biomedical Optics 010901-9 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

diverging lens elements, was employed in early surgical microscopes to correct the primary
spectrum, leaving the secondary spectrum being the main factor limiting the image
quality.16,19 The apochromatic lens is the answer to that problem. It not only corrects for two
wavelengths (red and blue) to reduce spherical aberration but also utilizes the exceptional quality
optical materials that have unusual and desirable characteristics to reduce chromatic aberration
for three wavelengths (red, green, and blue).19,129

3.1.3 Focusing
Focusing is essential for a clear view. Surgeons would want the surgical site to be in focus
throughout the surgery. However, the shape of organs or the deep cavities makes it impossible
for the whole surgical site to be perfectly on the focal plane. Depth of focus, in other words,
depth of field (DOF), is a term that indicates the area in front of and behind the point of perfect
focus where the sharp focus is maintained. It depends on many factors, including but not limited
to the quality of optical design, the size of objective lens aperture relative to the focal length of
the objective lens, and the magnification of the object, and it is reciprocal of the resolution.115
A good surgical microscope should have an adequate depth of focus without sacrificing too
much resolution to keep the scene sharp. Another important term is parfocal, which means
an optical system can stay in focus even with magnification changes.130 Due to the need of
switching magnification during surgery, a surgical microscope being parfocal saves surgeons
from repeated refocusing.
Microscopes need to be well focused before the operation, and when the position of the
microscope is adjusted during surgery, refocus is needed. A fast focusing capability can save
setup time for surgery. Various methods have been proposed for the automatic focusing of the
surgical microscope. Nohda131 proposed an automatic focusing device for a stereoscopic micro-
scope, which detects the position of the reflected image of infrared LED (IR-LED) using a focus-
ing screen. The positions of the IR-LED and the focusing screen are conjugate with the in-focus
position of the sample; hence, the reflected image of the infrared diode is at the center of the
focusing screen when the sample is in focus. Jorgens and Faltermeier132 proposed using the
interaction of an active light-projecting system and a passive video system to focus on both
covered and uncovered objects illuminated by the transmitted and reflected lights. Vry et al.133
proposed a high-precision optical arrangement for stereomicroscope autofocusing, where a cyl-
inder optic is employed to project a bar-shaped mark onto the object. Many current microscopes
are equipped with fast autofocusing optics, which uses two laser beams acting as a focusing
reference to find a focus point rapidly. The focus point works for not only the main viewing
position but also the assistant position and camera. Furthermore, methods have been developed
to maintain the surgical microscope in focus at different viewing points. For example, Heller134
proposed a mechanical control unit for a surgical microscope support stand, and the unit
constrains the microscope to move along a spherical surface so the focusing status can be
maintained.

3.2 Illumination System

Illumination is another key factor besides the optical system for the imaging quality of a micro-
scope. Successful surgical illumination has four key factors, namely the luminance, shadow
management, volume of light, and heat. A bright view of the whole surgical site throughout
the surgery is always desired. The original illuminator in the earliest surgical microscopes was
an independent bulb externally mounted on the side of the microscope. Light transmitted to
the surgical site likely creates shadows, and thus illumination of deep cavities was hardly
possible.20,115 Modern microscopes have adopted high-power light sources with stable light
intensity and close-to-sunlight color temperature.73,122,135 With the built-in coaxial illuminator,
light is rerouted to the viewing axis and projected down through the objective lens.115 It is ben-
eficial to remove shadows in cavities and complex structures and especially cause a red glow of
the retina that assists cataract surgery. Light management methods have also been developed to
guarantee a stable and relatively safe illumination regardless of any change of magnification or
working distance. In addition, some modern surgical microscopes offer an option to set up

Journal of Biomedical Optics 010901-10 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

various lighting profiles for different tissues, so by controlling the light sensitivity of the inte-
grated recording camera, a change of color temperature may appear as images of the surgical
field are displayed on the monitor.
Despite numerous advantages of surgical microscope illumination, it is still worth noting that
many up-to-date neuro and spine surgical microscopes use light sources of the highest intensity
to provide the best brightness and clearness for human eyes regardless of magnification and
working distance. However, the high power can damage the underlying tissue. Though manu-
facturers of surgical microscopes provide safety warnings of possible damage, specific settings
of the illumination are not regulated.136 Nevertheless, the International Organization for
Standardization (ISO) 10936-2 standard and the American National Standards Institute
(ANSI) Z80.38 standard have set requirements for the maximum retinal exposure limit and the
stability of light intensity of ophthalmic surgical microscopes.137,138 Besides, the International
Electrotechnical Commission (IEC) has set general requirements for the characteristics of sur-
gical lighting, including a central illuminance of 40,000 to 160,000 lux, a color rendering index
between 85 and 100 Ra, and a color temperature of 3000 to 6700 K. This standard does not apply
to the lights for surgical microscopes since they are excluded as “special purpose lights with
different conditions of use,” but these requirements may offer a general idea for the requirement
of surgical illumination.

3.2.1 Light source

Except for the traditional incandescent bulbs used in old surgical microscopes, there are mainly
three types of light sources, i.e., xenon lamp, halogen lamp, and the LED. LED can provide
illumination in the visible wavelength range with good brightness, good stability, longer life,
less power consumption, and extremely low heat; therefore, it is preferred in many ophthalmic
and ENT microscopes.139 However, LED as a surgical light source also has disadvantages: the
higher color temperature and narrower wavelength range make the light not as close to sunlight;
its spectrum is insufficient for fluorescence-guided applications especially ICG imaging, where
an excitation light in the NIR range is needed; moreover, it is not easy to replace.
Xenon lamp and halogen lamp are two options to address these needs. Xenon lamp emits
light with a broad spectrum from ultraviolet (UV) (185 nm) to infrared (2000 nm). The spectrum
is relatively smooth in the visible range, but it has some spikes in the near-infrared (NIR) range.
Xenon light has a color temperature of 4000 to 6000 K, which is similar to sunlight. Therefore,
the bright-white light is able to offer a naturally colored view of the anatomy. Halogen lamp also
covers a wide and continuous spectrum including visible and NIR light, but it has a slightly lower
color temperature (3200 to 5000 K), which means the light does not look as “white” as xenon
light. Both xenon and halogen lamps can provide a stable illumination with DC regulation power
employed. Nevertheless, the surgical microscopes do not use all the wavelength range of xenon
lamp or halogen lamp. Actually, UV light and infrared light above ∼1100 nm are filtered out for
surgical microscopes to avoid various possible damages to the patient’s skin or eye caused by
exposure in this wavelength range.140,141 Xenon and halogen light sources are commonly used in
neurosurgical microscopes because of the need for intraoperative fluorescence imaging. They are
also utilized in some ophthalmic and plastic microscopes. For example, some ophthalmic micro-
scopes may employ a dual-illumination system combining LED and halogen for Red Reflex and
normal illumination. Both halogen lamps and xenon lamps emit much heat. Therefore, in a sur-
gical microscope, the light source is installed away from optics, and a fiber guide is used to
transmit light from the light source to optics without carrying the heat.

3.2.2 Illumination arrangement

The tissue surface being viewed under a surgical microscope during operation is usually wet and
highly reflective. The light that comes from an angle can be easily reflected away and cause a
dark view, as Fig. 6(a) shows. Coaxial illumination is the solution to this situation. Different from
lateral illumination where light comes from the side, coaxial illumination matches the optical
axes of illumination and visualization (lens).142,143 Illumination from the light source that locates
on the side is diverted and projected almost parallel to the axis of the lens, as shown in Fig. 6(b).

Journal of Biomedical Optics 010901-11 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

Fig. 6 Illustration of coaxial illumination and comparison with side illumination: (a) side illumination
and (b) coaxial illumination.

Fig. 7 Comparison of illumination effect with and without SAI.124

Therefore, light vertically illuminates the tissue surface and is reflected directly to the lens,
not having much loss. Coaxial illumination reduces the diameter of the illuminated area,144
moreover, it can be directed into narrow and deep cavities, which is helpful for neurosurgery,
ENT surgery, and endodontics.72,144
The light path for coaxial illumination in nonophthalmic surgical microscopes, such as
neuro or ENT microscopes, usually forms a small angle with the observation axis in the range
of 6°.144–146 In some contemporary surgical microscopes, it is called small angle illumination
(SAI),124 which provides a concentrated and evenly distributed light beam, a bright view, and
an improved depth perception, as Fig. 7 shows. With SAI, the shadow that appears at the edge of
the viewing field is significantly reduced. Illumination with an even smaller angle is important
when it comes to certain ophthalmic interventions especially cataract operations, where the ver-
tically impinging light gets diffusely reflected by the fundus and the pupil under operation shines
reddish, which is called red reflex.144,145,147 The production of red reflex requires a small angle
between the illumination beam path and observation beam path, in the range of 0 deg to
2 deg.144,146 Although the red reflex helps under certain circumstances, it does not help as much
in revealing good plasticity without the shadows on the structures in the interior eye.145
Therefore, both types of illumination, namely the 6 deg and the 0 deg illumination, are usually
equipped in ophthalmic surgical microscopes.

3.2.3 Light management

A desirable illumination for a surgical microscope should provide a stable brightness for the
viewing area regardless of the change of working distance or magnification. In fact, irradiance
(irradiation of a surface, W∕m2 ) of a microscope light source increases with decreasing spot size
and decreasing working distance.136 With an unchanged illumination setting, the increased work-
ing distance can cause insufficient irradiance, while the decreased working distance excessive
irradiance. Insufficient irradiance makes the view unclear, while excessive irradiance may cause
soft tissue burns.148 Similarly, decreased magnification, which enlarges the FOV, may lead to the

Journal of Biomedical Optics 010901-12 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

Fig. 8 Light management related to working distance and spot size: (a) automatic adaption of light
intensity with decreased working distance and (b) automatic adjustment of the illuminating area
with increased magnification.

dimness, while increased magnification may burn tissue outside of the FOV. To address this
issue, many contemporary surgical microscopes are equipped with smart light management,
which adjusts light intensity automatically with the change of working distance [Fig. 8(a)]
or magnification [Fig. 8(b)].

3.3 Mechanical System and Automation

The structure of the whole surgical microscope system can be delicate and complicated. It assem-
bles every part of the system and makes them work together harmoniously. A well-designed
system can assist surgeons with good stability, sterility, easy operation, as well as comfort.
Mechanical stability is the second most important criterion in selecting a surgical microscope.130
The drift or vibrating of a microscope after positioning distracts surgeons’ focus on the surgical
site. Therefore, superior suspension and balancing mechanisms are important. Microscope drap-
ing is a necessary requirement for sterilization in the OR. A good draping design saves the setup
time for the microscope and avoids the effect of glare.149 Various controlling methods have been
developed to enable hands-free operation for surgeons.134,150–152 Moreover, different parts of a
surgical microscope have been designed to improve its ergonomics and maneuverability.60,153
This section will discuss some important features that affect the operation of surgical micro-
scopes involving its mechanical design and electrical automation.

3.3.1 Balancing and positioning

As is known, surgical microscopes should be quick and effortless to move and remain stationary
once the position is established.130 Balancing of the forces and moments from all directions
should be achieved, otherwise, brakes or bracing devices are needed to hold the microscope
in its position.154 Many suspension structures and balancing apparatus have been developed for
a fast and reliable balancing of microscope.154–160 Modern surgical microscopes have made it an
easy and time-saving process to balance. All six axes can get fully balanced with two pushes of a
button, and intraoperative rebalance can be quickly and accurately accomplished with a single
push of button on handgrip.
In recent years, a robotic autopositioning feature has been added to state-of-art surgical
microscopes.161–163 The robotic ability enables the microscope to orient the angle or change its
focal length so surgeons can target a specific point of interest, which is probably identified in a
preoperative imaging study. Oppenlander et al.162 developed the automatic positioning move-
ment control with three options. The first option is “auto lock current point,” which makes sur-
geons lock onto a target by changing the angle and focal length of the microscope to keep it in
focus at one point while being manually moved. The second option is “align parallel to plan,”

Journal of Biomedical Optics 010901-13 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

which positions the microscope to a preset angle and focus without any need to adjust the micro-
scope. The last one is “point to plan target,” which automatically adjusts the focus on a pre-
defined target. In a newly developed robotic visualization system, two robotic positioning
features, namely “point lock” and “position memory,” have brought many advantages in time,
functionality, and ergonomics.60 With “point lock,” the microscope head stays in focus when
being manually or automatically moved during surgery, so the surgeon can visualize different
angles of the same structure. “Position memory” makes the system able to “bookmark” positions
and transit quickly and smoothly back to these positions with no need to rediscover. In some
circumstances where the scope needs to be moved around to observe different structures or be
temporarily removed to get an x-ray, “position memory” can save plenty of time getting the
scope back to the same position. Previously, it was reported that around 40% of surgical duration
was spent on adjusting the microscope. But with all these robotic positioning features, the sur-
gical duration can be greatly reduced.60

3.3.2 Draping
Microscope drape is a very thin, transparent, and heat-resistant plastic film that houses the whole
surgical microscope, and it includes a transparent optical lens enclosing objective lens and ocu-
lar-housing extensions.164–166 The drape is seamed for sterile packaging to assure the microscope
sterility during surgery.167 To save the setup time of a surgical microscope, instant readiness is
required. Meanwhile, the drape must have adequate ocular pockets, not reduce the working dis-
tance, not interfere with surgeons’ operation or obstruct the view. Bala168 invented a microscope
drape assembly, which has four ocular pockets for different needs and does not affect the work-
ing distance or visualization by locating the objective lens window support within the objective
lens barrel.
Glare is one problem that comes with draping and illumination. As light passes through the
objective lens and illuminates the surgical field, some of the light would be reflected by the lens
cover on the drape, which can cause chromatic and spherical aberrations.149 Removing the cover,
however, can cause the contamination of surgical instruments. A dome-shaped objective lens
cover169 can not only reduce the reflection but also compromises the magnifying performance.
Surgical microscope manufacturer has brought up a solution by replacing the sterile lens cover
with a slanted one, and another attempt solution is to include the slanted lens cover in the sterile
microscope drape. Both methods can eliminate glare, with the price of increased costs or system
complexity. Langley149 has invented a glare elimination device for surgical microscopes. The
device includes three parts: the first part is to be attached to the surgical microscope, the second
part is for the sterile drape, and the third part is a body to connect the other two and provide an
angular offset. The device can be semipermanently attached to a surgical microscope with more
convenience. Weaver et al.170 proposed an apparatus that provides a secondary holder for a cover
to be applied to the objective lens barrel. The additional cover can be rocked and rotated easily to
a position where the view is not affected by glare.

3.3.3 Control
Surgical microscopes can be controlled in various ways to facilitate easy use of the microscope
and to free surgeons’ hands during surgery. Contemporary surgical microscopes are often
equipped with footswitch devices171 for generating control commands, touch-screen58,172,173 for
operation mode selection or switching images intraoperatively, or joystick control174 for highly
precise micropositioning. Mouth switch175–177 is a commonly employed controlling method.
Surgeons can use the mouth switch to change signals simply by holding the levers with a mouth,
in which way operation errors can be reduced, even with a large number of functions to
control.175 Eye controlling is another trend for surgical microscopes. Charlier et al.178 proposed
an eye-controlled surgical microscope, which used an IR-LED to illuminate the surgeon’s eye
and a charge-coupled device (CCD) sensor to detect the reflected infrared light from the sur-
geon’s eye for movement tracking. Similarly, Roduit et al.152 proposed an eye-guided controlling
technology, which used a CCD camera mounted on the right ocular of a microscope and con-
tinuously monitored the surgeon’s eye. With the eye-guided control function, surgeons can use

Journal of Biomedical Optics 010901-14 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

their eyes to perform multiple tasks including access to built-in data display, laser aiming, and
control of autofocusing. Voice control is a sterile remote control that facilitates operator in either
sterile or nonsterile region and does not require the operator to take action.151 Furthermore,
Pitskhelauri et al.150 developed a device named Mari, which allows hands-free utilization of
surgical microscopes. The device was attached to the eyepieces of a surgical microscope, and
operators can use the joystick and electric switch to do multifunction control of the microscope.

3.3.4 Ergonomics and maneuverability

Except premium optics, good illumination, and various image-guided surgical functions, one
nonnegligible benefit of surgical microscope compared with traditional loupes is the ergonom-
ics,25,73,130 which guarantees a comfortable and flexible working position for surgeons and
reduces the risk of back and neck musculoskeletal injuries.25 Meanwhile, maneuverability is
valued for the simplification of microscope operations.130 Therefore, the microscopes of the time
are equipped with a full range of movement and tilt of the optics carrier, as well as a selection
of binoculars with full 360-deg rotation for different heights and positioning needs. Some
microscopes have large HD monitors so that surgeons can all work with an upright position.
Eye-to-object distance115 indicates the distance from the observer’s eye to the focus point of
the microscope. Surgeons are likely to be more comfortable with a longer eye-to-object distance.
In addition, new designs of surgical microscopes are trying to provide longer working distances
up to 600 mm179 to offer better ergonomics, easy maneuvering, and more working space that
allows long instruments. For example, Horizontal Optics Technology, which is employed in the
state-of-art surgical microscope, enables a compact optics carrier and further ergonomics.180
Figure 9 shows how a surgical microscope can improve ergonomics for an endodontic surgeon
as an example.

3.4 Visualization System

Clear and bright visualization of the surgical site is the ultimate goal of using a surgical micro-
scope. Except the good image quality provided by high-precision optics and sufficient illumi-
nation, the stereoscopic view that offers depth information is another non-negligible benefit of
the binocular surgical microscope. Despite that stereopsis is the result of optical design, it
influences how surgeons obtain information from and feel about what they see.
Users of surgical microscopes can observe the surgical site in various ways. A microscope
head usually has one main observation port and one rear or lateral port for co-observers, who can
be assistants, students, or trainees. Cameras182 or other imaging systems183 can also be adapted to
these optical ports for video recording or photography of the ongoing surgery. All optical ports
offer an identical FOV, which beats surgical loupes and enables “cosurgery.”25,184 With the image

Fig. 9 Illustration of improved ergonomics with surgical microscope.181

Journal of Biomedical Optics 010901-15 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

injection technique, not only the white-light image of patient tissue but also pre- and intraoper-
ative images can be overlaid accurately with the white-light image for navigation.57,58 HD
display105,185 and 3D display22 have been employed in the surgical microscope for sharing
of the view with high resolution and enlarged stereoscopic images. Other visual methods, such
as using smartphones for recording and virtual reality (VR) headsets for visualization, have also
been developed.186 With the advanced technologies applied, surgical microscopes can help
surgeons see much easier than ever before.

3.4.1 Stereopsis
Stereopsis is a key feature of binocular surgical microscopes. While the monocular depth
cues lie in perspective projection, occlusion, size, shading, and motion parallax, the stereoscopic
depth is based on the slight disparities between two images presented to two eyes.187 Stereo
microscopes use two afocal relay zoom lens systems for the two channels of a binocular tube,
and their axes are parallel to and offset from the axis of the objective lens.188 The light coming
out of the objective lens is divided into two parts and forms two slightly different images into two
channels. In surgery, especially when working with magnification, perspective, and size cues
may be lost; therefore, the stereopsis brought by binocular is essential to provide a 3D impression
of the surgical field. The depth information can aid the detection of diagnostically relevant
shapes, orientations, and positions of anatomical features, especially when monocular cues are
absent or unreliable.187 For example, it is vital for dentists to construct 3D structures in patients’
mouth74 and for neurosurgeons to understand complex volumetric relationships of neuroana-
tomical structures.60
An optical design that enhances stereo visualization for surgical microscopes is
FushionOptics technology,189 which sets two separate beam paths in the optical head, providing
the DOF and high resolution, respectively. The two paths are then merged in the observer’s brain
into a single, optical spatial image. Because of this combination of depth and resolution, the
interruptions for refocusing can be avoided.

3.4.2 Share of view

There is usually more than one observer during surgery, which makes the “share of view” an
important and necessary feature for surgical microscopes. In some procedures, meaningful as-
sistance has to be given by a cosurgeon sharing the same view with surgical microscope.25,117,190
It aids not only assistance but also teaching and participation of trainees.25 The simplest way to
share the identical surgical view is to use an optical splitter to split the light into two eyepieces.142
Nowadays, surgical microscopes can have multiple optical ports for the main observer, assistant
observer, and external cameras. Some models have integrated HD cameras and monitors so the
whole team can share the view on the screen.19,25,117,191

3.4.3 High-definition display and 3D visualization

Many new surgical microscope models, especially neurosurgical microscopes and ophthalmic
microscopes, are equipped with HD video cameras and large HD monitors, so subtle details can
be viewed more clearly and shared by the whole team.56,58 In addition, 3D screens, which employ
passive linear polarization technology, have been brought to the operating room to deliver depth
perception.22,192–194 Observers need to wear goggles to have a real-time 3D view, which gives a
realistic appraisal of certain features. It was reported that screens possibly offer better contrast of
the visual field than eyepieces and image injection in some cases.58 Moreover, utilizing screens
enables the heads-up display, which is beneficial for surgeons’ spinal health during long
procedures.
A screen can show not only the white-light image of the surgical site but also other images,
such as intraoperative OCT images, for surgical guidance. The images can be shown separately,
overlaid on the white-light image,58 or even in picture-in-picture endoscopic assistance view60
for endoscopic microinspection tools, as shown in Fig. 10.

Journal of Biomedical Optics 010901-16 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

Fig. 10 Screens for visualization during surgery: (a) intraoperative OCT images shown separately
on 6.5-in screen, with white-light image simultaneously on 21.5-in screen, and injected in ocu-
lars,58 (b) surgeons using the 3D display in a seated position with goggles,22 and (c) picture-
in-picture 3D visualization of endoscopic assistance.60

3.5 Augmented Reality

During surgeries, especially neurosurgeries, image-guided surgical navigation systems are criti-
cal for surgical outcomes.195,196 Although surgeons can obtain the knowledge of anatomical
structures of patients via preoperative images such as CT images, they have to work with radi-
ologists to build up the anatomical structure model in mind preoperatively.197 Moreover, there is
difficulty for them to relate the preoperative x-ray information to the appearance of the surgical
view.63 Surgical navigation systems that only display 2D images on screens require that surgeons
perform the 2D-to-3D transformation themselves in mind, and surgeons need to switch views
constantly between screen and patient, which disturbs the surgical workflow.196,198,199
AR can be very helpful with preoperative planning and intraoperative surgical navigation.
It provides the visualization of anatomical structures beneath human skin intraoperatively by
overlaying segmented preoperative images to the corresponding area on the human body.
Attempts to apply AR in neurosurgery, general surgery, orthopedic surgery, maxillofacial
surgery, otolaryngology, and cardiovascular and thoracic surgery have been proved successful
and promising.200 The concept of AR is to overlay real-life structures with artificial
elements.196,197,201 Not only the 3D model but also the detailed anatomical structures can be
illustrated by the overlaid image. The images to be overlaid with the real-life environment can
be CT, MRI, and angiography,52,63,198,199,202–207 ultrasound,199,201 NIR fluorescence,54,208,209 or

Journal of Biomedical Optics 010901-17 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

OCT images,51 depending on the operation target. The representation of virtual images can be
surface mesh, transparency, texture map, or wireframe.203,206,210 AR differs from VR, with which
the user is surrounded by a virtual world (immersion) and interacts with the virtual world
(presence).211 In surgery, VR refers to a virtual patient on a physical model of the pathology,
surgical instruments, and connectors of all VR–reality interfaces.197
There are three core components of AR.212 The first one is a virtual image or environment,
which refers to the computer-generated 3D reconstruction of a subsurface target with color or
texture-coded differentiation between anatomical structures. The other two core components are
the registration of the virtual environment with real space, and the display technology to combine
the virtual and real environment, respectively. In clinical use, the overlaid images can be
displayed on many surfaces: monitors,213–216 optics (i.e., microscope),50–52,57,63,199,205,217,218
head-mounted devices (i.e., smart glasses),219–222 semitransparent surfaces,223–225 and the
patient.207,226–228 Using AR with a surgical microscope facilitates navigation with multiple
magnifications and would not require additional AR system cost since surgical microscopes are
available in most modern operating rooms.210 In this review, we focus on the allocation of
AR with surgical microscopes.

3.5.1 Augmented surgical microscope

The microscope-based AR systems have been found particularly useful in neurosurgery, which is
the earliest adopter of AR.210,212 The majority of applications for recent neurosurgical AR is
tumor resection, followed by neurovascular surgery and spinal procedures.7 For tumor resection,
AR with the segmented CT or MRI image helps with the margin definition during surgery, AR
overlay of volumetric CT/MRI data with no additional surgical time or complications reduces
both intensive care unit and hospital length of stay.229 AR for vascular neurosurgery has focused
on the augmentation of stereomicroscopes,212 with either fluorescence from intraoperative
ICG angiography or segmented preoperative CTA/MRA/digital subtraction angiography
(DSA).57,209,230,231 The overlay of the target vasculature optimizes craniotomy placement, dural
opening, and skin incision.57,212,230,231 Moreover, it is significant that the microscope-based AR
system does not require the bayonet pointer typical for common neuronavigational systems.210
The first augmented monoscopic surgical microscope,106 which was used for cranial surgery,
was proposed in 1985. The segmented 2D preoperative CT images were displayed in monocular
and were registered to the operating table using an acoustic localizer system, but this system was
unable to track tools in real time. The first augmented stereoscopic operating microscope was
proposed in 1995.63 It achieved multicolor displaying of segmented 3D cross-sectional MRI/CT
data into both microscope oculars via a beam-splitter, either as solid or wire-mesh overlays.
Fiducials on the skin surface were used for patient registration. The overlay accuracy was 2
to 3 mm, and the interactive update rate was 2 Hz. The system used an LED-based 3D localizer
for calibration, microscope pose tracking, and patient tracking. Therefore, the microscope was
capable of free movement while maintaining the overlay accuracy. Later in 2000,232 several
improvements were achieved to the system, including automated calibration, the bone-implanted
fiducial added for registration, and the locking acrylic dental stent for patient tracking. The clini-
cal overlay errors were 0.5 to 1.0 mm on bone fiducials and 0.5 to 4.0 mm on target structures. In
1996,233 the S.M.N. system developed by Carl Zeiss (Germany) as a neuronavigational system
was first installed at Rennes Pontchaillou Hospital. The system was integrated with an OPMI ES
surgical microscope, and it utilized a 3D optical localizer for tracking, which was comprised of
infrared emitters and three linear 1D cameras. Fiducial markers such as bone or skin markers
were used for registration. The virtual images were either MRI/CT slices or 3D rendered images,
and they were both injected into the microscope ocular and displayed on the monitor. Inside of
the microscope ocular, surgeons had access to menus, so they could interact with the user
interface.

3.5.2 Technical implement of augmented surgical microscopes

The procedures of an augmented surgical microscope include calibration of the optical system,
tracking, registration, and display.232 The AR accuracy mostly depends on the accuracy of the

Journal of Biomedical Optics 010901-18 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

tracking technology, the registration procedure, the camera calibration, and the image scanning
device (e.g., CT or MRI scanner).198 Registration is the process of relating two or more data sets
to each other to match their content.234 It is essential for augmented surgical microscopes because
the system is very sensitive to misalignments of the virtual image and the real environment (the
patient tissue). Calibration of binocular optics and registration limit application accuracy
mostly.232 The term tracking for AR refers to the pose estimation of objects in real time.234
The display of an augmented surgical microscope mostly refers to the image injection into
microscope oculars but also the monitors that are equipped in some new-generation surgical
microscopes.
Early image overlay in a microscope-based neuronavigational system injects 2D contours
into one eyepiece.202 Therefore, surgeons needed to either scroll through different image planes
and merge them in mind or look away from the microscope for a 3D impression.205 The con-
ceptual description of the AR idea of 3D stereoscopic overlay of the operating field in a surgical
microscope appeared in 2003 proposed by Aschke et al.205 The process is divided into two
phases, namely the preoperative phase and the intraoperative phase. In the preoperative phase,
calibrations under different magnifications are accomplished to obtain lens error values. Patient
image data, e.g., from MRI, fMRI, or ultrasound image are segmented manually and a reference
model is generated. Then, the image data and the reference model are matched to patient’s
anatomy by the registration process. In the intraoperative phase, new intraoperative image data
such as intraoperative MRI are registered to the reference model, and the coordinates of intra-
operative image data are utilized to segment edges.
The aim of calibration is to produce a projection matrix that will give the pixel position in the
injected image of any 3D point relative to the frame of reference of the microscope.63 The proc-
ess determines all camera parameters including the correction of optical errors generated by
nonperfect lenses.205 Usually, several calibrations need to be done under different magnifications
because the lens error values change with zooms,199,205,235 and the preoperative calibration
process takes about 10 to 20 min.57 Edwards et al.63 used a calibration object that consisted
of a number of localizer LEDs within the working region to enable the calculation of room
coordinates. Mun et al.221 proposed a calibration algorithm that uses image intensity rather than
fiducials and is adaptable for different lens distortion models. In some recent papers, the checker-
board patterns were used to calibrate the stereo camera integrated into the microscope head,
as Fig. 11 shows.218,236
There are several tracking methods based on different devices. Friets et al.202 used an ultra-
sound range finding system, while Doyle237 used a magnetic field digitizer instead. However,
optical tracking is the most popular way63,199,206,210,232,233,235,238 and has been in use in modern
operating rooms for intraoperative navigation. It is facilitated by the widely available cameras,
and it does not require any wire connection between the system and the tracked object. LEDs,
IR-LEDs, or other optical trackers can be attached to both the microscope and the patient, as
shown in Fig. 12(a), and a 3D optical localizer is utilized to track these light points.63,206,235
An acrylic dental stent with imaging and physical locators was developed and attached firmly

Fig. 11 Illustration of the calibration method using checkerboard pattern: (a) system setup and
(b) various transforms involved with the calibration method.236

Journal of Biomedical Optics 010901-19 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Ma and Fei: Comprehensive review of surgical microscopes: technology development. . .

Fig. 12 Tracking methods: (a) surgical microscope with optical tracker (yellow arrow)235 and
(b) surgical microscope with integrated tracking camera.199

to the patient’s upper teeth so the tracker was brought close to the volume of interest, and
it allowed free movement of the patient’s head within the line-of-sight.232 Garcia Giraldez
et al.199 integrated a 3D tracking camera onto the housing of a surgical microscope tube, and
the camera tracked the movements of surgical tools and the patient, as shown in Fig. 12(b). Gard
et al.218 used an image-based method to track a green-colored instrument tip under a surgical
microscope for trajectory creation in tympanoplasty.
Registration is most commonly accomplished with fiducial markers,57,207,235 skin surface,63
and manual procedures238 as well as skull-implanted or dental-fixed fiducials.239,240 It is reported
that fiducial markers or skin surface registration are the easiest, fastest, and most accurate ways
with respect to manual registration,210 and they are less invasive and laborious than skull-
implanted or dental-fixed fiducials.210 Overall, the registration errors for clinical AR range from
0.3 to 4.2 mm, mostly 2 to 3 mm.199,206,207,212,218,232,235,241
When combined with surgical microscopes, the display method of AR is usually image
injection57,208 into either one microscope ocular or both. Displaying the bright-field image and
the augmented image in two separate oculars may cause strain and fatigue since there is no cue
for spatial coregistration of two types of images. Two embodiments of an augmented stereo-
scopic microscope using ICG fluorescence images were proposed by Romanowski et al.242
In the single-channel embodiment, the objective lens receives both NIR and visible bright-field
images of the examined object simultaneously. The augmentation module (beam splitter between
the objective lens and ocular lenses) separates the NIR image from the visible image, and then
the NIR image is processed to generate a synthetic image. The synthetic image is directed into an
eyepiece, while the visible image is shown in two eyepieces with different light intensity pro-
portions. In the dual-channel embodiment, two cameras capture the NIR image and the visible
bright-field image, respectively. After image processing, the ICG fluorescence image is overlaid
onto the optical image of the same FOV, and the combined image can be directed into both
eyepieces. Figure 13 shows the schematic of a single-channel design of an augmented surgical
microscope for NIR fluorescence imaging. Furthermore, the three ways for 3D data
representation234 are slice rendering (one slice of the volume data), surface rendering that shows
the transition between structures, or volume rendering. Figure 14 shows the overlay of injected
2D and 3D models during bypass surgery.

3.5.3 Challenges
Although AR has been a fast-developed technology, challenges still exist for it in clinical use,
such as the registration error from compounded sources, the visual and tactile temporal asyn-
chrony caused by system latency, the inattentional obstructed view by virtual component, and the
time consumption for registration and verification.210 Injection of 3D images into microscope
oculars alleviates the visual fatigue caused by the accommodation-divergence discrepancy, but
the focal plane of the virtual image remains incongruent with that of the target.212 The impaired

Journal of Biomedical Optics 010901-20 January 2021 • Vol. 26(1)

Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Biomedical-Optics on 04 Jan 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use