MicronTracker Developer Manual
MicronTracker Developer Manual
Developer’s Manual
MTC 3.8
October 3 2016
WARNING: This document and the associated demonstration application source code, in
electronic or printed form, are proprietary and confidential to ClaroNav Inc. No part of this
document, the detailed programming interface documentation and the associated example
source code, shall be copied, published or used for a purpose other than product usage
without the prior written consent of ClaroNav Inc.
Table of Contents
1 Warnings and precautions ............................................................................... 5
1.1 Warnings and precautions ..........................................................................................5
1.2 Safety warnings..........................................................................................................5
1.3 Care ............................................................................................................................6
1.4 Disposal .....................................................................................................................6
1.5 Symbols on the MicronTracker..................................................................................7
2 Product Description and Intended Use........................................................... 8
2.1 Intended Use Statement .............................................................................................8
2.2 Purpose of this Manual ..............................................................................................8
2.3 Third-Generation Optical Tracking ...........................................................................9
2.4 Principles of Operation ............................................................................................11
2.5 Terminology and Conventions .................................................................................12
2.6 Contact Us................................................................................................................14
3 Windows Installation...................................................................................... 15
3.1 MicronTracker parts list ...........................................................................................15
3.2 PC Host Configuration .............................................................................................15
3.3 Enabling the IEEE 1394 Link ..................................................................................16
3.4 Removing an older installation ................................................................................18
3.5 Internet downloadable upgrades ..............................................................................18
3.6 Installing the MicronTracker Camera Device Drivers and USB .............................18
3.7 Installation Verification ...........................................................................................26
4 Linux Installation ........................................................................................... 27
4.1 PC Host Configuration .............................................................................................27
4.2 Linux Installation .....................................................................................................27
5 Mac OSX Installation ..................................................................................... 29
5.1 Release Note ............................................................................................................29
5.2 PC Host Configuration .............................................................................................29
5.3 Mac OSX Installation ..............................................................................................29
6 Product Overview ........................................................................................... 31
6.1 Camera Models and Specifications ..........................................................................31
6.2 Field of Measurement (FOM) ..................................................................................33
6.3 Extended FOM and small marker detection ............................................................34
6.4 Multi-Camera ...........................................................................................................34
1.3 Care
Handling
MicronTracker is a precise optical instrument and should be treated as such. When
handling the camera unit, avoid touching the lenses. Avoid excessive shaking,
dropping or mishandling of the device. The case is designed for maximum
protection, but when not in use keep the MicronTracker in its foam-padded box
and avoid shocks.
Cleaning
Keep the lenses free of dust or stains using a photographic lens cleaning solution
and a lint free paper or cloth. Do not apply excessive force.
1.4 Disposal
It is the full responsibility of the application product developer to correctly use the
measurement data provided a MicronTracker product, and for validating the
suitability and safety of the system incorporating a MicronTracker in the overall
system’s intended use.
1
ClaroNav Inc. holds patents on key enabling technological innovations related to 3 rd
generation optical tracking.
Third generation trackers are fully passive. They use available visible light
illumination to observe targets stereoscopically much like humans do. Tracked
IR camera +
IR light
2nd Generation: active IR camera
Retro-reflective
balls
Custom electronics
Visible pattern
objects are marked with a visible painted or printed target pattern. The camera is
interfaced to the computer using a standard digital link (e.g., FireWire or USB
2.0). Software running on the PC processes the camera images to detect and
pinpoint visible target patterns in the images. The MicronTracker’s target pattern
is a set of high-contrast regions called a Marker. The regions forming the marker
are geometrically defined and arranged according to specific rules dictated by the
tracker’s designers.
Third generation trackers offer many advantages over trackers of earlier
generations, amongst them:
Simpler, more reliable, lower-cost, design, which uses off-the-shelf
electronics;
Smaller and lighter camera that can be placed closer to the FOM, easier to
ship, move, store and manipulate;
and the other is the vector's head. Although not an absolute restriction, the
Xnormals of the XPs at the ends of a vector would usually be identical (or at least
very similar). Vectors in the FOM are detected and tracked, even when they are
not identified as being part of a facet.
Facet: Two vectors in a rigid configuration that
complies with MicronTracker’s facet restrictions (one Marker
vector is longer than the other, and the angle between
the vectors is in the 8-172 degrees range, see 10.5). The 1..N
vectors would usually be on the same plane, or at least F
have parallel Xnormals, but this is not a requirement. A F
Facet
a
facet, containing 3 or 4 Xpoints, is the smallest set of a
shorter cc
longer
Xpoints that can be tracked in 6D (3 spatial coordinates e
and 3 axes of rotation). e
t
Vector tV
Marker: One or more facets in a rigid configuration, e
usually attached to an object to be tracked. In a multi- base headc
facet marker, only one of its facets needs to be t
Xpoint X
o
observed in a frame to allow MicronTracker’s software
P
r
to detect and report on the marker’s pose.
o
Marker Template: A named description of a marker’s geometry, enabling i
its detection and pose measurement. Marker templates are stored on disk as n text
t
z Xnormal
z y
x
files.
camera’s Z-axis (depth direction) when the facet is facing the camera. The Y-axis
is defined according to the right-handed coordinate system’s rules (see above
figure for examples).
Marker coordinates: Identical to the coordinates of the first facet described
in the marker’s template (even when that facet is not visible).
Camera coordinates: Defined during calibration. The origin is
approximately at the center of the camera. The Z-axis extends to the front of the
camera (depth), X towards the right of the camera and Y down. When viewing the
camera’s images, the camera and the screen coordinate systems are aligned (X
right, Y down and Z in).
z
y
NOTE: The camera coordinate axes orientations are not accurately aligned with
the orientation of the enclosure’s external faces, and the FOM is not accurately
symmetric around each axis.
2.6 Contact Us
It is quite possible that the environment in which you will be using your
MicronTracker is different from any of the environments in which it was tested in
the past, and, thus, you may encounter some difficulties or have new requirements
that have not been fully addressed yet. We, at ClaroNav Inc., are keenly interested
in your experience evaluating the device and embedding it in your application. We
commit to being highly responsive to any feedback we receive.
Our contact coordinates:
ClaroNav Inc.,
1140 Sheppard Avenue West – Unit 10
Toronto, Ontario, Canada, M3K 2A2
Tel: +1.647.951.1525, Fax +1.647.951.1524
www.claronav.com
[email protected]
3 Windows Installation
3.1 MicronTracker parts list
The MicronTracker package contains:
One MicronTracker camera
One IEEE 1394/FireWire PCI Host Card (Supports serial bus data transfer
rate of 100/200/400Mbps and 800Mbps for H3-60)
One 4.5m 6 pins to 6 pins IEEE 1394 (FireWire) cable only for Sx60, Hx40
and Hx60 models
One 4.5m 9 pins to 9 pins FireWire 800 IEEE 1394b cable only for H3-60
model.
One General Purpose Input Output (GPIO) connector (Only Hx40, Hx60,
Sx60 and H3-60 models)
The MicronTracker USB flash drive
A board demonstrating two markers, one of them supporting tooltip
calibration
A polished aluminum bar (AccBar) to be used with R-Fine software for
verification and refinement of the calibration
A special CoolCard marker, made of the Duramark material, for adjusting the
light coolness camera setting
Printed sheets showing examples of marker patterns
Mini tripod
If your package does not contain the any of the above items, please contact
ClaroNav Inc.support at: [email protected]
NOTE: Ensure that you use a power adapter with enough power to enable the
MicronTracker to function properly (12VDC, 2.5W per camera).
Figure 1: The Device Manager dialog, showing that the computer detected the 1394 bus
controller and the PCI card is correctly installed.
To install the IEEE 1394 OHCI PCI host adapter 3-port 400 MB/sec or 2-port
1394b 800Mbps you will need complete the following steps:
1. Make sure that the computer is turned off.
2. Unplug the power cord from the computer.
3. Remove the side panel from the computer.
4. Place the card in an open PCI slot. Making sure that it is firmly seated. Note
that using 1394b card some of the pins will remain outside of the PCI slot (see
below).
5. It is also recommended to connect the power pin on the Firewire card to the
power supply on the mother board.
6. Replace the side panel on the computer.
7. Plug back the power cord and turn the computer on.
8. The screen will show a New Hardware Found dialog box. Then an Add New
Hardware Wizard box will appear. Follow the prompts given by the Wizard.
9. To ensure that the PCI card is properly installed, open the Windows Control
Panel, click on the System icon, then the Hardware tab and finally select
Device Manager. The presence of the IEEE 1394 Bus host controller entry as
shown in fig. 1 indicates that the PCI card has successfully been installed.
NOTE: Since the procedure involves installing device drivers, the user performing
the installation needs to have administrative privileges. Otherwise, the installation
may fail.
NOTE: Do not connect the camera to your computer until you have completed the
software installation steps.
Figure 2: The Micron Tracker installation interface. You will be prompted to install
FlyCapture before the Micron Tracker installation proceeds.
The installer provides 32bit or 64bit MicronTracker releases for 32bit or 64bit
operating systems. The appropriate FlyCapture and MicronTracker will be
installed on your system.
FlyCapture SDK is the low level video libraries and it also installs all the required
drivers.
Simply click on the appropriate menus on the main installer and then follow the
instructions of the individual installers.
NOTE: The installer will detect whether your Windows is 32 or 64 bit and install
the appropriate version. Those who wish to develop 32 bit applications on their 64
bit machines will still be able to do so, as the appropriate files are still installed.
After the Welcome screen a dialog to select where the application folder should be
located is presented to the user.
After that, the installer will start transferring files. When terminated, the installer
will display the Wizard Complete dialog (fig. 6).
Figure 12: Found New Hardware Dialog box, click Next to continue
In Windows 7, you will then be presented with a Warning dialog informing you
that the camera driver is not Microsoft certified. Click “Continue Anyway” and
ignore the warning. (fig. 8).
Figure 13: Microsoft uncertified driver warning, just ignore it and click Continue Anyway
Complete the process by clicking Finish on the Completing Found New Hardware
Wizard dialog.
Figure 15: DriverControlGUI, Lists all 1394 devices and their drivers
4 Linux Installation
4.1 PC Host Configuration
To be able to install and properly run the MicronTracker software, the following
system configuration is required:
A Linux distribution with kernel 3.2 or higher including gcc. Previous
versions of the kernel had inconsistent FireWire support.
2 GHz or faster Pentium processor (3GHz recommended)
500 MB RAM
200MB of available hard disk space
An available IEEE 1394 (FireWire) socket or an available PCI slot for the
IEEE 1394 card provided with the MicronTracker.
Display of 1024x768 or higher resolution
4.2.1 Dependencies
The Linux release comes in the form of a gzip tar archive. After extracting the
sources and libraries to the filesystem you need to:
o Run "Ubuntu Software Center" and install the following development
libraries:
a) libvxl1-dev
b) libfltk1.3-dev
c) libxft-dev
o Set the MTHome environment variable to where you have extracted the
tarball. An example environment.sh file is included for your convenience.
o You will also find an empty CalibrationFiles directory at the higher level
where you extract the tar file. You need to copy the calibration file of your
camera from the USB into this directory. The directory where you’ll find it on
the USB is:
CalibrationFiles
And the file will be named:
BumbleBee_xxxxxx.calib
FlyCapture_xxxxxx.calib
Where xxxxxx will be the serial number of your MicronTracker.
o The release includes a precompiled demo in the "bin" directory. select the
corresponding x64 or x32 bin directory. Make sure that execute permissions are set.
Issue these commands
chmod +x mtsimpledemoc
chmod +x mtdemocpp
o Now the binary files should be executable. issue the following command to run the
simpledemoc app:
./mtsimpledemoc
o The Source code of the MTDemoCPP and MTSimpleDemoC are available in the
NetbeansProjects folder. You would need to install JDK7u10 and Netbeans 7.2.1 for
CPP development in order to open these projects, compile and build them.
Additional useful third party libraries are included in the tarballs folder. They are
not required for using MicronTracker, but you may find them useful when
experimenting with other uses for the stereo video input, such as augmented
reality.
NOTE:
6 Product Overview
6.1 Camera Models and Specifications
MicronTracker cameras are customized versions of PointGrey’s class-leading
Bumblebee stereo cameras. Four models are currently offered, XB3-BW-H3-60,
BB2-BW-Sx60, BB2-BW-Hx40 and BB2-BW-Hx60 (Hx40, Sx60 and Hx60 are
drop-in replacements for the original H40, S60 and H60 models and feature higher
frame rate and GPIO connector for external trigger and strobe functionality). The
following table describes and compares their key characteristics.
Specification \
H3-602 Sx60 Hx40 Hx60
Model
FOM, spherical
section 240x200x160 200x130x100
115x70x55 cm 120x120x90 cm
(radius x width x cm cm
height)
Measurement rate 16 Hz 48 Hz 20Hz 20Hz
Calibration
0.20 mm RMS 0.25 mm RMS 0.20 mm RMS 0.35 mm RMS
accuracy1
Jitter, static target2 0.007mm RMS 0.007mm RMS 0.015mm RMS 0.015mm RMS
Jitter, moving
0.07 mm RMS 0.07 mm RMS 0.14 mm RMS 0.14 mm RMS
target2
Processing
time/frame
15 ms 7 ms 10 ms 10 ms
(2.5GHz CPU
Q8300, 4 markers)
Lag ~60 ms ~20 ms ~50 ms ~50 ms
Sensors resolution 1280x960 640x480 1024x768 1024x768
Lenses / H°xV° 6 mm 50°x38° 6 mm 50°x38° 4 mm 70°x52° 6 mm 50°x38°
Case dimensions 283x43x49 164x43x54 164x43x54 164x43x54
(WxHxD, pprox..) mm mm mm mm
Weight ( pprox..) 505gr 342gr 342gr 342gr
IEEE-1394b
Electrical Interface IEEE-1394a (FireWire), 400 Mbps
(FireWire), 800 Mbps
Voltage
8-30V via IEEE-1394 interface or GPIO connector
requirement
Power consumption 4W at 12V 2.5W at 12V
Mount ¼” thread tripod mount
Operating
18° to 30°C (64° to 86°F)
temperature
Operating 700 - 1060 hPa (up to 2000 m)
pressure/altitude
Operating humidity 30% -80% non-condensing
Storage
-30° to 60°C (-22° to 140°F)
Temperature
Storage humidity 10% - 90% non-condensing
2 Last four characters of the camera’s full name are used as short form.
Certifications IEC 60601-1, FCC Part 15 Class A, RoHS and WEEE compliant
Drift: temperature,
pressure, light
intensity, marker
insignificant
orientation, motion
blur, exposure
settings
Hazard warnings Marker out of FOM; Thermal instability; Temperature out of range; Shadow over
issued target
Operating system MS-Windows7/Win 8/Win 10, Linux, Mac OSX – 32bit and 64bit
Minimal computer
2 GHz CPU, 1GB RAM, 1GB disk space
hardware
NOTES:
1) Single target position accuracy. For Hx40, ~30,000 averaged positions at distances
of 40-85cm, for Sx60, ~10,000 averaged positions at distances of 50-85cm, for
Hx60, ~30,000 averaged positions at distances of 60-155cm, for H3-60, ~50,000
averaged positions at distances of 60-170cm. Calibration accuracy decreases when
smaller footprint detection is enabled.
2) Single target at a distance of 75cm, 150 Lux, 20ms shutter.
3) The illumination variation range can be extended to 20:1 with the use of black/gray
markers (see 10.2).
The camera’s optics and electronics are housed in an internal aluminum case and
an external rubber shell. The external shell is designed to protect the internal case
against electric and physical shock. The camera has been certified for compliance
with medical safety standards and carefully calibrated to perform with the external
shell.
NOTE: Removing the protective shell would nullify the certification and degrade
the camera’s measurement accuracy.
6.4 Multi-Camera
MicronTracker’s software is designed to allow interfacing to a multi-camera set-
up with ease. Starting from MicronTracker version 3.7, each camera requires full
bandwidth to operate properly. Hx40, Hx60 and Sx60 require 400Mbps bandwidth
and H3-60 model requires 800Mbps bandwidth. Firewire 1394a cards provide
400Mbps bandwidth therefore each 1394a camera must be connected to a separate
firewire card. Firewire 1394b cards provide 800Mbps bandwidth; therefore it is
possible to connect two 400Mbps cameras to the same card. This design
guaranties each camera will work at full framerate.
Note that Cameras on separate buses do not automatically synchronize with each
other and a synchronization unit (sold separately) is required. When cameras
capture frames in an unsynchronized fashion, measurements of the same marker
taken by different cameras are not simultaneous. This is, however, not a
significant drawback for most applications. If synchronization is required, there
are both HW and software solutions available. Please contact ClaroNav Inc.for
further information.
Software
The following diagram shows the various software modules delivered as part of
the MicronTracker system and their dependency hierarchy. Modules below the
double line are supplied as executable libraries, and above it as example source
code.
o
MTC library: C API
r
FlyCapture/Digiclops libraries
Camera firmware
Bumblebee driver
gle-board computer
w
re
6.4.1 Firmware
The camera firmware runs on a dedicated processor inside the camera and
communicates with the computer over the IEEE-1394 bus. The firmware controls
the sensing and communications hardware and is critical to accurate performance
of the product. Firmware version names contain 4 numbers
(major.minor.type.revision). The camera is calibrated for a specific firmware
version and updates to the firmware may require the camera to be recalibrated.
Detecting and starting up the cameras connected to the computer, then loading
their calibration files;
Loading Marker templates from files;
Capturing and processing camera frames on demand to report on detected
markers and their poses;
Creating or modifying marker templates;
Customizing many aspects of the measurement operations to fit specific
application needs.
Applications written in any language may call MTC directly to take full advantage
of the MicronTracker’s capabilities. MTC completely encapsulates and abstracts
the lower-level camera control functions, making it unnecessary for the
application to make calls to any of the lower-level modules.
MTC version names contain 3 numbers: major.minor.revision. The major version
number, currently 2, remains fixed as long as backwards compatibility is
maintained, usually over a number of years. The minor version number
increments whenever new features are added, usually every few months. The
revision, or “patch”, number increments with versions containing new bug fixes,
but no API changes or feature additions.
The GPIO pins are TTL 3.3V pins. Inputs can be configured to accept external
trigger signals. When configured as inputs, the pins are internally pulled high
using weak pull-up resistors to allow easy triggering of the camera by simply
shorting the pin to ground (GND). The inputs are protected from both over and
under voltage. It is recommended, however, that they only be connected to 5V or
3.3V digital logic signals. Outputs can be configured to send an output signal or
strobe pulse. When configured as outputs, each line can sink 10mA of current.
NOTE: To use the RS232 functionality, a level converter must be used to convert
the TTL digital logic levels to RS232 voltage levels. See B&B Electronics
(http://www.bb-elec.com/) P/N: 232LPTTL for an example.
NOTE: The GPIO connector shipped with the MicronTracker is only an example
of the type of connector required. To maintain EMC certification as a medical
device, any application that make use of the GPIO port should make sure that a
non conductive shell shall be present on the GPIO connector.
7 Accuracy Performance
7.1 Reporting Terms
Accuracy, or trueness, is the closeness of measured values to their expected
quantity as defined by some standard. The difference between the true value and
the value reported by the instrument is called the measurement error. In
measuring a spatial position, the measurement error is a 3-vector (dx, dy, dz),
where each component is the difference between the measured position and the
true position projected onto the corresponding axis. The error value is often
reduced to a single number, which is the Euclidean length of the error vector, i.e.,
(dx2 +dy2+dz2).
Due to a variety of uncontrolled factors, repeated measurements of the same
quantity under different conditions generate slightly different values. It is,
therefore, common to take a statistical view of accuracy, describing it in terms of
error distribution around the true value.
A number of statistical values may be used to quantify error distribution in a
single number. The maximum error (or its close relative, the median error) is
easy to define, calculate and understand. Unfortunately, it is not a useful term in
characterizing measurement performance, since its value critically depends on the
number of samples taken and, due to random factors, may vary widely from one
experiment to another unless the number of samples is very large, sometimes
impractically so. The average error is also easy to define and calculate. It does
not, however, reflect well enough how “safe” it is to rely on the reported value. In
averaging, small errors are assigned the same importance as large errors, while
users often consider large errors more important than small ones. The average
error, therefore, tends to generate lower values than what users feel is the “true”
inaccuracy of the instrument.
Perhaps the most useful statistical error quantification is the % confidence
interval (CI), defined as the value below which a specified % of the samples fall.
For example, the value reported for the 95% confidence interval (95%CI) implies
that only 5% of the samples had higher error values. Unfortunately, CI values
have not been widely reported in the past, and there is no consensus regarding
what specific interval to use.
The most frequently used measure of spatial measurement accuracy is the Root
Mean Squared Error (RMSE), calculated by averaging the squares of the
individual error values, then taking the square root of the average. If all the errors
have the same magnitude, RMSE generates the same value as the average error.
When some of the errors are larger than others, however, they are given more
weight, resulting in the RMSE being larger than the average.
In the many experiments conducted to measure the accuracy of MicronTracker
products we found the ratios between RMSE and 95%CI to be very similar across
experiments:
95%CI 1.85 x RMSE
7.4 Calibration
During calibration, a sequence of steps is executed to extract individual camera
characteristics, which are later used in converting the input pixel values into 3D
target positions. In the most critical calibration step coefficients are extracted for
the equations that convert pixel locations in the image into the corresponding
projection rays in camera coordinate space (see 2.4). To obtain those coefficients,
a 3D grid of targets at nominal spatial relationships to each other is presented to
the camera, and the projection equations’ coefficients are tuned to obtain the
closest match possible between the nominal spatial positions of the targets and the
equation representing the location of their projections in each image. Depending
on the model, 10,000-50,000 grid positions are used in the calibration of each
MicronTracker camera. Thus, many grid positions contribute to the calculation of
each projection coefficient. The effects of drift and jitter are reduced by
controlling the environment in which the calibration is made and by averaging the
results of repeated measurements of the same grid position.
Errors may be introduced during the calibration itself as a result of:
Residual drift and jitter errors;
Interpolation and grid registration errors;
Deviations between nominal target grid positions and true positions.
Due to the large amount of redundancy in the process, the accumulated effect of
these errors expresses itself as a vector between each nominal grid position and its
position as calculated from the pairs of images acquired during the calibration.
The calibration accuracy is calculated from the magnitude of these error vectors.
The RMSE of the calibration accuracy has been adopted by suppliers of
competing pose tracking products and is, therefore, particularly useful in obtaining
a simple comparative accuracy yardstick.
A detailed record of the calibration process, including the calibration error
statistics and the range of coordinates over which it was measured, is stored in the
calibration file of each camera, and can be easily obtained through the API (see
8.3). ClaroNav Inc.guarantees that the calibration error of each camera shipped is
lower than specified.
Grid registration errors are minimal at the center of the volume over which the
calibration is performed, increasing towards its edges and beyond. To alert users
to the degraded accuracy, an “outside calibrated volume” hazard warning is
appended to the reported marker’s pose when the depth (z coordinate) exceeds the
farthest end of the FOM (1200/1150/2000/2400mm for H40,Hx40/S60,
Sx60/H60,Hx60/H3-60 models).
The specified maximum calibration error for MicronTracker is
0.20mm/0.25mm/0.35mm/0.20mm RMS for H40, Hx40, Sx60/H60, Hx60/H3-
60. The actual calibration error for a specific camera is recorded in the calibration
file header and can be viewed using the demonstration application (<Calibration
info> in the <Display> menu), or accessed programmatically through the MTC
API (Camera_CalibrationInfo or Camera_CalibrationFileInfo).
measurements. The following table summarizes how various potential drift factors
impact the performance of the MicronTracker:
7.6 Jitter
Measurement jitter is the direct result of image noise. Image noise, in turn, is
affected by the camera’s gain setting. MicronTracker’s automatic exposure
algorithms have been designed to optimize exposure settings for minimal
measurement jitter. Proprietary jitter-filtering algorithms further reduce
measurement jitter when markers are momentarily static. The jitter filtering has no
negative impact on measurement accuracy or response lag.
The following table shows the Hx40 jitter’s standard deviation under a selection
of typical measurement conditions. In general, jitter for static targets, such as for
reference markers, is very low under all conditions. Jitter magnitude for moving
targets is proportional to the root of the gain factor and the distance.
NOTE: The VB6 Demo application code is used internally by ClaroNav during
development and quality control, which sometimes requires access to low-level or
advanced features not supported as part of the public API. The development-only
parts of the code are conditionally compiled, depending on the value of the “dev”
pre-processor variable (i.e., they are bracketed between “#if dev then” and “#end
if”).
The Activate menu allows starting/stopping the data capture or the frame
processing. The frame update rate and the data capture status are displayed at the
top right of the images. <Externally Trigger> and < Strobe Signal> are available
for Sx60, Hx40, Hx60 and H3-60 cameras. Enabling <Externally trigger> will
stop the normal capturing mode and will put the camera in different mode. In this
mode the camera, does capturing and frame processing right after receiving an
external signal trough GPIO connector. The <Strobe Signal> doesn’t interfere the
normal execution of the application and just transmits electrical signals when the
camera captures new images. Toggling on <Background Process> makes the
Enable <Markers Position at ref Space> to display the pose data at the
Reference space. Both marker and the tooltip pose data will be displayed at the
reference space. By disabling this option it will display the pose data at the
“Camera-Space”.
When <Magnified tooltip> is activated and a marker with a corresponding tooltip
is detected, the regions around the tooltip in both left and right images are
magnified 4X (from the original video image), contrast-enhanced, and displayed
as insets within the video images. A cross mark shows the assumed location of the
tooltip, as extrapolated from the marker location. This provides an excellent
indication of the exact location actually being reported as being at the tip of the
tool, and provides a simple visual inspection to detect problems with tooltip
registration (see 8.6.3), marker attachment or tooltip step bending.
When the <Vectors> option is on, the vectors of identified facets are shown in
cyan (the longer vector, defining the X axis) and magenta (the shorter one).
Vectors that were not matched to a known facet are shown in yellow.
When exposure parameters are displayed, the demo also shows an estimate of the
amount of illumination, in Lux, falling on the brightest marker tracked.
The demo includes support for identifying a variable number of vectors used for
marking a variable number of locations, and not associated with any specific
marker. These locations are called “fiducials” and are assumed to be marked by
vectors with length of 10mm or less (this parameter can be programmatically
modified by changing the value of the FiducialVectorMaxLength property of
the main form). When the <Fiducials> option is turned on, such vectors are
marked in the image by a green circle and the letter “f”.
The calibration record of your camera can be viewed by selecting <Calibration
info>.
<Save left image…> saves the left (top) picture, exactly as it appears on the
screen (not as captured), compressed using JPEG, in a file.
The Video Recorder menu demonstrates the creation of a time lapse AVI file
documenting a procedure in video. When recording is activated, the user is
prompted to select the output AVI file path and the video compressor to use. The
time interval between recorded frames is determined by the setting of the
tmrAviSaver.Interval property of the main form (500 milliseconds by
default). To minimize interruptions to the pose measurement cycle on slower
computers, an uncompressed format is recommended. A blinking red dot appears
at the top right of the top image when recording is in progress.
To eliminate image brightness oscillations during the video recording, you should
ensure that the <Hist. Equalize> option in the Display menu is checked.
8.5 Help
<MTC API Documentation> opens a web browser to show the home page of the
API documentation. The <About…> dialog box provides information on version
number of the executable files being used, and allows access to complete system
information.
After saving the template, the database is automatically checked to see if the new
marker’s facet is too similar to other facets in the database, given the current
setting of the template matching tolerance (see 12.3). If a
similarity warning appears, you can choose to reduce the tolerance (Options
menu), delete the similar marker(s) from
the database, delete, redesign and re-
register the new marker, or ignore the
warning (taking the risk of possible
misidentification of markers).
Markers are stored in text files (.ini
format) in the “Markers” folder, which is
automatically created in the directory
where the application’s executable resides.
As necessary, they can be manually edited,
e.g., by using “Notepad” or “Wordpad”.
They may also be copied to other
computers and used with any
MicronTracker model.
The Templates form also enables to define
a reference marker. Select the Marker
properties Tab and from the known
Templates list select a marker and then
click on “Set this as the reference” button.
From this moment the selected marker will
be recognized as the reference marker for
all other known markers.
By clicking on each known marker from
the list, it will be displaying the name of
reference marker for that particular
marker. Note that if you select the
reference marker from the list it will show
“Camera-Space” as a reference for the reference marker.
NOTE: to remove incompatibility problems, the decimal dot in numbers stored in
marker files represented by a period, regardless of the OS regional number
format settings. Commas inserted manually would be interpreted as dots when the
marker files are loaded.
NOTE: The mechanical structure of the Ttblock example does not provide highly
accurate results (typical errors can be up to 1mm in position and 20 degrees in
orientation). For more accurate results you can either order pre-calibrated tools
from Claron, or use a more robust and accurate tip calibration setup/procedure of
your own.
To perform the registration, ensure that only the tool’s marker and the “Ttblock”
marker are detected. Place the tip of the tool in the small notch on the board,
touching the origin point of Ttblock (the middle of the long vector), with the tip’s
stem at an elevation of 90 degrees directly above the orientation line pointing at
the marker origin. Click on the <sample tooltip> button to begin sampling. Once
a sufficient number of samples was collected (e.g., 50), click on the button again
to complete the sampling and save the result by clicking on <Accept and Save>.
You should now observe the tooltip arrow overlaid on the video images. The tip
of the arrow points at the positive Z direction in the tooltip coordinate system, and
its side lobes lie in the tip’s XZ plane.
Without changing the relative position of the new tool with respect to the
camera, reposition the camera and collect another set of samples by clicking on
<Sample Facet> to toggle sampling on.
Click on <Accept & Save>. The marker is now in the database. Its name
should appear next to the marker in the upper (left) image whenever the marker is
in the FOM.
After saving the template, the database is automatically checked to see if the new
marker’s facet is too similar to other facets in the database, given the current
setting of the template matching tolerance (see 12.3). If a similarity warning
appears, you can choose to reduce the tolerance (Options menu), delete the similar
marker(s) from the database, delete, redesign and re-register the new marker, or
ignore the warning (taking the risk of possible misidentification of markers).
Markers are stored in text files (.ini format) in the “Markers” folder, which is
automatically created in the directory where the application’s executable resides.
As necessary, they can be manually edited, e.g., by using “Notepad” or
“Wordpad”. They may also be copied to other computers and used with any
MicronTracker model.
Click on the “COP” button. Once enough samples are collected the Xpoints will
turn into green. Click again on the “COP” to toggle it off. Now the “Accept &
Save” button should be enabled. Click on it to save the template file.
You may need to close and run the application. From the activate menu select
“Clouds Processing” and from the Display menu select “Display Clouds”. The
registered cloud should be detected and the pose data will be displayed on the
screen.
Once the COP gets detected, a COP object will be available with the following
information:
a. The name of the COP
b. Collection of Xpoints objects in the COP and their corresponding indexes and
3D positions in the Camera space. And information to determine whether
they have been visible to the camera or not.
[MarkerNames]
Count=1
Marker0=TTBlock
[TTBlock]
Tolerance=0.75
VectorCount=2
Vector0=90.0
Vector1=86.5
Each marker then has its own section, where fields “Tolerance”, “VectorCount”,
and fields for each vector length named “Vector#” are specified.
After loading the marker template dialog, an entry for each specified marker will
be listed in the drop down menu. By selecting the desired marker, the name will
automatically populate in the marker name text box. You can then proceed with
the normal registration process and the vectors lengths will be filtered
accordingly.
easily change the name and/or location of the file by modifying the function
tmrCheckFile_Timer.
When <2 markers> statistics is chosen, the program shows distance and angle
statistics relating to records containing exactly two markers (only). Otherwise, it
shows statistics relating to the tooltip position of the first marker in each record
(which is identical to the marker’s origin position if a tooltip is not registered for
that marker).
1. 3D arrow (at the center of the screen), pointing in the direction the
AccBar must be moved to reach its required location.
2. (Optional) outline of the FOM region inside which the AccBar must be
placed.
3. Virtual AccBar icon, showing how the tool must be oriented in the
camera’s view (aligned with either X, Y or Z axis of the camera).
Aligning the actual tool with its virtual image will correctly place the
AccBar inside the current sampling region.
4. A status bar in the bottom right corner, displaying the percentage of
AccBar samples collected so far.
5. A status message (bottom left corner) reporting the progress of sample
collection. The message can be one of the following:
o AccBar not found (the marker is not detected by MTC);
o Sampling disabled: reduced accuracy (a hazard warning is
issued);
o Low light (the operating environment is not sufficiently
illuminated);
The Application menu provides the option of restarting the R-Fine’s sample
collection procedure, or exiting the application.
Much of the selections in the Display menu are the same as those for the VB6
Demo (see 8.3). In addition, <Region Borders> option is provided to toggle
on or off the display of the 3D outline, highlighting the location in the FOM
where the AccBar must be placed.
The <Documentation> opens a default web browser to the home page of the
R-Fine documentation. The <About…> dialog box shows version numbers of
the executable and DLL files, and allows access to complete system
information.
Once all of the required samples have been collected, R-Fine performs their
statistical analysis and automatically executes a refinement algorithm, which
adjusts the calibration to best fit the data. The improvement in accuracy resulting
from the refinement is measured as a percentage of a combined decrease in the
AccBar length variability and projection ray convergence values (see 7.4.1).
Upon successful refinement (resulting in a non-zero improvement), the application
provides a choice of permanently modifying the calibration by updating the
camera’s calibration file. If automatic saving is enabled, R-Fine will instead save
the improvement result, and show the Results dialog for information.
If an attempt to refine the calibration failed, the user is provided with an option to
retry the procedure by collecting a new set of AccBar samples (this will restart R-
Fine), or exit the application. Refinement failure is usually an indication that the
camera’s calibration hasn’t yet undergone a significant degradation in accuracy.
This outcome should also be supported by a low measure of length variability
before and after the refinement (typical variability results for a recently calibrated
camera are in 0.25-0.35 mm range for Sx60, 0.20-0.30 mm range for H40 and
Hx40, 0.30-0.40 mm range for H60 and Hx60, 0.20-0.35 mm for H3-60).
Sometimes it is required to move the bar towards or away from the camera
After all samples have been collected, the R-Fine application will determine
whether the camera is out of alignment or not. The application will automatically
update the calibration when there is improvement. If more than 10% improvement
is observed, we ask that you repeat the process until you achieve less than or equal
10.2 XP contrast
For best visibility and accuracy under typical lighting conditions, the pattern
making up each XP should be printed using the highest possible contrast between
the dark and white regions, at the highest possible resolution. The surface should
be as non-reflecting (matte) as possible. The brightness of the surface should not
be easily affected by contact with human hands. If it would be used for along
period, it needs to be easy to clean and, as necessary, sterilize.
NOTE: Non-reflective XPs with deep black and bright white regions not only
provide the best identification reliability, but also improve accuracy by ensuring
each region appears highly uniform in the video images.
Under intense or mixed lighting conditions, Xpoint contrast may be reduced to
allow its detection in a situation in which it may otherwise be over-exposed
(“washed-out”). For example, XPs printed in black on a 33% gray background
(1/3 the brightness of white) would still be detected concurrently with a white-
background marker even when it is illuminated by a light that is 15 times more
intense (see pictures below).
Uniform lighting
spotlight
MicronTracker Developer’s Manual, MTC3.8 73
NOTE: Slightly larger black/gray Xpoint regions are required to obtain the same
maximum detection range as black/white Xpoint regions.
10.3 XP flatness
The surface on which XPs are printed should be flat. A slightly curved Xpoint
region would still be detected, but may produce position reports that will vary
somewhat depending on the orientation of the Xnormal relative to the camera.
This small drift may still be acceptable for some applications.
10.4 Xlines
The two Xlines do not need to be perfectly perpendicular to one another, but this
is desirable for maximum angular detection range.
To eliminate measurement errors resulting from incorrect determination of the
extent of the Xpoint region, Xlines should preferably terminate at a sharp angle.
This is especially important when two Xpoint regions are placed next to each
other at an angle.
Good Bad
! ! ?
If the neighboring Xpoints are a mirror image of each other, it is often useful to
have them share an Xline as shown below.
XLine sharing
NOTE: Two XPs sharing an Xline automatically form a vector. This vector,
however, does not necessarily have to participate in the facet to which the XPs
belong. The marker templates dialog of the VB demo provides a simple way to
ignore such vectors when registering a template by setting a minimal vector
length threshold (see 8.6.1).
The facet pose (coordinate transform) reported by the MicronTracker software is
the one that minimizes RMS error between the facet template positioned at the
reported pose and the actual measured positions of each Xpoint. Therefore, all else
being equal, the more Xpoints in a facet, the lower the overall pose measurement
error. Since two vectors can either share or not share a single Xpoint, the number
of Xpoints in a single facet is either 3 or 4. It is possible, however, to construct a
flat marker with any number of Xpoints higher than 3 (except 5) by combining
multiple facets into a multi-facet marker (see 10.6.2 below).
If d is small relative to l, the tip error is dominated by the Xpoint error (roughly,
calibration error + jitter, typically in the range 0.2mm-0.4mm RMS). If d is large
relative to l, the error is dominated by the ratio d/l. As a rule of thumb, to ensure
that RMS error at the tip is sub-millimeter, keep ld.
l d
c
a
m
10.5.3 Facet design alternatives e
r
10.5.3.1 For tooltip sensing a
When the marker’s purpose is to provide the tooltip sensing, and the orientation is
unimportant or secondary, two simple designs, shown below, should be
considered.
The L design provides the most efficient use of marker area, and is thus preferable
when minimizing marker size is paramount. Accuracy degrades quite rapidly
away from the X axis direction (the blue arrow), and it is thus very important to
place the marker such that the X axis will pass through, or very near, the tip.
The narrow X design requires a larger area at the front, but reduces tip position
RMS error, by 10-20% when the tip is placed along the X axis, and by up to 50%
when the tip is off-axis. It also improves the roll angle sensing accuracy. Note that
the angle between the two vectors needs to exceed 8 degrees.
10.5.3.2 For general pose sensing
When angular measurement accuracy is equally important in all directions, the
following two simple designs should be considered.
“L” design
“X” design
another facet on top allows detection from above as well, providing the full
angular detection range needed in practice. The picture below shows two views of
an example of such an arrangement. The three facets contain 3 Xpoints each.
The design below is intended for a marker to be placed to one side of the region in
which measurements are performed, and is, therefore, designed to provide
increased angular accuracy in the direction of that region.
4-facet
mark
designed
or impr
v
d
ni-di
ectional accuracy.
10.9 Materials
Paper printed using an inkjet or laser printer and glued to a flat surface is perfectly
suitable for prototyping and experimentation. For parts used in a clinical setting, it
is desirable to use a highly durable material, one that can be easily cleaned and
can withstand many cycles of steam sterilization. Such a material is, for example,
“0.008” hardcoated fine textured weatherable Polyester” available from Cellotape-
Landmark Label.
Circular or over XP regions made of such material can be obtained from ClaroNav
under the trade name Duramarks. They can be fixated to a marker carrying surface
in an accurately repeatable manner by creating 1mm deep “pockets” in that
surface. Larger marker stickers can be custom printed on order by professional
label makers, such as Cellotape-Landmark.
11 Programming Interface
11.1 Programming Model
The figure above shows the object classes used in controlling the sensor and
obtaining pose measurements. One object of each of three classes, Cameras,
Markers and Xpoints, is automatically created when the interface library starts
running. The application refers to those instances directly by their class name
without the use of an instance handle.
The Cameras object represents the array of MicronTracker cameras performing
pose measurements. Calling Cameras_AttachAvailableCameras establishes
the communication link with all the cameras physically connected to the computer
and loads the proper calibration file for each of the cameras. Handles to individual
camera objects representing attached cameras may be obtained using
Cameras_ItemGet. Each camera can then be individually accessed and
controlled. However, the most common operations relating to cameras can be
called collectively through the Cameras interface. Upon terminating the use of
cameras, Cameras_Detach must be called to signal the driver to disassociate
itself from the application process.
The Markers object represents all the markers being potentially tracked by all
cameras, i.e., markers with a registered template. Each marker is represented by a
separate Marker object, which, in turn, maintains a collection of Facet objects, one
per facet, in its template. Each Facet object maintains two Vector objects. A
There are two different measurement cycles, Markers Processing and Xpoints
processing:
1. Markers Processing: A measurement cycle for Markers Processing includes
the following steps:
Calling Cameras_GrabFrame to transfer the latest frame to the MTC’s
buffers;
Calling Markers_ProcessFrame to extract measurements from the
frame buffers;
Calling Markers_IdentifiedMarkersGet to obtain a collection of
handles to the identified markers;
For each identified marker in the collection, calling
Marker_Marker2CameraXfGet to set the attributes of an application-
owned Xform3D object to the pose of the marker. Optionally, the tooltip
pose, obtained via Marker_Tooltip2MarkerXfGet, may be
concatenated to the marker’s pose to obtain the pose of the tooltip in the
camera’s coordinates.
Accessing the Xform3D attributes to perform application-specific
operations.
NOTE: To speed up frame grabbing and markers processing, MicronTracker can
also do all the computation in the background when enabled with
Markers_BackGroundProcessSet().
This will make MicronTracker grab and process frames automatically in a
background thread and return to the user the most recently processed pose data
when requested.
12 Advanced Topics
12.1 Camera Exposure Controls
12.1.1 Exposure, Gain and Shutter
To enable detecting and correctly pinpointing an Xpoint in the image, a minimal
contrast of about 20 pixels values between its dark/bright regions is necessary.
Furthermore, its bright regions should not be over-exposed, i.e., the pixel values
there should fall below the maximum pixel value (255). The range of light
intensity encountered during operation is extremely wide (50-100,000 Lux),
making it necessary for the MicronTracker software to frequently adjusting the
camera exposure settings, which is represented by the Exposure property of the
Camera class.
The Exposure property value is actually a product of two other properties, GainF
(gain factor) and ShutterMsecs. ShutterMsecs corresponds to the length of the
period, in milliseconds, in which each pixel in the sensor is exposed to photons
before a measurement is taken (in the range 0.13 to 33 ms for the S60, 0.17 to 66
ms for the H40, 0.17 to 67 ms for the H60 model or 0.01 to 67 ms for H3-60 and
Sx60 on 15fps mode) 0.01-50 ms for Hx40 and Hx60. The GainF parameter
controls the amount of amplification the analog signal received from each pixel
undergoes before being converted into a digital value (in the range of 1-20 for
H40, 1-32 for S60, 1-24 for Sx60, Hx40 and Hx60 and 1-22 for H60). Gain is
similar to the ISO rating of a film. A higher gain increases the sensitivity of the
camera, but also increases the noise level. Noise level, and with it measurement
jitter, increases proportionally to the square root of the gain factor (e.g., a gain
factor of 4 will produce roughly twice as much jitter as a gain of 1).
In a given lighting situation, a range of different combinations of shutter and gain
can be used to achieve a similar distribution of gray levels in the image. Lower
gain will result in lower jitter. However, under low lighting conditions low gain
values will also result in longer shutter periods, in turn blurring moving objects
and causing potential loss of tracking during marker movement.
Since the shutter time controls the amount of motion blur for a given marker
movement, ShutterMsecsLimiter should be set to the highest value in which, under
typical operating conditions, tracking is still maintained. Similarly, GainFLimiter
should be set to a value corresponding to the maximum measurement jitter
acceptable.
When lighting is strong (over 500 Lux), the settings of the limiters should have no
noticeable effect on tracking performance. Their effect is more noticeable in
medium or dim lighting conditions, where trade-offs are required. Applications
facet and the between the relative positions of Xpoints in different facets (see
10.5). It may be useful to increase that value from its factory default setting
(1mm) when measurements need to be performed on large markers at distances
larger than 1.5m from the camera (which is outside the calibrated volume!), or
when it is important to continue pose tracking despite some distortion of the
marker geometry (for example, when the marker is placed directly on a patient’s
skin). It may be useful to decrease its value when a large number of slightly
different markers need to be distinguished, and the measurements are performed at
distances shorter than the full depth of the calibrated volume. The Options dialog
of the demo application allows experimenting with the tolerance setting (see8.9)
The method Validate of the Markers class provides a quick and useful way to
detect potential misidentification problems in the set of marker templates given
the current TemplateMatchToleranceMM value. It should be executed whenever
changes to either the marker templates or the value of
TemplateMatchToleranceMM may have been made (see usage examples in the
VB6 demo application code).
13 Standards
MicronTracker camera models Hx40, Hx60, Sx60 and H3-60 have been certified
to comply with the following Standards:
CAN/CSA C22.2 No. 60601-1:2008-Class II device
ANSI/AAMI ES 60601-1/A2:2010
IEC 60601-1:2005- CB Test for Class II device
IEC 60601-1-2:2004 – Class B device
IEC 61000-4-2:2001 (Electrostatic Discharge)
IEC 61000-4-3:2002 (Radiated field immunity)
IEC 61000-4-4:2004 (Electrical Fast Transients / Bursts)
IEC 61000-4-6:2003 (Conducted RF Immunity)
IEC 61000-4-8:2001 (Power Frequency Magnetic Field)
CISPR 11 (Radiated Emissions)
IEC 60601-1:1988 + A1:1991 + A2:1995 and National Deviations to:
CANADA – CAN/CSA-C22.2 No. 601.1-M90 (Update No.2, 11/2003)
United States UL 60601-1: 2003
NOTE: that camera enclosure leakage needs to be measured in the end use application.
The MicronTracker also complies with Directive 2011/65/EU of the European Parliament and
of the Council of 8 June 2011 on the restriction of the use of certain hazardous substances in
electrical and electronic equipment.