Multispectral imaging.

Applications
Vladimir Bochko Jarmo Alander

University of Vaasa

Vladimir Bochko Multispectral imaging. Applications 1/26

Contents

• Introduction.
• PCA.
• Skin wound classification.
• Dichromatic reflection model.
• Image colorization.
• Image fusion using image processing.
• Estimating spectral reflectance of painting.
• Electronic endoscope. Image fusion using variable
selection.

Vladimir Bochko Multispectral imaging. Applications 2/26

Multispectral imaging

Vladimir Bochko Multispectral imaging. Applications 3/26

Why multispectral and hyperspectral
imaging systems?
The color models is based on the CIE 1931 Standard Observer. The
Standard observer is assumed to integrate the light source power spectrum
S(λ) and the spectral reflectance of the object R(λ) via three matching
functions x(λ), y(λ) and z(λ). There are problems with color imaging
systems composed by 3 channeles Red, Gree and Blue [5]:
• The spectral characteristics of digital cameras are different from
matching functions and differ from each other.
• Difficultes in reproduction images under particular illuminant if images
are captured under different illuminant.
• Metamerism. Two different spectra may be reproduced as the same
color at the particular illuminant and to be different colors at another
illuminant.
The solution is multispectral imaging systems.

Vladimir Bochko Multispectral imaging. Applications 4/26

Visual and near-infrared images
(VIS-NIR)
• A visible (VIS) and near-infrared (NIR) range are desirable in many
applications.
• The VIS range is from 380 to 780 nm. The components are reproduced
as grayscale images or an RGB color image.
• The NIR range is from 780 to 2500 nm. The components are
reproduced as grayscale images or any three components can be taken
and used as R,G,B components. i.e. pseudocolor reproduction.
• The NIR light penetrate deeper in materials, e.g. paint or tissue, and
may show information invisible by a human eye.
• In addition, the contrast agent may be injected and propagated into
blood or lymphatic vessels. For example, idocyanine green gives a
fluorescent light at 830 nm after the exciting illumination at 780 nm. The
fluorescent light highlight the vessels in the NIR image. This is used for
analysis in medicine.

Vladimir Bochko Multispectral imaging. Applications 5/26

Multispectral imaging
• Professor Yoichi Miyake (left), Chiba University, Japan. Talk:
Multispectral imaging and its application to medicine: The development
of diagnosis system for life style diseases using color information,
University of Vaasa, June, 2011.

Vladimir Bochko Multispectral imaging. Applications 6/26

Multispectral imaging

• Mitsubishi 5-band spectral camera.

• Canon 8 ink jet printer.
• Spectral scanner by Mitsubishi Heavy Industry. The sensor
range 380-860nm, at 10nm.

Vladimir Bochko Multispectral imaging. Applications 7/26

Image reshaping for computation
• We reshape a 3D image using concatenation along columns or rows.
This is needed for PCA, MLR, PCR and PLS. After computing for
reproduction we reshape data to obtain the 3D image.
• In SOLO [9] it is called folded (left) and unfolded (right) images.

Multispectral 4 1 4
image 2D matrix

Samples/Pixels
3D matrix
100
ts
en
on
mp
Co

100 10000
Variables/Components

Vladimir Bochko Multispectral imaging. Applications 8/26

Spectral images and color
reproduction

• Multispectral image 81 components taken in the range

380-780nm, at 5nm, by a Specim camera.
• Multispectral image can be presented by a cube with two
spatial and one spectral (wavelength) dimension.
• The RGB color image can be reproduced from the
multispectral image using the color matching functions
x(λ), y(λ) and z(λ) and a spectral power density of the
light souce S(λ)(e.g. D65).
• The spectral reflectance of the object R(λ).

Vladimir Bochko Multispectral imaging. Applications 9/26

Spectral images and color
reproduction

In Fig. S(λ) is normalized to 100 at 560 nm.

Vladimir Bochko Multispectral imaging. Applications 10/26

Spectral images and color
reproduction
Z 780
• x, y, z, are the color matching functions.
X=k R(λ)S(λ)x(λ)dλ • R(λ) is a spectral reflectance factor of
380
the material.
• S(λ) is a relative spectral power
Z 780
distribution of the light source.
Y =k R(λ)S(λ)y(λ)dλ
380 • λ is a wavelength.
• k is a normalizing constant.
Z 780
Z=k R(λ)S(λ)z(λ)dλ • Note: in colorimetry X, Y and Z are
380 measured in percents and k is
multiplied by 100.
1     
k = R 780  R a1,1 a1,2 a1,3  X 
S(λ)y(λ)dλ
    
380 G = a2,1 a2,2 a2,3   Y 
    
B  a3,1 a3,2 a3,3   Z 

Vladimir Bochko Multispectral imaging. Applications 11/26

PCA

k=1, 89.25% k=2, 97.78% k=3, 99.31% k=10, 99.92%

Vladimir Bochko Multispectral imaging. Applications 12/26

Skin wound image analysis
• The RGB color image (left). In the center of yellow area the spot is gray.
• The red component of the RGB color image was fused with the NIR
image using the multiresolution pixel fusion and replaced by the fused
image.
• The RGB color image with the fused red component (right). In the
center of yellow area the spot is red.

Vladimir Bochko Multispectral imaging. Applications 13/26

Skin wound image analysis
• The developed multispectral camera acquires images: NIR, red, green
and blue, using LED light. The red, green and blue are reproduced as a
color image [1].
• Preprocessing. Image calibrated using reference white and black like in
spectrophotometry.
• Support Vector (SV) classifier.
• Postprocessing. Either morphological operations (dilation, erosion) or
median filtering.

Test image
Preprocessing

Normalized image
SVM model
SV classifier

Segmented image
Postprocessing

Smoothed image

Vladimir Bochko Multispectral imaging. Applications 14/26

Skin wound image analysis
• Skin wound image segmentation. The NIR and color images are
combined and used as a four component image.
• After segmentation we measured area of skin wound and observed a
wound area change over the time. Color images (top row), NIR images
(middle row) and segmented images (botttom row).

Vladimir Bochko Multispectral imaging. Applications 15/26

Dichromatic reflection model
• The Dichromatic Reflection Model describes light reflection from
inhomogeneous dielectric materials as a function of two independent
reflection processes: body (diffuse) and surface (specular) reflection [6].
• The model is applicable to: plastics, paints, papers, ceramics, textiles,
vinyl, tiles, fruits, leaves, woods and human skin. It is not applicable for
metals.

Surface reflection Surface Incident light

normal
Body reflection

Surface

Medium

Colorants

Vladimir Bochko Multispectral imaging. Applications 16/26

Dichromatic reflection model
• We consider the segmented uniformly colored regions (dielectric
surfaces) under single color illumination.
• The physical approach suggests the dichromatic reflection model for
describing the data structure in the RGB color space.
• For analysis only regions with cherries are used. The highlight cluster
caused by surface reflection.

Vladimir Bochko Multispectral imaging. Applications 17/26

Image colorization

• The dichromatic reflection model is used for colorization.

• The orange and red pepper colors were used for colorization of a
synthetic teapot [3].

Vladimir Bochko Multispectral imaging. Applications 18/26

Image fusion using image processing
• Brain surgery images, color (upper-left) and NIR (upper-right). Images courtesy
of Mr. Kari Koivuporras.
• They can be fused using alpha blending (bottom-left) or image colorization
(bottom-right).
• The other approach based on PCA and pseudocoloring will be implemented in
SOLO.

Vladimir Bochko Multispectral imaging. Applications 19/26

Estimating spectral reflectance of
painting
The system consists of the single chip high quality CCD camera and the
rotating color filter wheel comprising several color filters [2].

Painting

Color
filters

Illuminant

Digital camera

Vladimir Bochko Multispectral imaging. Applications 20/26

Estimating spectral reflectance of
painting
Hermitage. Scientists of the department of scientific and technical
examination use NIR imaging for authentication of work of art.

Vladimir Bochko Multispectral imaging. Applications 21/26

Spectral electronic endoscope.
Image fusion using variable selection.
Recently, a new electronic endoscope based on a conventional electronic
endoscope was developed to reproduce arbitrary multispectral images of the
mucous membrane such as gullet, colon and stomach [7].

The conventional electronic endoscope for color images is shown in Figure.

Vladimir Bochko Multispectral imaging. Applications 22/26

Spectral electronic endoscope

• Original color image (left). Selection of components by doctors (middle

image) [7].
• The way for automatic component selection using a Fourier transform
and an area defined by a chromaticity histogram (right) [4].

Vladimir Bochko Multispectral imaging. Applications 23/26

Spectral electronic endoscope
• Color images (top).
• The automatically fused components of multispectral images (bottom).

Vladimir Bochko Multispectral imaging. Applications 24/26

References
Bochko V., Vlisuo P., Harju T. and Alander J., Lower Extremity Ulcer Segmentation of Visual and
Near-Infrared Imagery, Skin Research and Technology, Vol. 16, 2010, pp. 190-197. doi:
10.1111/j.1600-0846.2009.00415.x.

Bochko V., Tsumura N. and Miyake Y., Spectral Color System for Estimating Spectral reflectance of Paint,
Journal of Imaging Science and Technology, Vol. 51, No. 1, 2007, pp. 70-78.

Pipirigeanu A., Bochko V. and Parkkinen J., A Computationally Efficient Technique for Image Colorization,
The Computational Color Imaging Workshop (CCIW’09), Saint Etienne, France, March 26 and 27, 2009.

Bochko V., Miyake Y., Tsumura N., Nakaguchi T., Image Fusion in Spectral Electronic Endoscope, The Ninth
International Symposium on Multispectral Color Science and Application, Taipei, Taiwan, 30 May - 1 June,
2007. pp. 109-115.

Hill B., High Quality Color Image Reproduction: The Multispectral Solution. The Ninth International
Symposium on Multispectral Color Science and Application, Taipei, Taiwan, 30 May - 1 June, 2007. pp. 1-7.

Klinker, G., J., Shafer, S., A., Kanade, T.: A Physical Approach to Color Image Understanding. International
Journal of Computer Vision 4 (1990) 7-38.

Miyake Y., Kouzu T. Takeuchi S., Yamataka S. Nakaguchi T., Tsumura N., Development of New Electronic
Endoscopes Using the Spectral Images of an Internal Organ, The 13th Color Imaging Conference CIC
2005, Scottsdale, Arizona, pp. 261-263, 2005.

Miyake Y., Evaluation of Image Quality Based on Human Visual Characteristics, The First International
Workshop on Image Media Quality and its Applications, Nagoya, Japan, pp. 10-14, 2005.

SOLO toolbox: http://wiki.eigenvector.com/index.php?title=Main Page

Vladimir Bochko Multispectral imaging. Applications 25/26

Questions

Vladimir Bochko Multispectral imaging. Applications 26/26