International Journal Of Computer Science And Applications Vol. 6, No.

2, Apr 2013 ISSN: 0974-1011 (Open Access)

Available at: www.researchpublications.org

Image Stitching Techniques-An overview

Ms. Vrushali S. Sakharkar Prof. Dr. S. R. Gupta
Computer Science and Engineering Computer Science and Engineering
PRMIT & R, Badnera, Amravati PRMIT & R, Badnera, Amravati
[email protected]

Abstract — In this paper, we describe various Image create the high resolution photo-mosaics used to produce
stitching techniques used to combine multiple images today’s digital maps and satellite photos. They also come
together to make a wide angle picture called Panorama. bundled with most digital cameras currently being sold, and
Algorithms for stitching and aligning have many can be used to create beautiful ultra wide-angle panoramas.
applications in computer vision like texture synthesis, We describe various Image Stitching phases in section 1.In
object recognition, medical imaging. Images of any object section 2,we give details of Image Acquisition phase. Image
or place are acquired using 360. rotating camera and the remapping is an essential process of Image Stitching described
overlapped portion of images should be aligned properly. in section3. Finally aligned images are blended together to
Image stitching requires identical exposures to produce avoid seams in the stitched image this procedure is known as
seamless results. It is also known as mosaicing. Stitching Image Blending specified in Section 4.
can be done into three stages Acquisition, Remapping,
A. Different kinds of Image Stitching
Blending. We have described different techniques for all
the stages. Image acquisition can be done using rotating There are different kinds of Stitching,
camera, translating camera and hand held camera. Image • Mosaic - stitch multiple rows of pictures that were
Remapping involves Alignment, conversion of camera taken without rotating the camera around a single
image to Sphere, Selection of final projection, Exposure point, but with the camera kept perpendicular with
setting. Image remapping is also called Warping. The final the subject.
warped images should blend perfectly so the visible seams • Panorama (single-row) - stitch a single row of
between the images can be removed. Image blending pictures (created by rotating the camera around a
techniques like Average Blending, Alpha Blending, single point in a flat plane, which is normally parallel
Feathering, Pyramid blending used to remove Exposure with the horizon).
difference, Blurring, Ghosting. Finally unwanted
• Panorama (multi-row) - stitch multiple rows of
information can be cropped using various cropping
pictures (created by rotating the camera around a
techniques like graph cuts, gradient blending etc.
single point in a flat plane but tilting or pitching the
Keywords-component: Image Stitching, Image Blending, camera up and/or down so that for each row of
Image Warping. pictures the lens is not necessarily parallel with the
plane of rotation).
• Panorama (pano-camera) - just stitch together the
ends of panoramic picture created with a panoramic
I. INTRODUCTION camera.
Image stitching or photo stitching is the process of • Spherical panorama - stitch any number of pictures in
combining multiple photographic images with overlapping such a way as to create a spherical panorama, the
fields of view to produce a segmented panorama or high- important distinction between this and single or
resolution image. Commonly performed through the use of multi-row panoramas is that the "poles" (i.e the very
computer software, most approaches to image stitching require top and bottom of the image) must be stitched also so
nearly exact overlaps between images and identical exposures that the user can look straight up and down and see a
to produce seamless results. It is also known as mosaicing. smoothly blended image.
"Stitching" refers to the technique of using a computer to 1) Stages of the Image Stitching Process
merge images together to create a large image, preferably a) Image Acquisition:
without it being at all noticeable that the generated image has This stage of Image stitching is to do calibration of Images. It
been created by computer. Algorithms for aligning images and requires selection of the position and acquisition of images. In
stitching them into seamless photo-mosaics are among the this step, a decision needs to be made on the type of resultant
oldest and most widely used in computer vision. Frame-rate panoramic images. According to the required panoramic
image alignment is used in every camcorder that has an images, different image acquisition methods are used to
“image stabilization” feature. Image stitching algorithms acquire the series of images.

NCAICN-2013, PRMITR,Badnera
324
International Journal Of Computer Science And Applications Vol. 6, No.2, Apr 2013 ISSN: 0974-1011 (Open Access)

Available at: www.researchpublications.org

sliding plate. The third set-up is where the camera is held in a

person’s hands and the person takes the images by turning
b) Image Remapping (Image Warping):
around on the same spot, or walking in a direction
Remapping is the process of changing the image geometry perpendicular to the camera’s view direction. In all three set-
in order to fit to the adjacent images that should contribute to a up’s, a still image camera has been used to take the images.
panorama. This remapping process "warps" the images in such The camera co-ordinate system is shown in Fig,2 where the
a way so that they can be aligned perfectly. Without this Z-axis points towards the object of interest and the Y-axis
remapping process, the images will not join together correctly. passes through the optical axis of the camera.
It includes conversion of camera image to sphere, Image
alignment, setting screen projection.
c) Image Blending:
Once the source pixels have been remapped onto the final
composite surface, we must still decide how to blend them in
order to create an attractive looking panorama. Blending is
used to remove visible seams (due to exposure differences),
blurring (due to miss-registration), or ghosting (due to moving
objects). Fig. 2 Camera co-ordinate system
d) Cropping: The camera’s angles of view in the horizontal and vertical
Image cropping refers to removing unwanted areas from a directions decide each image’s coverage in the horizontal and
photographic or illustrated image. One of the most basic vertical directions. The angles of view are defined in Fig.3,
Image manipulation processes, it is performed in order to where the angles and respectively represent the camera’s
remove an unwanted subject or change its aspect ratio, or to angles of view in the horizontal and the vertical directions.
improve the overall composition.

Fig. 3 The horizontal and vertical angles of view.

The template is used to format your paper and style the text.
All margins, column widths, line spaces, and text fonts are
prescribed; please do not alter them. You may note
peculiarities. For example, the head margin in this template
measures proportionately more than is customary. This
measurement and others are deliberate, using specifications that
anticipate your paper as one part of the entire proceedings, and
not as an independent document. Please do not revise any of
Fig.1 Flowchart of Producing Panoramic Image the current designations.

A. Acquisition by Camera Translations:

II. IMAGE ACQUISITION In this acquisition method, the camera is shifted in a
direction parallel to the imaging plane. On a smaller scale, the
Different image acquisition methods can be used to camera can be mounted onto a sliding plate to achieve the
acquire input images that produce different types of panoramic translations. The camera and the sliding plate are placed
images, depending on the type of panoramic images required directly in front of the objects of interest and an image is taken
and the availability of equipment. Three set-up’s to acquire with each translation of the camera until the series of images
images for panoramic image generation are described and cover the desired range. Fig 4. Shows the setup of this method,
discussed in this section. In the first set-up, the camera is set where the camera is aligned with the sliding plate so that the
upon a tripod and the images are obtained by rotating the imaging plane parallel to the orientation of the sliding plate is.
camera. The second set-up places the camera on a sliding plate
and the images are obtained by shifting the camera on the

NCAICN-2013, PRMITR,Badnera
325
International Journal Of Computer Science And Applications Vol. 6, No.2, Apr 2013 ISSN: 0974-1011 (Open Access)

Available at: www.researchpublications.org

in this situation, it is even more difficult to control the distance

shifted between each image and keep each image on the same
imaging plane. Therefore, apart from the difference in the size
of the overlapping regions, the image planes of the acquired
images have different orientations and cause problems in
image stitching. It is often desirable to have larger overlapping
Fig.4 Geometry for image acquisition by camera regions between adjacent images to reduce the effects of the
translations. above mentioned problems in acquiring images free-handed.
Given the camera translation, t, the distance between Larger overlapping regions implies that the camera rotations,
camera and object of interest, d, and the camera’s horizontal or translations between successive images are smaller, thus
angle of view, the ratio of the overlapping region to the whole reducing the amount of inconsistencies between images.
image can be estimated by Eqn. 2, Nevertheless, acquiring images by a hand held camera is very
easy to manage and can be performed in many locations where
l / L = 1 - t / 2 d tan(1/2) ---- (2) it might be difficult to set up equipment such as the tripod or
the sliding plate. If care is taken during the acquisition of the
Nevertheless, the actual size of the overlapping region
images, it is possible to produce panoramic images of similar
between successive images is determined by the accuracy in
quality to those generated with images acquired by mounted
setting up the camera. In acquiring images by translation, it is
cameras.
important to ensure that the image planes are parallel to the
direction of camera translations. Otherwise, the size of the
objects in the images varies as the camera is shifted, causing III. IMAGE REMAPPING (IMAGE WARPING)
problems in image stitching. One disadvantage of this method Remapping is the process of changing the image geometry
is that the required translation, t, increases as the distance in order to fit to the adjacent images that should contribute to a
between the camera and the object of interest, d, increases, if panorama. This remapping process "warps" the images in such
the acquired images are to have the same sized overlapping a way so that they can be aligned perfectly. Without this
region. Therefore, acquiring images when the object of interest remapping process, the images will not join together correctly.
is far away from the camera is more difficult due to the It highly depends on aligning, since any alignment change of
magnitude of required translations. Furthermore, since the an image requires a different remapping. Remapping is an
acquired images are all on the same plane, the panoramic inherent part of the stitching process. Remapping includes the
images obtained from images acquired by translation does not following steps
provide the feeling of looking into a 3D environment as in the
case for the panoramic images obtained from images acquired A. Conversion of camera image to sphere:
by rotation. The camera image has to be transformed into angle space,
which here is visualized as projecting the image from a virtual
B. Acquisition by a Hand held Camera: camera, reversing the light path, onto a sphere
This acquisition method is comparatively easy to perform;
the user simply holds the camera and takes images of his/her
surroundings by either rotating on the same spot or moving in
a predefined direction roughly parallel to the image plane.
However, images acquired by this method can be difficult to
stitch, due to the amount of unwanted camera rotation or
translation during the acquisition of images.
In the case of the user turning with the camera to obtain the
images, the user acts as the tripod of the camera in Section Fig. 5Conversion of camera image to sphere
2.3.1. However, rather than rotating about, or approximately
about the vertical axis of the camera, there are inaccuracies in B. Image Alignment (Image Registration):
the alignment of the rotating axis with the vertical axis. It is
also difficult to control the angles rotated between successive Generally, one is free to choose any orientation of the
images. Therefore, the sizes of the overlapping regions virtual camera/projector. But for panoramic images, one wants
between adjacent images have a greater variation than for to align these images. The usual software approach is to mark
images acquired by a camera mounted on a tripod. a set of same features in image pairs, and have the distance
between such markers pairs minimized in an overall
When the user holds up the camera and moves in one optimization process. In this section we describe different
direction to acquire the images, the user acts as the sliding image alignment techniques.
plate in acquisition by translation in Section 2.3.2. However,

NCAICN-2013, PRMITR,Badnera
326
International Journal Of Computer Science And Applications Vol. 6, No.2, Apr 2013 ISSN: 0974-1011 (Open Access)

Available at: www.researchpublications.org

Because there is no restriction on the input images, graph

search is applied to find connected components of image
matches such that each connected component will correspond
to a panorama. Finally for each connected component Bundle
adjustment is performed to solve for joint camera parameters,
Fig. 6 Image Alignment by matching features and the panorama is rendered using multi-band blending.

1) Feature based registration:

A feature (key point) is a significant piece of information
extracted from an image which provides more detailed 2) Recognizing panorama:
understanding of the image. In other words, features are
To perform fully automated image stitching is a technique
objects, which should be distinct, salient and preferable
frequently spread over the image and easy to detect. The to recognize which images actually go together, which Brown
objects can be for example landmarks or segmented areas and and Lowe (2003) call recognizing panoramas. If the user takes
images in sequence so that each image overlaps its
even intensity can be seen as a feature. The number of
predecessor and also specifies the first and last images to be
common detected objects (seen in both images) must be
stitched, bundle adjustment combined with the process of
sufficiently high to be able to perform a good estimation of the
transformation model. The features can then be seen as points, topology inference can be used to automatically assemble a
lines, and regions. The first task of the registration is to decide panorama (Sawhney and Kumar 1999). However, users often
what kind of feature we search for, and how to represent them. jump around when taking panoramas, e.g., they may start a
new row on top of a previous one, or jump back to take a
Most of the feature based registrations techniques can be
repeated shot, or create 360• panoramas where end-to-end
divided into four steps.
overlaps need to be discovered. Furthermore, the ability to
a) Feature detection: discover multiple panoramas taken by a user over an extended
period of time can be a big convenience.
The salient and distinctive features are detected. After
deciding what kind of features we are looking after we apply
suitable detector method on the images. The method could be
either worked interactively with an expert or automatically.
The choice is depending on the situation. Automatically
methods usually applied when extrinsic methods are used, or
when geometrically landmarks are used. For region features (a)
we can use segmentation methods. Line features are usually
detected by means of a edge detector like the Canny detector.
When we are looking for point features corner detectors e.g.
the Harris corner detector is suitable.
b) Scale-invariant feature transform (SIFT):
It is an algorithm to detect and describe local features in
images The SIFT features are local and based on the (b)
appearance of the object at particular interest points, and are
invariant to image scale and rotation. They are also robust to
changes in illumination, noise, and minor changes in
viewpoint. In addition to these properties, they are highly
distinctive, relatively easy to extract, allow for correct object
identification with low probability of mismatch and are easy to
match against a (large) database of local features. Object
description by set of SIFT features is also robust to partial (c)
occlusion; as few as 3 SIFT features from an object are
enough to compute its location and pose. Fig. 7 Recognizing panoramas (Brown et al. 2004): (a)
input images with pair wise matches; (b) images grouped
into connected components (panoramas); (c) individual
c) Holography: panoramas registered and blended into stitched
composites.
Holography between pairs of images is then computed using
RANSAC and a probabilistic model is used for verification.

NCAICN-2013, PRMITR,Badnera
327
International Journal Of Computer Science And Applications Vol. 6, No.2, Apr 2013 ISSN: 0974-1011 (Open Access)

Available at: www.researchpublications.org

To recognize panoramas, Brown and Lowe (2003) first find neighbor is found for each feature in the first image, using the
all pair wise image overlaps using a feature-based method and indexing structure to rapidly find candidates, and then
then find connected components in the overlap graph to comparing feature descriptors to find the best match.
“recognize” individual panoramas (Figure 8). The feature- RANSAC is then used to find a set of inlier matches, using a
based matching stage first extracts SIFT feature locations and pairs of matches to hypothesize a similarity motion model that
feature descriptors (Lowe 2004) from all the input images and is then used to count the number of inliers.
then places these in an indexing structure, as described in §4.2.
For each image pair under consideration, the nearest matching
sphere has been imaged. If the case of full 360◦ panoramas,
C.Selection of Projection:
a better choice might be to choose the middle image from
The first choice to be made is how to represent the final the sequence of inputs, or sometimes the first image,
image. If only a few images are stitched together, a natural assuming this contains the object of greatest interest. In all
approach is to select one of the images as the reference and of these cases, having the user control the final view is
to then warp all of the other images into the reference often highly desirable.
coordinate system. The resulting composite is sometimes
called a flat panorama, since the projection onto the final E.Exposure Setting:
surface is still a perspective projection. For larger fields of
Input images need not have the same exposure, or white
view, however, we cannot maintain a flat representation
balance. Together with vignetting (images are darker in the
without excessively stretching pixels near the border of the
corners than in the middle) the required corrections can be
image. (In practice, flat panoramas start to look severely
derived from images with a suitably large overlap area.
distorted once the field of view exceeds 90• or so.) The
usual choice for compositing larger panoramas is to use a
cylindrical (Szeliski 1994, Chen 1995) or spherical IV. IMAGE BLENDING
(Szeliski and Shum 1997) projection. In fact, any surface Once we have registered all of the input images with
used for environment mapping in computer graphics can be respect to each other, we need to decide how to produce the
used, including a cube map that represents the full viewing final stitched (mosaic) image. This involves selecting a
sphere. final compositing surface (flat, cylindrical, spherical, etc.)
and view (reference image). It also involves selecting
which pixels contribute to the final composite and how to
optimally blend these pixels to minimize visible seams,
blur, and ghosting. Once the source pixels have been
mapped onto the final composite surface, we must still
decide how to blend them in order to create an attractive
looking panorama. If all of the images are in perfect
registration and identically exposed, this is an easy problem
(any pixel or combination will do). However, for real
images, visible seams (due to exposure differences),
Fig. 8 Projection of sphere image to screen blurring (due to mis-registration), or ghosting (due to
moving objects) can occur. Creating clean, pleasing
looking panoramas involves both deciding which pixels to
D.Selection of View use and how to weight or blend them. In this section we
describe different image blending techniques.
Once we have chosen the output parameterization, we
still need to determine which part of the scene will be
A. Alpha blending:
centered in the final view. As mentioned above, for a flat
composite, we can choose one of the images as a reference. 1) Alpha Channel:
Often, a reasonable choice is the one that is geometrically The alpha channel is normally used as an opacity
most central. For example, for rotational panoramas channel. If a pixel has a value of 0% in its alpha channel, it
represented as a collection of 3D rotation matrices, we can is fully transparent (and, thus, invisible), whereas a value of
choose the image whose z-axis is closest to the average z- 100% in the alpha channel gives a fully opaque pixel
axis (assuming a reasonable field of view). Alternatively, (traditional digital images). Values between 0% and 100%
we can use the average z-axis (or quaternion, but this is make it possible for pixels to show through a background
trickier) to define the reference rotation matrix. like a glass, an effect not possible with simple binary
For larger (e.g., cylindrical or spherical) panoramas, (transparent or opaque) transparency. It allows easy image
we can still use the same heuristic if a subset of the viewing compositing. Alpha channel values can be expressed as a

NCAICN-2013, PRMITR,Badnera
328
International Journal Of Computer Science And Applications Vol. 6, No.2, Apr 2013 ISSN: 0974-1011 (Open Access)

Available at: www.researchpublications.org

percentage, integer, or real number between 0 and 1 like Figure 9c,9d, while that obtained with the spline is shown
RGB parameters. In graphics, a portion of each pixel's data in Figure 14e.
that is reserved for transparency information. 32-bit
graphics systems contain four channels -- three 8-bit
channels for red, green, and blue RGB and one 8-bit alpha
channel. The alpha channel is really a mask -- it specifies
how the pixel's colors should be merged with another pixel
when the two are overlaid, one on top of the other.
(a) ( b)
B. Feathering:
A better approach to averaging is to weight pixels near
the center of the image more heavily and to down-weight
pixels near the edges. When an image has some cutout
regions, down-weighting pixels near the edges of both (c) (d) (e)
cutouts and edges is preferable. This can be done by Fig9.Pyramidal Blending
computing a distance map or grassfire transform. Weighted V. Image Stiching Application:
averaging with a distance map is often called feathering
and does a reasonable job of blending over exposure Image Stiching has been proposed in the literature as a
differences however, blurring and ghosting can still be means for different applications. The main applications ce
problems.
C. Pyramidal blending: 1. texture synthesis,
Once the seams have been placed and unwanted objects 2. object recognition
removed, we still need to blend the images to compensate 3. medical imaging.
for exposure differences and other mis-alignments. An 4. To produce digital maps
attractive solution to this problem was developed by Burt 5. To produce satellite photos
and Adelson Instead of using a single transition width, a
frequency adaptive width is used by creating a band-pass
(Laplacian) pyramid and making the transition widths a VI Conclusion:
function of the pyramid level. First, each warped image is Image stiching plays a vital role in 21st century which is
converted into a band-pass (Laplacian) pyramid. Next, the emerging with various research programs & advances for
masks associated with each source image are converted into more sophisticated & enjoyable life. Image enhancement
a lowpass (Gaussian) pyramid and used to perform a per- has importance in entertainment. It evolves the
level feathered blend of the band-pass images. Finally, the enhancement of photographs, in medical field. image
composite image is reconstructed by interpolating and processing is used to produce digital maps & satellite
summing all of the pyramid levels (band-pass images). photos , which yields more accurate results. This is a
1) The Laplacian Pyramid: rapidly growing field & has a very wide scope.
REFERENCES
The Gaussian pyramid is a set of low-pass filtered
images. in order to obtain the band-pass images required [1] P.J.Burt, E.H. Andelson, “A multiresolution spline with application
to image mosaics”, ACM transactions on Graphics 2 (4) (1983).
for the multiresolution spline we subtract each level of the
pyramid from the next lowest level. Because these arrays [2] M. Brown and D. G. Lowe, “Recognizing Panorama”,-2003
differ in sample density, it is necessary to interpolate new [3] 3. Richard Szeliski, “Image Alignment and Stitching”-Microsoft
samples between those of a given array before it is Reasearch lab 2005.
subtracted from the next lowest array. Interpolation can be [4] Jiaya jia, Chi-keung Tang, “Image Stitching using Structure
achieved by reversing the REDUCE process. We shall call Deformation”-IEEE transaction on pattern analysis and machine
intelligence, vol 30-2008
this an E XPAND operation. Let G l,k be the image obtained
by expanding Gl, k times. Then, [ 5 ] E. H. Adelson, C. H. Anderson, J. R. Bergen “Pyramid methods in
image processing”-1984
[6] J.-Y. Guillemaut, J. Kilner, J. Starck, A. Hilton, “Dynamic
Feathering: Minimising Blending Artefacts i
An example shows the result of splining two quite [7] n View-Dependent Rendering” - IEEE transaction on Image
different images, an apple and an orange (Figures 9a and processing 2008..
9b). The mosaic obtained without a spline is given in [8] Jiaya Jia, Chi-Keung Tang, “Eliminating Structure and Intensity
Misalignment in Image Stitching” Research Grant Council of Hong
Kong Special Administration Region, China (AOE/E-1999).

NCAICN-2013, PRMITR,Badnera
329
International Journal Of Computer Science And Applications Vol. 6, No.2, Apr 2013 ISSN: 0974-1011 (Open Access)

Available at: www.researchpublications.org

[9] H.-Y. Shum and R. Szeliski. “Construction of panoramic mosaics [10] Xia Wan and C.-C. Jay Kuo” A New Approach to Image Retrieval
with global and local alignment”. International Journal of Computer With Hierarchical Color Clustering” ieee transactions on circuits and
Vision, 36(2):101–130, February 2000. Erratum published July systems for video technology, september 1998
2002,48(2):151-152.

NCAICN-2013, PRMITR,Badnera
330