COL780 / JRL780

Assignment-2: Creating Panoramas using Image Stitching

Deadline: 28 Feb 2023

1 Motivation
Panoramic photography produces images that capture a wider field of view than possible in a single
standard camera shot. By stitching together multiple images rotated around the camera’s vertical
axis, impressive panoramic photos can be constructed showing an entire landscape or scene beyond
what a single image could capture. A Panorama enables photographers to create immersive experi-
ences and showcase sprawling vistas and environments. Furthermore, panoramas have applications
in document scanning, satellite/aerial imaging, and other fields requiring the capture of wide fields
of view.

2 Problem Statement
The objective of this assignment is to create an automated image stitching system that produces
seamless panoramic images from either a set of images with overlapping fields of view or frames
extracted from a short video. The system should take images/video frames as input and combine
them into a single unified panorama spanning the full horizontal field of view captured. The output
is a high-resolution panoramic image stitching the aligned images/frames together into a wide vista.
We describe the task in detail in the next section.

3 Task Description
3.1 Image Preprocessing
The preprocessing stage is needed to enhance the image quality to ensure effective stitching. Fol-
lowing are some commonly used preprocessing techniques in image stitching. You can use some of
the techniques specified below for image preprocessing if required.

• Contrast Adjustment: This is used to bring out details in different parts of the image. It is
particularly important for scenes with varying lighting conditions.
• Noise Reduction: Noise reduction techniques mitigate the impact of artifacts introduced
during image acquisition. This ensures a cleaner input for downstream algorithms.

1
 (a) Image 1 (b) Image 2 (c) Image 3

(d) Image 4 (e) Image 5 (f) Image 6

Figure 1: Input images for Panorama Creation

• Color Correction (if applicable): Correction of color imbalances allows us to maintain a con-
sistent color profile across all input images.

3.2 Image Registration

Image registration is the process of aligning images (estimating the transformation between image
points) captured from different viewpoints so that they can be stitched together. in this section,
you need to implement an algorithm to estimate the homography between two input images. The
following techniques are generally used for Image Registration.

2
 Figure 2: Panorama Image

• Feature Based Methods These involve identifying distinctive features in each image, such as
corners or keypoints, and matching them across images to estimate the transform between
them. Common feature-matching algorithms include SIFT (Scale-Invariant Feature Trans-
form) or ORB (Oriented FAST and Rotated BRIEF).
• Intensity-based Methods: These methods involve optimizing a similarity metric to align pixel
intensities between images. Common techniques include Normalized Cross-Correlation, Mu-
tual Information, and Sum of Squared Differences. This approach is effective when distinct
features are not prominent. Optimization techniques, such as gradient descent, can be used
for intensity-based registration.

Experiment with the above methods to find the one that suits best for the given task. You can
also any combination of the above techniques or some other approach for Image Registration.

3.3 Panorama Creation from Images

Using the Registration Algorithm that you implemented in the previous section, create a Panorama
by stitching the image inputs together. You need to warp the images into a common coordinate
system to align them and blend the images seamlessly together to produce a smooth output. Ex-
periment with different blending methods like Alpha Blending, Laplacian Blending and 2-Band
Blending

3.4 Panorama Creation from Videos

In this section, you will be given a short video of a scene. You need to extract images from the
video and use your Algorithm for Panorama Creation implemented in the previous section.

3.5 Report and Evaluation

Prepare a detailed report describing your approach, challenges faced, and results obtained. Evaluate
the quality of your Image Registration and Panorama Creation.

3
3.6 Bonus
• Optical Flow for Registration Explore the Lucas-Kanade Algorithm for Optical flow to
track the movement of pixels between consecutive frames for Image registration.
• Graph Cut Blending Explore using Graph Cuts for Image Blending.

4 Submission Guidelines
• Dataset: The dataset for this assignment can be found here. You have been provided with 3
panoramic scenes with 6 images each and 1 video capture file to test your Panorama Creation
Algorithms. We have a separate held-out test dataset which is similar to the one given to
you. We will test the performance of your algorithms on that dataset.
• The entire code must be submitted in the form of python files, along with a main.py which
will be executed in two parts during evaluation/demo as follows:
python3 main.py <part id> <input path> <output path>

There are two tasks. part id will be either 1 or 2 depending on the task. You need to write
the panoramic image to the path specified by <output path>.

– For the first task, <input path> will contain the location of the folder containing the
images.

python3 main.py 1 <img dir> <output path>

– For the second part, <input path> will contain the location of the video file.

python3 main.py 2 <video path> <output path>

4.1 Instructions
• You can use OpenCV functions to read and write images. You CANNOT use other inbuilt
image processing algorithms from OpenCV unless mentioned in the assignment description.
Among other third-party libraries, only Numpy, Matplotlib, and PIL functions are allowed.

• Submit the code in the form of Python files only. You should have a main.py file which will
be executed during the demo.
• You are required to submit the code, data, and a detailed report for this assignment and zip
the contents with file name ENTRY NUMBER.zip on Moodle.

• This is an individual assignment. Mention the names of all the students you have discussed
within the report. Any sort of plagiarism will lead to a zero in the assignment straightaway
and stricter punishment depending on the case.