1|Page

DEPARTMENT OF MECHATRONICS ENGINEERING

UNIVERSITY OF ENGINEERING & TECHNOLOGY, PESHAWAR
Digital Image Processing, 7th Semester

Student Name: Imad Ahmad

Reg No: 20PWMCT0732

Subject: Digital Image Processing

Assignment No 3

Lab Rubrics:
BelowBasic (1)
Criteria Excellent (4) Proficient (3) Basic (2) Student’s
Score
Report is mostly
Report is
as per the
Report is as per disorganized Sections/Steps
guidelines and
the guidelines. and follows are not
To organize the most
All sections/steps some ordered and
lab report and
sections/steps are ordered well guidelines Report is not
practice
are clearly but requires but most of as per the
th e writing skills organized in a minor the guidelines guidelines
as per the logical order. improvements. are missing
guidelines

The report
completely The report
discusses the The report The report is
discusses the
required task discusses the totally irrelevant
required task in
lab work but to the lab work
own words with
To discuss the have
some relevant
actual
additional irrelevant
task information information

Calculations Calculations and Most data Calculations and

and data analysis and data analyses of
were performed observations
lab were missing
data analyses accurately, but were
To perform minor errors recorded
were made both adequately,
calculations and were performed
in calculations but with
data analysis clearly, and in applying several
concisely, and correct units significant

Course Instructor: Dr. Riaz Akbar Shah

 accurately, with errors
correct
or omissions.
units.

Graphs, if Graphs, if Graphs, if Major

necessary, were necessary, were necessary, components of
drawn drawn were drawn lab
To present results accurately and adequately but were
in the form of neatly and were inadequately. missing
graphs clearly labelled.

Course Instructor: Dr. Riaz Akbar Shah

2|Page

Table of Contents
Contents
Objectives............................................................................................................................................................

Theory.................................................................................................................................................................

Types of Robotic Arm..........................................................................................................................................

Lab-Volt Automation 5250-A0 Servo Robot Training System..............................................................................

ProArm RS2200 Robot by Marcraft Int ................................................................................................................

Mobile Robot........................................................................................................................................................

The Khepera IV Mobile Robot.............................................................................................................................

Unimate PUMA 500..............................................................................................................................................

Pick and Place Robotic Arm..................................................................................................................................

Conclusion..........................................................................................................................................................

References..........................................................................................................................................................

Course Instructor: Dr. Riaz Akbar Shah

3|Page

What is Image Segmentation?

Image segmentation, a fundamental concept in computer vision, is dividing a digital picture into several
segments, which are sometimes referred to as image regions or objects. This procedure converts a picture into
a more relevant and easier-to-understand representation. Image segmentation classifies pixels so that those
with the same label have similar properties. This method is useful for finding items and boundaries inside
photos. In medical imaging, for example, segmentation may generate 3D reconstructions from CT images
using geometry reconstruction methods.

Types of Image Segmentation

Semantic Segmentation: This method determines the class to which each pixel belongs. In a picture
containing numerous individuals, for example, all pixels linked with humans will have the same class label,
however background pixels would be classed differently.

Instance Segmentation: Every pixel is associated with a specific instance of the item. It is concerned
with distinguishing different things of interest in a picture. In a photograph with numerous people, for
example, each person would be segmented as a separate item.

Course Instructor: Dr. Riaz Akbar Shah

4|Page

Panoptic Segmentation: Panoptic segmentation, a hybrid of semantic and instance segmentation,

determines the class to which each pixel belongs while discriminating between distinct instances of the same
class.

What is Panoptic Segmentation?

The term "panoptic" refers to everything seen in a single perspective. Panoptic segmentation in computer
vision provides a holistic method to segmentation, effortlessly combining the capabilities of instance and
semantic segmentation.

Panoptic segmentation is a complex approach that classifies every pixel in an image based on its class label
while identifying the exact instance of that class to which it belongs. In a picture with many automobiles, for
example, panoptic segmentation would recognize and differentiate each car, generating a unique instance ID
for each.

The complete breadth of this approach distinguishes it from other segmentation jobs. While semantic
segmentation allocates pixels to classes without discriminating between specific instances, instance
segmentation does so. Every pixel in an image processed using panoptic segmentation would have two
associated values: a label indicating its class and an instance number. Pixels that belong to "stuff" regions,
which are harder to quantify (like the sky or pavement), might have an instance number reflecting that
categorization or none at all. In contrast, pixels belonging to "things" (countable objects like cars or people)
would have unique instance IDs.

This enhanced segmentation approach might be used in a variety of industries, including medical imaging,
driverless cars, and digital image processing. Because of its capacity to give a thorough comprehension of
pictures, it is a crucial tool in the ever-changing field of computer vision.

Course Instructor: Dr. Riaz Akbar Shah

5|Page

Working Mechanism:
Panoptic segmentation has emerged as a game changer in computer vision. It's a hybrid strategy that combines
the best of semantic and instance segmentation. Whereas semantic segmentation categorizes each pixel,
instance segmentation recognizes specific object instances. Panoptic segmentation, on the other hand,
performs both: it classifies every pixel and provides a unique instance ID to distinct objects.

The Efficient Panoptic Segmentation (EfficientPS) approach is one of the most advanced in panoptic
segmentation. Deep learning and neural networks are used in this method to generate high-quality
segmentation results. EfficientPS is intended to be both computationally efficient and effective in terms of
segmentation quality. It processes input photos and generates segmentation masks using feature pyramid
networks and convolutional layers. The COCO dataset is also used for training and validation, ensuring that
the models are exposed to a wide range of pictures and settings.

The benefit of panoptic segmentation, particularly approaches like EfficientPS, is that it may give a precise,
pixel-level comprehension of pictures. This is extremely useful in real-world applications such as driverless
cars, where recognizing the category (road, pedestrian, vehicle) and the environment is critical.

Key Components of Panoptic Segmentation

Consider a painter who not only recognizes every object in a scene but also colors inside the lines, ensuring
that every detail is highlighted. In the area of computer vision, this is the magic of panoptic segmentation.
Understanding its major components allows us to understand how it successfully delineates and classifies
every pixel in a picture, ensuring coherence and distinction.

Fully Convolutional Network (FCN) and Mask R-CNN

FCNs (Fully Convolutional Networks) have emerged as a critical component in panoptic segmentation. FCN's
strength is its ability to analyze pictures of varied sizes and provide outputs of equivalent sizes. By
categorizing each pixel into a semantic label, this network collects patterns from an infinite number of things,
Course Instructor: Dr. Riaz Akbar Shah
6|Page

such as the sky or highways. It is intended to function from pixel to pixel, providing a thorough, spatially
dense forecast.

Fully Convolutional Neural Networks

Mask R-CNN, an extension of the Faster R-CNN, on the other hand, is critical in distinguishing countable
items. While Faster R-CNN is good at recognizing bounding boxes, Mask R-CNN adds a parallel branch to
predict an object mask. This implies that Mask R-CNN recognizes and builds a high-quality segmentation
mask for each identified item. Because of its dual feature, it is an important tool for jobs that need object
identification and pixel-level segmentation, such as detecting and separating individual automobiles in a
traffic scene.

Mask RCNN Architecture

FCN and Mask R-CNN form the backbone of panoptic segmentation, ensuring that every pixel in an image is
accurately classified and, if applicable, associated with a unique instance ID.

Practical Applications of Panoptic Segmentation

Medical Imaging
In medical imaging, panoptic segmentation has achieved considerable advances. By combining the power of
semantic and instance segmentation, panoptic segmentation provides a rich and comprehensive perspective of
medical pictures. This is especially useful in tumor cell identification, as the model detects the presence of
tumor cells while also distinguishing between individual cells. Such accuracy is required for precise
diagnoses, which allows medical practitioners to develop more effective treatment strategies. Using datasets
like COCO and Cityscapes in conjunction with deep learning algorithms guarantees that segmentation models
are trained on high-quality data, which improves their accuracy in medical diagnostics.

Autonomous Vehicles
Another arena in which panoptic segmentation excels is that of autonomous cars. Understanding the
environment is critical for self-driving automobiles. Panoptic segmentation helps with this by offering a pixel-
level comprehension of the environment. It is critical in estimating distance to object, allowing the vehicle to
make intelligent judgments in real time. Panoptic segmentation provides safer navigation for autonomous cars
by discriminating between countable items (such as pedestrians and other vehicles) and uncountable objects
(such as highways and sky).

Digital Image Processing

Modern smartphone cameras are technological marvels, and panoptic segmentation expands their possibilities.
Portrait mode, bokeh mode, and auto-focus all use picture segmentation to distinguish between the subject and
the backdrop. This enables the generation of high-quality pictures with depth effects. The combination of

Course Instructor: Dr. Riaz Akbar Shah

7|Page

semantic and instance segmentation means that the camera can recognize and focus on the subject while
blurring away the backdrop, producing spectacular images.

Course Instructor: Dr. Riaz Akbar Shah