Scribd
Upload
42 views

Object Detection Using Depth and Color Images

The document discusses using depth and color images for object detection. It explores two approaches: 1) Training separate models on each image type and combining them, which failed. 2) Training a neural network on RGB images and HHA depth images separately then combining, which showed better results. The goal is detecting a custom object (camera) using both image types. Future work includes better integrating depth and color images to improve accuracy for real-world use.

Uploaded by

Aadith Thillai Arasu S
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
42 views

Object Detection Using Depth and Color Images

The document discusses using depth and color images for object detection. It explores two approaches: 1) Training separate models on each image type and combining them, which failed. 2) Training a neural network on RGB images and HHA depth images separately then combining, which showed better results. The goal is detecting a custom object (camera) using both image types. Future work includes better integrating depth and color images to improve accuracy for real-world use.

Uploaded by

Aadith Thillai Arasu S
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 12

Object Detection Using

Depth and Color Images


Team: CrazyOnes
Problem
● As use of depth sensors has increased widely due to the application in
various gaming consoles and autonomous car, it is important to understand
the use of depth data.
● Depth data helps in getting perspective about relative distances between
objects present in the image along with adding one extra dimension to the
available information.
● We are analysing how this depth data can be used for increasing the
accuracy of existing object detection models using machine and deep
learning.
Existing Object Detection
● Mask R-CNN - Detection with Segmentation
● YOLOv2 - Faster and Stronger
● MobileNet by Google - Lightweight

Although this models provide state-of-the-art accuracy


with speed on real-world scenarios:

● Accuracy for pre-trained objects is hard to achieve


on custom objects.
● No use of depth sensors information
● Object View matters a lot...
Approach 1
● We have 2 images for 1 particular scene, let’s train them separately. Then
combine the models to achieve a better accuracy in detection.

● Used YOLO-v3 for default object detection for both the images but...
Approach 1

This approach fails miserably as YOLO fails to extract any information from the
depth image of data. Faster R-CNN also fails on this depth images taken from the
NYU Depth Dataset V2.
Approach 2
● Train neural network on RGB image as well as separately on HHA format of
the depth image

Depth Image HHA Image


Hackathon Goal
● Training our Model on one custom object using RGB and depth images and
then detecting it in test images with both the types of data
● Selected Object: Camera
Training Dataset
● Open Images Dataset V4 + by Google contaning 600 categories
● Microsoft Kinect RGB-D dataset
● Sample:
Architecture
Demonstration
Future Work
● Integrating HHA image with the RGB one for better model accuracy
● Use of fine-tuned YOLO so that it can be used in real-world live scenarios
Thank You

You might also like