Developing Image Processing and

Computer Vision Systems Using MATLAB

and Simulink

Vinod Geo Thomas

Senior Training Engineer
MathWorks

© 2013 The MathWorks, Inc.1

2
Why use MATLAB for Image and Video
Processing?

Read/Write Visualize and

Connect directly
many image explore images
to cameras
formats interactively

Block process
Use a large Quickly build
large images to
library of inbuilt custom IP
avoid memory
functions algorithms
issues

Process images
faster with
multiple cores
and clusters

3
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

4
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

5
Underexposure Photo

Improper
Camera
Settings??

6
White Balanced Photo

Much
Better!

7
Image Acquisition Toolbox

Configure, acquire, and preview live video data using a

graphical interface

Acquire images and

video directly into
MATLAB and Simulink

Synchronize
multimodal
devices

Configure
device
properties 8
Image Acquisition Toolbox Hardware Support

9
 From Sensor Data to Image
(Converting raw data from image sensor to color adjusted RGB)

Image Sensor

Bayer pattern
Color Filter
Sensor Arrays

Raw Image RGB Color Image

10
 Digital Camera Pipeline

Noise
Reduction Demosaic
Denoise electrical , Tone
and physical noise. Interpolate raw
Mapping Color
image data to
Adjust
RGB Calibrate sensor
RGB values to •White balance
reference •Gamma
Correction

Example of digital camera pipeline

(Processes may be different depends on sensors) 11
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

12
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

13
Examples of Computer Vision with MATLAB

14
Computer Vision System Toolbox

Design and simulate computer vision

and video processing systems

 Feature detection, extraction and

matching
 Feature-based registration
 Object detection and tracking
 Stereo vision
 Video processing
 Motion estimation
 Video display and graphics
15
Object Detection and Tracking

Face Detection

KLT Tracking

People Detection

16
Categorical Object Detection

 How do we detect a category or “general type”

– Faces
– People
– Cars
– …

17
Category Detection = Features + Machine
Learning

Typical Machine
Typical Features Machine Learning
learning classifiers
• Histogram of • Requires input • SVM
Gradients (HOG) data and known
responses • Adaboost
• Haar like
Features • Builds a model to • Cascade of
predict Adaboost
• Local binary responses to
patterns new data
• K-nearest
neighbour

18
Histogram of Oriented Gradients (HOG)

 Descriptor to characterize
local object appearance
 Compute gradients in image
using [-1, 0, 1] mask with no
smoothing, divide it into cells
 Compute histogram of
gradient orientations on each
cell
 Group cells into overlapping
blocks, normalize vector of
histogram values
 Slide over all relevant
windows/regions

19
Support Vector Machines (SVMs)

 Type of supervised learning

 Represent data as features in

‘feature-space’

 Linear SVM is a classifier that

maximizes distance between two
classes (margin) in feature-space

 ‘Trained’ SVM accepts test data

20
Demo: People Detector

 vision.PeopleDetector detects upright people

 Uses HOG features and trained SVM classifier

21
Demo: Viola-Jones Face Detection

 Algorithm details
– Haar-like features
– Gentle Adaboost classifiers
for feature selection
– Cascading of classifiers to
quickly weed out negative
candidates

22
Cascade of Classifiers in
CascadeObjectDetector

Each stage of cascade is Gentle Adaboost,

an ensemble of weak learners

Each stage rejects negative samples using

a weighted vote of these weak learners

The samples not rejected are passed to the

next stage

Positive detection means the sample

passed all stages of the cascade

23
Object Tracking Workflow

Detection

First, detect a face

(using
Viola Jones
Algorithm)
Then, detect features using
Minimum eigenvalues (corners)
Tracking

Track Features using

vision.PointTracker 24
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

25
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

26
Parallel Computing Products

27
 Parallel Computing enables you to …

Larger Compute Pool Larger Memory Pool

Speed up Computations Work with Large Data

11 26 41

12 27 42

13 28 43

14 29 44

15 30 45

16 31 46

17 32 47

17 33 48

19 34 49

20 35 50

21 36 51

22 37 52

28
GPU Support with Parallel Computing Toolbox

 NVIDIA GPUs with compute capability 1.3 or greater

– Includes Tesla 10-series
and 20-series products
(e.g., NVIDIA Tesla C2075 GPU:
448 processors, 6 GB memory)
– http://www.nvidia.com/object/cuda_gpus.html

 GPU enabled Image Processing Toolbox functions:

imrotate(), imfilter(), imdilate(), imerode(), imopen(),
imclose(), imtophat(), imbothat(), imshow(), padarray(),
bwlookup()

Example: imfilter : 37x37 Filter on 3840 x 5120 pixels image

13 seconds (CPU Only )  1 seconds (GPU: Tesla C2050)
29
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

30
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

31
Targeting DSPs and FPGAs

MATLAB and Simulink

• Algorithm development
• Debugging and profiling
• System design
Fixed-Point Modeling

Generate code Verify design

Link to Embedded Software

Integrated Development Environment

• Compiler DSP/
• Build automation tools FPGA
• Debugger
32
Fixed-Point Analysis

 Convert floating point to optimized fixed-point models

– Automatic tracking of signal range (also intermediate quantities)
– Word / Fraction lengths recommendation Automatically
identify and solve
 Bit-true models in the same environment fixed-point issues

33
Automatic Translation of MATLAB to C
MATLAB Coder

te
ra
ite

With MATLAB Coder, design engineers can

• Maintain one design in MATLAB

• Design faster and get to C/C++ quickly
• Test more systematically and frequently
• Spend more time improving algorithms in MATLAB

34
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

35
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

36
Simulink: The Simulation Platform

 Hierarchical block diagram design

and simulation tool
 Built-in notion of time
 Visualize Signals
 Co-develop with C / M / HDL code
 Integrated with MATLAB

37
 Simulink System Design Environment

Input from
Hardware IDE Connection
(Ex: Texas Instruments,
Analog Devices, Green
Hills, Altium, Eclipse)

EDA Connection
(Ex: Xilinx, Altera、Mentor
Graphics, Cadence)
Co-Simulation
with CPU/DSP

Co-Simulation
with FPGA

38
Run on Target Hardware
What is it?

 A Simulink feature that generates an executable

application from a model and runs it on supported target
hardware
 Available from the model’s Tools menu
Tools  Run on Target Hardware

 Includes a Target Installer to select and install target

hardware support packages

39
BeagleBoard
 An open-source single-board “laptop”
 Compatible with USB devices like keyboard, mouse,
and web cam
 Stereo audio in and out

40
Installing a Target Hardware Support Package

41
Target Hardware Support Packages

 Target hardware support packages provide a collection of

software components for the specified target hardware

42
Workflow
How does it work?

1. Connect target hardware to host computer

2. Install Target Hardware Support Package

3. Create a model

4. Prepare to Run

5. Run

43
AGENDA

Image Acquisition DEMO

Algorithm Development using Image DEMO

Processing and Computer Vision System
TB

DEMO
Working with Large Images

DEMO
Targeting DSP’s

Simulink Workflow DEMO

44
What you saw today

 A typical workflow for building a system using the Image

Processing and Computer Vision System Toolbox.

45
 Training Services
Exploit the full potential of MathWorks products

Flexible delivery options:

 Public training available in several cities
 Onsite training with standard or
customized courses
 Web-based training with live, interactive
instructor-led courses

More than 30 course offerings:

 Introductory and intermediate training on MATLAB, Simulink,
Stateflow, code generation, and Polyspace products
 Specialized courses in control design, signal processing, parallel computing,
code generation, communications, financial analysis,
and other areas

Email: [email protected] URL: http://www.mathworks.in/services/training Phone: 080-6632-600046

 Scheduled Public Training for Sep–Dec 2013
Course Name Location Training dates
Statistical Methods in MATLAB Bangalore 02- 03 Sep 2013
MATLAB based Optimization Techniques Bangalore 04 Sep 2013
Physical Modeling of Multi-Domain Systems using Simscape Bangalore 05 Sep 2013
Delhi 23-25 Sep 2013
Pune 07-09 Oct 2013
MATLAB Fundamentals Bangalore 21-23 Oct 2013
Web based 05- 07 Nov 2013
Chennai 09-11 Dec 2013
Delhi 26-27 Sep 2013
Pune 10-11 Oct 2013
Simulink for System and Algorithm Modeling Bangalore 24-25 Oct 2013
Web based 12-13 Nov 2013
Chennai 12-13 Dec 2013
MATLAB Programming Techniques Bangalore 18-19 Nov 2013
MATLAB for Data Processing and Visualization Bangalore 20 Nov 2013
MATLAB for Building Graphical User Interface Bangalore 21 Nov 2013
Generating HDL Code from Simulink Bangalore 28-29 Nov 2013
Email: [email protected] URL: http://www.mathworks.in/services/training Phone: 080-6632-600047
 MathWorks Certification Program- for the first
time in India!

MathWorks Certified MATLAB Associate Exam

Why certification?
 Validates proficiency with MATLAB
 Can help accelerate professional growth
 Can help increase productivity and project success and thereby
prove to be a strategic investment

 Certification exam administered in English at MathWorks facilities

in Bangalore on Nov 27,2013

Email: [email protected] URL: http://www.mathworks.in/services/training Phone: 080-6632-600048

Thank you!

© 2013 The MathWorks, Inc. MATLAB and Simulink are registered trademarks
of The MathWorks, Inc. See www.mathworks.com/trademarks for a list of
additional trademarks. Other product or brand names may be trademarks or
registered trademarks of their respective holders.

49