® ®

Using MATLAB® and Simulink® for

Image and Video Processing
Sofia Mosesson

© 2007 The MathWorks, Inc.

Application Engineer
 ® ®

Agenda

9:00 a.m. Using MATLAB® for Image Processing

10:30 a.m. Break

10:45 a.m. Using Simulink® for Video Processing

11:45 a.m. System Design and Implementation

12:00 a.m. MathWorks Overview and Wrap-up

2
 ® ®

Some Image Processing Applications

Automotive Defense Biometrics Medical diagnosis

Electronics Microscopy Semiconductors Drug discovery

3
 ® ®

Common Challenges

Gather data from multiple sources

Closely inspect images to gain insight

Automate standard image analysis tasks

Create new algorithms or modify existing ones

Share results and generate reports

Create applications and deliver tools

4
 ® ®

Using MATLAB for Image Processing

Image processing Image analysis

Visualization
Algorithm
development

Image acquisition
Idea sharing &
reporting

Application
Data access deployment

5
 ® ®

Demo: Lane Marking Detection

Identify lane markings in a sample image using two techniques

6
 ® ®

The language of technical

computing

Data access

Visualization

Data analysis

Algorithm development

Sharing results

7
 ® ®

Go Farther with MATLAB and Toolboxes

Signal Processing
Toolbox Mapping Toolbox
Statistics Toolbox

Image Processing
Database Toolbox Toolbox

Image Acquisition
Toolbox MATLAB Compiler

13
 ® ®

Image Processing Toolbox

Perform image processing, analysis, visualization, and

algorithm development

Image visualization

Image pre- and postprocessing

Image analysis

Spatial transformations

Image registration

Color image processing

14
 ® ®

Image Visualization

Graphical tools

Overview

Pixel region detail

Image information

Contrast adjustment

Pan and zoom

Open and modular

Easy to customize

Easy to extend

15
 ® ®

Image Pre- and Post-Processing

Enhancement

Noise removal

Deblurring

Region-based processing

Transforms (FFT, DCT,…)

16
 ® ®

Image Analysis

Edge detection

Segmentation

Morphological operators

Image statistics

Boundary tracing

Region properties

Texture analysis

Hough transform

17
 ® ®

Spatial Transformations and

Image Registration

Supports many types of spatial
transformations

Affine, polynomial, user-defined,
projective, piecewise-linear

Apply multi-dimensional spatial
transformations with a flexible
engine

Perform control point-based image
registration

18
 ® ®

Color Image Processing with MATLAB

Convert between color spaces

Create device-independent image
processing algorithms

Manage ICC color profiles

Quantitatively measure color accuracy

Perform soft-proofing

Supported Color Spaces:

• sRGB
• CIELAB, CIEXYZ
• uvL, HSV, YCrCb, L*ch

19
 ® ®

Return to Lane Marking Demo

Directory browser

Workspace browser

Import wizard

imtool

imcrop

Cell mode execution

Thresholding

Morphological operators

Region properties

Hough transform

20
 ® ®

Image Acquisition Toolbox

Stream images and video

directly into MATLAB

Configure device properties

Background image acquisition

Live video previewing

Support for multiple hardware
vendors

21
 ® ®

Image Acquisition Toolbox:

Supported Hardware

Frame grabbers

DALSA Coreco

Data Translation

Matrox Imaging

Digital cameras

Hamamatsu

QImaging

DCAM-compatible cameras

Windows video devices (WDM or VFW)

Third-party adaptors

22
 ® ®

Capturing Video from a Webcam

>> vidobj = videoinput('winvideo'); Create

Create
...
>> inspect(vidobj) Configure
Configure
...
>> start(vidobj) Connect
Connect
...
>> data = getdata(vidobj); Access
Access
...
>> delete(vidobj);
>> clear(vidobj); } Clean-up
Clean-up

Demonstration next
23
 ® ®

Designing a Graphical User Interface with

MATLAB

GUIDE (>> guide)

Design and edit GUI

Add buttons, pull-down menus,
list boxes, MATLAB plots, and
ActiveX components

Add callbacks

Generate MATLAB code

Design it yourself in code

Demonstration next
24
 ® ®

Application Deployment with MATLAB

MATLAB Compiler

MATLAB® MATLAB MATLAB

Builder for Builder for Builder for
.NET Java Excel

.DLL
.EXE COM .NET Java
.LIB www Excel
Add-in

25
 ® ®

Application Deployment Workflow

MATLAB End-
End-User
Desktop Machine

27
 ® ®

Case Study: MetsGUI

Microscopy application in MATLAB to find metastasized

tissue (mets) in cancerous specimens

Use GUIDE to build or modify GUI

28
 ® ®

Why use MATLAB for Image Processing?

Complete environment

Verified and trusted algorithms

Faster than hand-coding

algorithms in C/C++

Open and flexible architecture

enables a customized workflow

29
 ® ®

Customer Profile: Given Imaging

The Challenge

To find an alternative to the endoscope
and other invasive gastrointestinal
imaging procedures

The Solution

Use MATLAB and MATLAB toolboxes to
implement and develop an alternative The M2A capsule, shown next to a US quarter
diagnostic tool

The Results “MATLAB® and the MATLAB Image

Fast, efficient development Processing Toolbox enabled us to cut
down development time by several

Easy access to precise
orders of magnitude."
diagnostic information Dr. Arkady Glukhovsky

As of 2007, shipped 500,000 PillCams Given Imaging

30
 ® ®

From a Leading Textbook Author ...

"I have used a number of commercial image

processing packages over the years, and
prefer The MathWorks Image Processing
Toolbox for several reasons: the wide
variety of functions it provides, the user’s
ability to write additional functions with
minimal effort, the quality of the software,
and the high level of support."

Rafael C. Gonzalez
Professor Emeritus
University of Tennessee

31
 ® ®

Agenda

9:00 a.m. Using MATLAB® for Image Processing

10:30 a.m. Break

10:45 a.m. Using Simulink® for Video Processing

11:45 a.m. System Design and Implementation

12:00 a.m. MathWorks Overview and Wrap-up

32
 ® ®

Agenda

9:00 a.m. Using MATLAB® for Image Processing

10:30 a.m. Break

10:45 a.m. Using Simulink® for Video Processing

11:45 a.m. System Design and Implementation

12:00 a.m. MathWorks Overview and Wrap-up

33
 ® ®

Tools for a Complete Workflow

Research and System Implementation

development design and test

Data
Data
Data
Data Environment Rapid
Rapid Verification,
Verification,
Analysis
Analysis&&
Access
Access effects Prototyping
Prototyping HIL
HILTest
Test
Visualization
Visualization
Embedded
Algorithm
Algorithm algorithms Code
Design
Design Code
Generation
Generation
System Embedded
Mathematical Embedded
Mathematical components Hardware
Modeling Hardware
Modeling
System
SystemModeling,
Modeling, Embedded
Embedded
Simulation
Simulationand
and Software
Software
Partitioning
Partitioning

34
 ® ®

Some Video Processing Applications

Unmanned vehicle Automotive Video
surveillance active safety conferencing

Security Office
cameras equipment Electronics

35
 ® ®

Challenges of Embedded Design

Extreme computational demands of video

Real-time and concurrency requirements

Manage trade-offs for embedded implementation

Power, performance, size, and cost

Target hardware options: RISC processors, DSPs, FPGAs, etc

Testing and validation without hardware

36
 ® ®

Model Based Design

37
 ® ®

Model-Based Design

Requirements and Design Implementation Test and

specifications verification

Model elaboration

Executable Design with Automatic Continuous

specifications simulation code verification
generation
38
 ® ®

Using Simulink for Video Processing

Automatically
generate C-code
Analyze system trade-offs Target embedded
devices

Implicit timing and Continuous

concurrency verification

Rapidly prototype ideas

39
 ® ®

What is Simulink?

Simulation, modeling, and design tool

GUI-based block diagram environment

Platform for Model-Based Design

40
 ® ®

Simulink Key Features

Hierarchical, component-based modeling

MATLAB® integration

Extensive library of predefined blocks

Application-specific libraries available

Open Application Program Interface (API)

41
 ® ®

Video and Image Processing Blockset

Model, simulate, implement, and verify

real-time video and imaging systems

Includes over 60 components and 100’s
of algorithms

Ideal for implementation of embedded
systems

Streaming Detection, Tracking, Background

video in/out thresholding counting estimation
42
 ® ®

Video and Image Processing Blockset

Analysis and enhancement

Conversions

Filtering

Geometric transforms

Morphological operations

Sinks

Sources

Statistics

Text and graphics

Transforms

Utilities
43
 ® ®

Demo: Edge Detection

Build a basic model in Simulink with the Video and Image

Processing Blockset

45
 ® ®

Demo: Edge Detection

Easy to import streaming
video into the simulation
Handy viewers for
inspecting video at any
point in the algorithm

Flexible block implementing

several edge detection methods
46
 ® ®

Demo: Edge Detection – Live Video

Build a basic model in Simulink with the Video and Image

Processing Blockset

47
 ® ®

Case Study: Counting Cells

Use a combination of basic morphological operators and blob

analysis to count cells in a video

49
 ® ®

Case Study: Video Stabilization

Stabilize video from a moving camera

50
 ® ®

Case Study: Lane Departure Warning

System

Detect lane markings in real time

Calculate distance from lane markings to determine lane
departure

51
 ® ®

Why use Simulink for Video Processing?

Modeling and simulation

Easily process streaming video

Visualize results at each step in your model

Move quickly from algorithm ideas to accurate system behavior

Extensive block library

Hundreds of blocks supporting video and image processing

No need to recreate standard algorithms

Analyze system tradeoffs

Perform “what-if” analyses

Compare algorithms against specifications for numerical accuracy,
computational complexity, and more

52
 ® ®

Agenda

9:00 a.m. Using MATLAB® for Image Processing

10:30 a.m. Break

10:45 a.m. Using Simulink® for Video Processing

11:45 a.m. System Design and Implementation

12:00 a.m. MathWorks Overview and Wrap-up

53
 ® ®

Model-Based Design with Simulink

How is this useful for video?

 Implicit timing and concurrency
 Extensive algorithm library

Fixed-point modeling

Automatic code generation

Rapid prototyping

54
 ® ®

Fixed-Point Modeling*

Avoid inaccurate results due to

finite word effects

Access tools built directly into
block interfaces

Easily change parameters to
model the impact of rounding,
overflow, and scaling

* Requires Simulink® Fixed Point for

integer and fixed-point data types

55
 ® ®

Targeting Embedded Microprocessors and DSPs

• Real-Time Workshop
• Real-Time Workshop
Embedded Coder

C-compiler
IDE
C-code (Integrated
Design
Environment)
Specific Board
Compiler, Linker,
and Loader
Specific Peripheral
S/W Drivers

57
 ® ®

Link for Code Composer Studio™ and

Target for TI C6000™

Link for Code Composer Studio™

Co-simulation, project automation,
debugging, and verification for
C2000, C5000, and C6000 DSPs

Target for TI C6000™

Includes target-optimized signal processing blocks

Supports on-chip peripherals and evaluation boards

DSPs: TI DM64x, C64x, C62x, and C67x families

Key benefit: rapid prototyping and evaluation with hardware in
the loop

58
 ® ®

Generating C code from a Simulink Model

Using Embedded Target for TI C6000 to prototype a Simulink-based
design on the DM642 platform

59
 ® ®

Co-Simulation and HDL Generation

MathWorks and Third-party Products

Synthesizable VHDL and Verilog from Simulink

NEW
Simulink HDL Coder (MathWorks)

Filter Design HDL Coder (MathWorks)

System Generator for DSP (Xilinx)

DSP Builder (Altera)

Synplify DSP (Synplicity)
Co-simulation of VHDL and Verilog in Simulink

Link for ModelSim (MathWorks) with ModelSim (Mentor)
NEW
Link for Incisive (MathWorks) with Incisive (Cadence)

ModelSim Xilinx Edition (Xilinx)

ModelSim Altera Edition (Altera)

66
 ® ®

The Value of Model-Based Design

Model-Based Design Innovation

Rapid design iterations

Executable specification

“What-if” studies

Design with simulation
Unique features and differentiators

Implementation through code Quality
generation
Reduce design errors

Continuous test and verification
Minimize hand coding errors

Unambiguous communication internally and
externally
Cost

Reduce expensive physical prototypes

Reduce re-work

Automate testing
Time-to-market

Get the product right the first time

67
 ® ®

Key Benefit of Continuous Test and

Verification
Where Errors Are Introduced... and Detected “Each delay in the
detection and correction
70% of a design error makes
60%
60%
it an order of magnitude
55%

50%
more expensive to fix.”
Clive Maxfield and Kuhoo Goyal
40%
“EDA: Where Electronics Begins”
30% TechBites Interactive, October 1, 2001
22% ISBN: 0971406308
21% 15%
20% 8%

10% 12%

0% 7% Detected

Spec Introduced
Design
Implement
Test
Source: “Migration from Simulation to
Verification with ModelSim” by Paul Yanik.
EDA Tech Forum, 2004 Mar 11, Newton MA
68
 ® ®

Doheny Eye Institute Enables The Blind

To See With MathWorks Tools
The Challenge

To develop a retinal prosthetics proof-of-
concept prototype that interfaces with a
permanent micro-electronic retinal implant

The Solution
Example of the Prototype

Used Simulink, Blocksets, Real-Time
Workshop, and Target for TI C6000 to build “We are working on real-time image
and validate a rapid prototype on a DM642 processing with the TI DM642
processor as the target. The Video
The Results and Image Processing Blockset
makes the task of creating our design

Completed phase II of research project and working prototypes much
ahead of schedule simpler.”

Currently in trials with patients though
Dr. James Weiland
Doheny Eye Institute at the University Director, Intraocular
of Southern California Retinal Prosthesis Lab

69
 ® ®

Return on Investment Study

2005 study by International Business Strategies

Analysis of five cases of using Simulink for electronics system
design

Case Study #4: Video processor design

Highlights:

Increased productivity of design engineers

Direct savings in design costs

Improved time-to-market

Increased IP reuse across design iterations and derivative projects

White paper available at www.mathworks.com/roi

70
 ® ®

Video and Imaging Design Products

Base (required):

MATLAB, Signal Processing Toolbox, Image Processing Toolbox

Simulink, Signal Processing Blockset, Video and Image Processing
Blockset

Fixed Point Modeling:

Fixed-Point Toolbox, Simulink Fixed Point, Simulink Accelerator

Implementation:

Real-Time Workshop

Real-Time Workshop Embedded Coder

Link for Code Composer Studio

Target for TI C6000™

Link for ModelSim

Partner products (Altera, Xilinx, Synplify, TI, Mentor Graphics, etc)

71
 ® ®

Agenda

9:00 a.m. Using MATLAB® for Image Processing

10:30 a.m. Break

10:45 a.m. Using Simulink® for Video Processing

11:45 a.m. System Design and Implementation

12:00 a.m. MathWorks Overview and Wrap-up

72
 ® ®

The MathWorks at a Glance

Headquarters:
Natick, Massachusetts USA

USA:
California, Michigan,
Washington DC, Texas

Europe:
UK, France, Germany,
Switzerland, Italy,
Spain, the Netherlands, Sweden

Asia-Pacific:
Korea, Australia

Worldwide training
and consulting Earth’s topography on an equidistant
cylindrical projection, created with the
Distributors in 25 countries MATLAB Mapping Toolbox

73
 ® ®

The MathWorks Today

■ Revenues exceeding $400M in 2006

■ Privately held
■ Over 1,600 employees worldwide
■ Worldwide revenue balance:
50% North America, 50% International
■ More than 1,000,000 users
in 175+ countries

1985 1990 1995 2000 2005

75
 ® ®

Key Industries

Aerospace and Defense

Automotive

Biotech and Pharmaceutical

Chemical/Petrochemical

Communications

Education

Electronics

Financial Services

Industrial Automation and
Machinery

Power and Energy

Semiconductor
76
 ® ®

Core MathWorks Products

The leading environment for

technical computing

Numeric computation

Data analysis and visualization

The de facto industry-standard,
high-level programming language
for algorithm development

Toolboxes for signal and image
processing, statistics, optimization,
symbolic math, and other areas

Foundation of The MathWorks
product family

77
 ® ®

Core MathWorks Products

The leading environment for modeling,

simulating, and implementing dynamic and
embedded systems

Linear, nonlinear, discrete-time, continuous-
time, hybrid, and multirate systems

Foundation for Model-Based Design,
including physical-domain modeling,
automatic code generation, and verification
and validation

Open architecture for integrating models
from other tools

Applications in controls, signal processing,
communications, and other system
engineering areas

78
 ® ®

Advantages of The MathWorks

Complete solutions for the entire development process

Customer advisory boards provide strong ties to our developers

Strong imaging expertise

Broad network of third-party developers

Technical support, training, and consulting services

79
 ® ®

Support and Community

80
 ® ®

Engineering expertise and deep product knowledge,

specializing in:

Application development using MATLAB

Model-Based Design using Simulink and Stateflow®

Embedded-system development

Enterprise-wide integration of MathWorks products into
engineering process and systems

Jumpstart services

Project-based services for a growing number of industries,

including aerospace and defense, automotive, communications,
power and marine, and financial services
www.mathworks.com/consulting 81
 ® ®

Three ways to get training

Public training

Offered throughout the world

Schedule and course information at www.mathworks.com/training

Onsite training

Bring training to your site, with course customization available

Web-based training

Instructor-led e-learning

Train at work or at home, with flexible dates and times

Example course topics

Introductory and intermediate training on MATLAB, Simulink,
and Stateflow

Specialized courses in control design, signal processing, test and
measurement, communications, financial analysis, and other areas
www.mathworks.com/training 82
 ® ®

MATLAB Central

File exchange and newsgroup access for MATLAB and

Simulink user community

184,000 visitors per month

More than 4,000 contributed files in the exchange

General-purpose functions,
industry- and application-specific tools and examples

100 new submissions per month

5,000 downloads per day

Web interface to the MATLAB Usenet forum – comp.soft.sys.matlab (CSSM): Over
6,000 posts a day

MATLAB Web Resources, Blogs, Programming Contests, and RSS feeds

www.mathworks.com/matlabcentral 83
 ® ®

The MathWorks Connections Program

More than 300 add-on products and services from partners that
complement and extend MathWorks products

Specialized third-party toolboxes for MATLAB

Interfaces to partners’ software and hardware products

Specialized training courses and consulting services

System integrators and suppliers that incorporate MathWorks products

www.mathworks.com/connections 84
 ® ®

990+ textbooks for educational and professional use,

in 26 languages

Controls
Mathematics

Signal Processing
Aerospace Engineering

Image Processing
Environmental Sciences

Biosciences
Chemistry

Communications
Finance

Mechanical Engineering
Electronics, and more

www.mathworks.com/books 85
 ® ®

Technical Support

Resources

Over 100 support engineers

All with MS degrees (EE, ME, CS)

Local support in North America, Europe,
and Asia

Comprehensive product-specific web
support resources

High Customer Satisfaction

95% of calls answered within
three minutes  

70% of issues resolved within
24 hours
 

80% of customers surveyed rate
satisfaction at 80-100% 

www.mathworks.com/support 86
 ® ®

Next Steps, Questions and Answers

Visit www.mathworks.com

Product literature

User Stories

Visit MATLAB Central

www.mathworks.com/matlabcentral

Click on “File Exchange”

Upcoming seminars in your area

Upcoming webinars

Feel free to ask questions afterwards

87