LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Prof. Heikki Klviinen Lappeenranta University of Technology, Finland

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Applications of Hough Transform for Image Processing and Analysis

Heikki Klviinen
Professor, Computer Science

*Laboratory of Information Processing

Department of Information Technology Lappeenranta University of Technology (LUT), FINLAND [email protected] http/www.lut.fi/~kalviai **Centre for Vision, Speech, and Signal Processing (CVSSP) University of Surrey, UNITED KINGDOM

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Lappeenranta
Helsinki Oslo Stockholm St.Petersburg Tallinn Moscow

London

Berlin

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Hough Transform
Shape detection
Lines, circles, ellipses, arbitrary shapes.

Motion detection and estimation

Simple and robust methods in 2D.

Mixed pixel classification

Large data sets of mixed pixels.

Image compression
Compression and better image quality.

Image databases
Matching of images.

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Hough surveys and comparisons

J. Illingworth, J. Kittler, A Survey of the Hough Transform, Computer Vision, Graphics, and Image Processing, 1988, vol. 44, pp. 87-116. V.F. Leavers, Survey: Which Hough Transform, CVGIP Image Understanding, 1993, vol. 58, no. 2, pp. 250-264. H. Klviinen , P. Hirvonen, L. Xu, E. Oja, Probabilistic, nonprobabilistic Hough transforms: overview and comparisons. Image, Vision Computing, 1995, vol. 13, no. 4, pp. 239-251. N. Kiryati, H. Klviinen, S. Alaoutinen, Randomized or Probabilistic Hough Transform: Unified Performance Evaluation, Pattern Recognition Letters, 2000, vol. 21, nos. 13-14, pp. 1157-1164.
5

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Compression, Similarity, Matching, Object Recognition

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Feature extraction using Hough Transform

Input Image

FEATURE EXTRACTION
Hough Transform Line parameters End-point detection

Reconstruction

Encoding

Feature Image

Feature File

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Hough Transform
A method for global feature extraction:
y = a x + b => b = -x a + y. For each pixel (x,y) compute a curve b = -x a + b into the parameter space. Alternatively the normal presentation of a line: = x cos + y sin

Hough Transform detects sets of pixels which represent geometric primitives in a binary image. Lines, circles, ellipses, arbitrary shapes, etc. Tolerant to noise and distortions in an image, but traditional versions suffer from problems with time and space complexities. New variants: probabilistic and deterministic Hough Transforms.
9

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Hough Transform (SHT)

10

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Kernel of the Hough Transform

1. 2. 3. 4. 5. Create the set D of all edge points in a binary picture. Transform each point in the set D into a parameterized curve in the parameter space. Increment the cells in the parameter space determined by the parametric curve. Detect local maxima in the accumulator array. Each local maximum may correspond to a parametric curve in the image space. Extract the curve segments using the knowledge of the maximum positions.

11

LAPPEENRANTA Randomized UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Hough Transform (RHT)

Developed in Lappeenranta University of Technology (LUT), FINLAND. Xu, L., Oja, E., Kultanen, P, A New Curve Detection Method: Randomized Hough Transform (RHT), Pattern Recognition Letters, vol. 11, no. 5., 1990, pp. 331-338.
12

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Kernel of the Randomized Hough Transform (RHT)

1. 2. 3. 4. 5. 6. Create the set D of all edge points in a binary edge picture. Select a point pair (d_i, d_j) randomly from the set D. If the points do not satisfy the predefined distance limits, go to Step 2; otherwise continue to Step 4. Solve the parameter space point (a, b) using the curve equation with the points (d_i, d_j). Accumulate the cell A(a, b) in the accumulator space. If the A(a, b) is equal to the threshold t, the parameters a and b describe the parameters of the detected curve; otherwise continue to Step 2.
13

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Advances of RHT over SHT

1. 2. 3. 4. Infinite scope parameter space. Arbitrarily high parameter resolution. High computational speed. Small storage.

14

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

RHT Extensions
Klviinen, H., Hirvonen, P., Xu, L., Oja, E., Probabilistic and Non-probabilistic Hough Transforms: Overview and Comparisons, Image and Vision Computing, Vol. 13, No. 4, 1995, pp. 239-251.

15

LAPPEENRANTA More complex UNIVERSITY OF TECHNOLOGYimages

THE DEPARTMENT OF INFORMATION TECHNOLOGY

16

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Motion Detection by RHT (MDRHT)

2D motion detection as sets of moving pixels. A set of moving edge points is assumed to illustrate a moving object frame by frame. The majority of the points are assumed to move rigidly. Two moving points is the simplest version. Extensions: (a) rotation and scaling, (b) exploiting gradient information of each edge point, (c) using three or more moving points as evidence, and (d) detecting multiple moving objects.
17

Motion Detection LAPPEENRANTA UNIVERSITY OF TECHNOLOGY Using RHT THE DEPARTMENT OF INFORMATION TECHNOLOGY (MDRHT)
Klviinen, H., Motion Detection Using the Randomized Hough Transform (RHT): Exploiting Gradient Information and Detecting Multiple Moving Objects, IEE Proceedings--Vision, Image and Signal Processing, Vol. 143, No. 6, 1996, pp. 361-369.
18

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Kernel of Motion Detection Using Randomized Hough Transform (MDRHT)

1. 2. 3. Create the sets B and C of edge points, each in one of two consecutive frames. Select point pairs (b_i,b_j) and (c_i,c_j) randomly from sets B and C, respectively. If the point pairs correspond, calculate the x- and y-translations dx=c_{ix}-b_{ix} and dy=c_{iy}-b_{iy} and go to Step 4; otherwise, go to Step 2. Accumulate the cell A(dx,dy). If the A(dx,dy) is equal to the threshold t, motion (dx,dy) has been detected; otherwise, go to Step 2.
19

4. 5.

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

20

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Detecting partially deformed motion

21

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Detecting multiple objects

22

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Mixed pixel classification

What is in a mixed pixel?: The identification of the constituent components and their proportions in a mixed pixel. For applications with large pixels and/or with large sets of mixed pixels (remote sensing). Bosdogianni, P.*, Klviinen, H., Petrou, M.*, and Kittler, J.*, Robust Unmixing of Large Sets of Mixed Pixels, Pattern Recognition Letters, Vol. 18, 1997, pp. 415-424. *Centre for Vision, Speech, and Signal Processing (CVSSP), University of Surrey, UK
23

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Linear mixing model

w = ax + by + cz w: reflectance of a mixed pixel (known). x, y,z: reflectances of pixels that belong to three different pure classes (known). a,b,c: proportions of the pure classes present in the mixed pixel (unknown). Assuming that a+b+c=1, we obtain w - z = (x-z)a + (y-z)b.

24

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Pure classes with mixed pixels and outliers

25

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Mixel pixel classification by RHT

1. Select one quadruple (x_1,y_1,z_1,w_1) from the first band and another quadruple (x_2,y_2,z_2,w_2) from the second band of the same pixel randomly. 2. Using two selected quadruples compute one (a,b) value in the parametric (a,b) space by w - z = (x-z)a + (y-z)b. 3. Accumulate the cell A(a,b) in the accumulator space. 4. If the A(a, b) is equal to the threshold t, the parameters a and b describe the parameters of the detected proportions; otherwise continue to Step 1.
26

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Advantages and questions

Fast computation and the small accumulator => the use of large datasets possible. Randomized Hough Transform needs less CPU time and memory than Standard Hough Transform when datasets are large. Hough methods are more robust than classical Least Square Methods in the presence of outliers. How high threshold? => e.g. with adaptive termination rules like a variable threshold according to data. More accuracy? => e.g. by averaging several RHT processes.
27

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Image Compression with Hough Feature Extraction *P. Frnti, *E. Ageenko, S. Kukkonen, H. Klviinen,
Using Hough Transform for Context-based Image Compression in Hybrid Raster/Vector Applications, Journal Of Electrical Imaging, 2002, vol. 11, no. 2, pp. 236-245 *Department of Computer Science University of Joensuu, Finland
28

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Goal: To use vector features in contextbased compression of binary images

Context-based compression Feature extraction using Hough transform Feature-based context modeling Feature-based filtering Results Conclusions
29

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Context-based compression
COMPRESSION
vector Feature extraction Feature File

DECOMPRESSION
Retrieval Analysis Editing

Input Image Filtering Compression

raster data

Output Image Decompression

30

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Feature extraction using Hough transform

Input Image

FEATURE EXTRACTION
Hough Transform Line parameters End-point detection

Reconstruction

Encoding

Feature Image

Feature File

31

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Feature-based context modeling (HTC)

COMPRESSION
Feature extraction Feature File vector

DECOMPRESSION
Reconstruction

Context modelling Input Image Feature Image Coding JBIG compression raster data Feature Image

Context modelling Output Image Decoding JBIG decompression

32

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Feature-based filtering: Near-lossless compression system (HTF-JBIG)

COMPRESSION
Feature extraction OPTIONAL Feature File raster data vector

DECOMPRESSION

Input Image

Feature Image Filtering JBIG compression

Output Image JBIG decompression

33

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Input image

NOISE REMOVAL

Noise removal procedure

XOR Feature Image Mismatch pixels Isolated pixel extraction Isolated mismatch pixels XOR

Output Image

34

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Feature Image

Input image

FILTERING
Noise removal

Filtering procedure

Dilation

Noise removal

Erosion

Noise removal

Output Image

35

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY
FIRST STAGE SECOND STAGE THIRD STAGE

Results of the filtering procedure

Input image

Filtering result (1st)

Filtering result (2nd)

Filtering result (3rd)

Hough Transform image

Feature image

Dilated feature image

Eroded feature image

Mismatch pixels (1st)

Mismatch pixels (2nd)

Mismatch pixels (3rd)

Filtered pixels (1st)

Filtered pixels (2nd)

Filtered pixels (3rd)

36

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Original, filtered, and difference images

37

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Test images: Bolt, Plan, House

Chair, Module, Plus

38

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Effects of the feature-based context modeling for the Bolt image

18,000 Compressed file size, bytes 16,000 14,000 12,000 10,000 8,000 6,000 4,000 2,000 0
JBIG Hybrid: 117 segments Hybrid: 289 segments Hybrid: 752 segments Hybrid: 1073 segments

702 1,734

4,512

6,438

12,966

12,598

12,177

11,549

11,514

Raster data

Vector data

39

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Storage sizes in bytes

Image
vector
BOLT PLAN HOUSE CHAIR MODULE PLUS TOTAL

Hybrid compression raster (JBIG)

12,966 5,098 15,688 52,384 7,671 17,609 111,416

Filtering only (HTFJBIG)

10,536 4,325 13,336 51,529 6,431 16,273 102,430

Filtering + Hybrid (HTF-HTC)

9,287 3,786 11,553 48,023 6,057 15,739 94,445

raster (HTC)
11,514 4,578 13,961 50,140 7,222 17,132 104,547

6,438 2,370 13,398 16,710 3,468 5,268 47,652

40

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Computation times of the HT-based compression

Compression
Compression 1:27 Filtering 2:05 Time (min:s) Feature extraction 1:46:28

Decompression
2:00 1:40 1:20 1:00 0:40 0:20 0:00
JBIG HTC HTF-JBIG

41

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Conclusions Two methods proposed for improving compression performance

Feature image as side information for compression Feature-based filtering for removing noise

Problems
Is an exact replica of the original image always needed? How to improve the quality of vectorizing?
42

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Image Databases and Image Matching with Hough Features

*P. Frnti, A. Mednonogov, V. Kyrki, H. Klviinen Content-Based Matching of Line-Drawing Images Using Hough Transform International Journal on Document Analysis and Recognition (IJDAR) 2000, vol. 3, no. 2, pp. 117-124 *Department of Computer Science, University of Joensuu, Finland

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Applications of Hough Transform for image databases

Content-based matching of line-drawing images using Hough Transform. Similarity of images in image databases. Hough Transform as a feature extractor.
Translation-, rotation-, and scale-invariant features from the accumulator matrix.

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Generated 3D images: query images

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Generated 3D images: test database

LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

THE DEPARTMENT OF INFORMATION TECHNOLOGY

Symbol library: noisy and rotated test images

47