J. Shanghai Jiaotong Univ. (Sci.

), 2014, 19(1): 79-83

DOI: 10.1007/s12204-013-1465-3

Sequential Similarity Detection Algorithm Based on

Image Edge Feature

MA Guo-hong∗ (), WANG Cong ( ), LIU Pei ( ), ZHU Shu-lin ()
(School of Mechanical & Electrial Engineering, Nanchang University, Nanchang 330031, China)

© Shanghai Jiaotong University and Springer-Verlag Berlin Heidelberg 2013

Abstract: This paper proposes a new sequential similarity detection algorithm (SSDA), which can overcome
matching error caused by grayscale distortion; meanwhile, time consumption is much less than that of regular
algorithms based on image feature. The algorithm adopts Sobel operator to deal with subgraph and template
image, and regards the region which has maximum relevance as final result. In order to solve time-consuming
problem existing in original algorithm, a coarse-to-fine matching method is put forward. Besides, the location
correlation keeps updating and remains the minimum value in the whole scanning process, which can significantly
decrease time consumption. Experiments show that the algorithm proposed in this article can not only overcome
gray distortion, but also ensure accuracy. Time consumption is at least one time orders of magnitude shorter than
that of primal algorithm.
Key words: welding image, feature matching, sequential similarity detection algorithm (SSDA), self-adaption
value
CLC number: TG 409 Document code: A

0 Introduction In acquiring target area, this approach is much more

efficient than those traditional image processing proce-
As to robot welding operation, motion trajectory dures. Therefore, integrated performance of the whole
control of intelligent welding robot mainly contains ini- intelligent welding system will be promoted. Conse-
tial seam location recognition and seam tracking. The quently, the algorithm proposed in this paper has a
autonomous localization of initial weld position is one promising industrial application.
of the key technologies to realize intellectualized weld-
ing. Some researchers put a T type mark beside the 1 Related Work
initial position, then carried out a series of image pro-
cessing procedures, such as smoothing, segmentation, Image matching is a fundamental task in image pro-
edge detection and Hough transform, and eventually cessing, which is used to match two or more images.
acquire target area[1]. Because this method is based It is widely used in military, industry, medical treat-
on image taken remotely and artificial marker, its re- ment, traffic and security etc. Image matching algo-
sults should be improved; other researchers get an in- rithms mainly fall into two categories.
tersection of work piece edge and seam edge through (1) One category is the algorithms based on gray. It
disposing the whole image, and regard the area with can be divided into two types according to similarity
the intersection at centre as result[2] . This means is measures[6]. One is gray absolute difference which pro-
fairly time-consuming. In the actual unstructured weld- duced by subgraph and template image, i.e. absolute
ing environment, interference generated by disruptive balance search (ABS); the other is maximal similarity,
factors (such as light shade, reflection and work piece i.e. normalized correlation (NC). There are three ap-
surface clearness) will hamper target area recognition. proaches to calculate ABS value: minimum square error
Few researchers adopt image matching method; they (MSE), minimum absolute difference (MAD) and maxi-
select a typical image as template according to prior mum pixel count (MPC). Principles of these approaches
knowledge[3-4] , or define it manually[5] . Target area can are similar. Definition of MAD is given as
be achieved through searching the whole image, and MAD(i, j) =
the center of initial seam position is extracted finally.
1    
M N

T (m, n) − S (i,j) (m, n) , (1)
Received date: 2012-09-28 M N m=1 n=1
Foundation item: the National Natural Science Founda-
tion of China (No. 61165008) where T is an m × n template graph, S is an M × N
∗E-mail: [email protected] reference graph, S (i,j) is a projection of T , and (i, j) is
80 J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 79-83

the coordinate of S which is the lower-left corner point ing form is difficult to be confirmed; for VMCA, the
of S (i,j) and also can be called reference point. The threshold is hard to choose in actual application; in-
meaning of these parameters is shown in Fig. 1. variant moments algorithm is tough to achieve real-time
although it has good stability.
M At present, research on improved SSDA mainly
S(i, j) chooses MAD similarity measure[9,15-16] . Its principle
is simple; meanwhile, it is fast. But result of these
improved algorithms which based on MAD will pro-
duce serious error for gray distortion image. Aiming at
this problem, this paper proposes another new SSDA,
which is based on image feature. It can avoid large time
N

consumption problem produced by prior analogous al-

gorithm upon image feature. A coarse-to-fine search
m
strategy is employed. Besides, self-adaption is used to
optimize the processing procedure. Experiments show
n

that this algorithm has a well integrated performance

and can be applied in intelligent welding system.
(a) Original image S (b) Template T
Fig. 1 Matching process 2 SSDA Based on Image Feature

In these approaches discussed above, metric has the Idea of the algorithm is shown as follows. Firstly
smallest value while T is just covering S (i,j) which is the edge detector is applied in subgrph and template image,
target area, but the result has an error while processing and its essence is to extract edges between target and
grayscale distortion image. Therefore, ABS algorithm background in image. Edge is an important feature of
only can be used in the practical field in which high image. Gray distortion or some other noise has little
speed and low calculation precision are required. In influence on it.
order to improve the ability of resisting grayscale dis- This paper adopts Sobel operator, which belongs to
tortion, Nprod and NNprod have been proposed. Al- the first order differential operators. Sobel operator can
though they can overcome the problem created by dis- not only well respond edge, but also repress noise. All
tortion, it is time-consuming[7] . In order to improve of these characteristics have a strong adaptability to
the image matching speed, sequential similarity detec- welding image in practical application. Calculation of
tion algorithm (SSDA) is put forward[8], it can discover Sobel gradient is based on 3 pixel×3 pixel area. Sobel
and abandon non-matching areas in advance. Besides, operator comprises two convolution kernels. Each pixel
the algorithm is simple. But its threshold value is fixed. point in image has to use kernels to get itself respond.
Meanwhile, plenty of time will be spent if a high thresh- Value in kernel equals to corresponding location value
old value is set, and a low value produces low preci- in masking T , as shown in Fig. 2. One kernel is used
sion. Some researchers have proposed improved algo- to detect horizontal gradient, and its result is largely
rithm to solve it[9] , and the time consumption is largely affected by vertical edge; the other is to detect vertical
decreased based on a self-adaptive threshold value fi- gradient, and horizontal edge has a significant impact
nally. on its response. Two convolution kernels are shown in
(2) The other is the algorithms based on feature. Fig. 2.
It utilizes spatial location relative invariant feature in
matching process. These features can be edge, curve,
−1 −2 −1 −1 0 1
corner point, intersection and centroid etc. Researchers
proposed a new SSDA based on image edge feature[10] .
Canny operator is adopted to extract characteristic 0 0 0 −2 0 2
points from subgraph, and to calculate those points re-
sponding pixel-value difference between subgraph and 1 2 1 −1 0 1
template image. The smallest value corresponding re-
gion is regarded as the target area. (a) x-direction (b) y-direction
In addition to those algorithm discussed above,
Fig. 2 Sobel convolution kernels
speed algorithms also contain wavelet pyramid[11] , ge-
netic algorithm[12], various mask correlation algorithm
(VMCA)[13] and invariant moments algorithm[14] , etc. After the gradient is gotten, a preset constant T is
Detail information of image being searched and tem- used to divide pixel points into two parts: one part will
plate image will be lost during the decomposing of be regarded as edge if its response G is greater than T ,
wavelet. It is difficult to guarantee matching precision; and the other will be considered as background if its
genetic algorithm always causes the problem which cod- response G is less than or equals to T . Relevance can
J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 79-83 81

be defined as will be affected by noise engendered in image transmis-


0, G  T sion, work piece surface cleanness and light conditions,
G= . (2) etc. By the experimental analysis, plenty of small size
1, G > T
pots appear if a small threshold value is taken (in order
Besides, experiments show that T value can affect the to emphasize the visual effect, pixel gray value of fea-
result of image matching effect. By analyzing welding ture point is set to 255, and value of non-feature point
environment image (work piece places on left side in is set to 0), which can seriously affect image matching
image; welding torch of robot lies on bottom right; tar- precision and time consumption. Thresholding pictures
get area is surrounded by an dotted box), image quality are shown in Fig. 3.

(a) Original image (b) Processing (threshold value 75) (c) Processing (threshold value 90)
Fig. 3 Threshold processing on seam image

While threshold value is 75, there are 25 555 feature ture point. The relationship can be shown in
points; meanwhile, big threshold will make the target
area covered by background. Consequently, an appro- A = A1 + A2 + A3 , (3)
priate value is important; a suitable value can not only
filter many useless points, but also remain the target where, A is the sum of feature point location non-
area. When the threshold is set to 90, the number of overlapping between subgraph and template; A1 is the
feature points decreases to 16 119. number of occurrences (when one point of subgraph is
This algorithm is aimed at initial seam location 1, corresponding spot in template is 0); A2 is number of
recognition before welding operation. Initial seam lo- another occurrences (one point of subgraph is 0 while
cation graphic contains seam, contour and surface of corresponding spot in template is 1); A3 is the last case
the workpiece, and padding (or fixture) located below (when one point of subgraph is 0, corresponding spot
workpiece, etc. All of this information can ensure that in template is 0).
the image which is also regarded as template has ob- The whole scan process adopts coarse-to-fine scan-
vious edge feature. Therefore, the chance that target ning method. In coarse matching part, an initial value
region appears in background area is impossible. A0 which indicates the feature point location relevance
between template and lower-left corner subgraph of ref-
If we directly make a location correlation detecting erence graph should be acquired firstly. Then scan the
between subgraph feature points and template feature rest subgraph, and compare each Ai (i = 1, 2, 3) pro-
points, a region which has a maximum location cor- duced by subgraph and template graph. When Ai is
relation will be considered as the target area. That greater than A0 , scanning process of a subgraph is not
approach can get an accurate result, but programs will finished yet, then scanning should stop immediately,
take a long time to complete. Because programs need and directly scan next subgraph. If Ai is less than A0
to scan (N − n + 1) × (M − m + 1) points (N , M are till the end, then Ai replace A0 before the next scanning
the size of S; n, m are the size of T ; the process is begins. Coarse matching result, denoted as point (X,
shown in Fig. 1), the algorithm calculates gradient of Y ), can be acquired after the whole reference graph.
every point in each subgraph S (i,j) in S, and counts Every subgraph of reference graph can be processed
location correlation between subgraph S (i,j) and tem- according to
plate T . Therefore, the means should be improved.

A0 , Ai > A0
3 Algorithm Optimization and Proce- A= . (4)
Ai , Ai  A0
dures
In fine matching process, the scanning area is a 2m ×
We select feature point location non-overlapping 2n rectangle area which center point is (X, Y ) and its
frequency as correlation measure. An image can be scan procedure is the same as coarse scanning.
equivalent to an empty two-dimensional array. Every Scanning strategy is shown as follows.
pixel is corresponding to array element one by one. An (1) One strategy is coarse matching. It is to make
element is set to 1 when its corresponding pixel is fea- the feature point location relevance between lower-left
82 J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 79-83

corner subgraph of reference graph and template as ini- shown in Table 2.

tial value A0 , and to keep the minimum relativity by
contrast location relevance between the rest subgraphs Table 1 Seam image matching data
and template graph. Matching
(2) The other strategy is fine matching. It is to scan Matching Time Matching
coordinate/
the rectangular area which takes the point (X, Y ) as algorithm consumption/s
pixel
error
the center, has a 2m × 2n size, and to repeat the same
MAD 9.625 (193, 209) (0, 0)
procedures in coarse matching. Eventually, the precise
NC 11.875 (193, 209) (0, 0)
reference point of target area can be acquired.
Self-adaption SSDA 0.078 (193, 209) (0, 0)
Flow chart of the whole algorithm is shown in Fig. 4.
This paper 0.285 (193, 209) (0, 0)

Start Table 2 Matching data of seam image suffered

gray distortion
m×n coarse matching
Matching
Matching Time Matching
Count initial feature point location coordinate/
algorithm consumption/s error
correlation A0 between lower-left suggraph pixel
of reference and template diagram MAD 9.390 (417, 113) (224, 96)
NC 11.922 (193, 209) (0, 0)
min {A0, Ai} Self-adaption SSDA 0.095 (198, 516) (5, 307)
This paper 0.311 (193, 209) (0, 0)
2m×2n fine matching
From experiments shown above we can know, results
min {A0, Aj}
are all right while image is without gray distortion. Be-
sides, time consumption of self-adaption SSDA is the
Target area
least. From the gray distortion image, outcome of MAD
and self-adaption SSDA all cause large error. Although
End
NC algorithm has no error, it is time-consuming. The
Fig. 4 Algorithm flow chart algorithm that this paper proposes not only has a pre-
cise result, but also costs less time.
4 Experiment Results and Analysis Meanwhile, we select Lena image to prove the versa-
tility of this algorithm, Lena image size is 256 pixel×256
Accuracy and speed are two important evaluation in- pixel, and template size is 64 pixel×64 pixel. Matching
dexes for an algorithm. In order to prove the quality result and responding data are shown respectively in
of the algorithm this paper proposes, we compare the Fig. 6 and Table 3.
algorithm with MAD, NC and self-adaption SSDA (its The algorithm that this paper proposes adopts a
step is the same as the algorithm this paper proposes). coarse-to-fine method. Firstly, subgraph moves on
Size of S which is being searched is 766 pixel × 578 original image at intervals of m rows and n columns.
pixel. Size of template T is 40 pixel × 30 pixel. Com- Besides, a self-adaption value that indicates feature
puter system is Windows XP Sp3, 1.73-GHz core and points’ location correlation is proposed. After get the
1-GB RAM; image collection card is DH-VT140, and coarse reference point of target area, then scan the re-
image processing software is Visual C++ 6.0. Match- gion which takes the coarse reference point as the cen-
ing result and responding data of seam image are shown ter, 2m × 2n size, and repeat the same matching proce-
in Fig. 5 and Table 1. dures in coarse matching. Finally, precise coordinates
In order to test the ability to resist gray distortion of of target area can be acquired. Meanwhile, its time
each algorithm, we select a template T which is suffered consumption is much less than that of MAD and NC
gray distortion. Matching data of each algorithm is algorithms. Since edge detection of every subgrph needs

(a) Seam image (b) Template T (c) Matching result

Fig. 5 Matching result of seam image
J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 79-83 83

(a) Lena image (b) Template T (c) Matching result

Fig. 6 Matching result of Lena image

Table 3 Lena image matching data

[3] Chen X Z, Chen S B, Lin T, et al. Practical method
Matching to locate the initial weld position usingvisual technol-
Matching Time Matching ogy [J]. International Journal of Advanced Manufac-
coordinate/
algorithm consumption/s error turing Technology, 2006, 30: 663-668.
pixel
[4] Zhu Zhen-you, Piao Yong-jie, Lin Tao, et al. Visual-
MAD 3.375 (116, 74) (0, 0) based research on weld seam initial position recogni-
NC 3.937 (116, 74) (0, 0) tion in local environment [J]. Transactions of the China
Self-adaption SSDA 0.063 (116, 74) (0, 0) Welding Institution, 2004, 25(2): 95-98 (in Chinese).
This paper 0.225 (116, 74) (0, 0)
[5] Liu Guo-ping, Wang Hong-liang, Hu Rong-hua, et al.
Application of template matching in welding image [J].
Welding Technology, 2004, 33(4): 14-15(in Chinese).
plenty of time, and self-adaption SSDA only needs to
[6] Zhu Yong-song, Guo Cheng-ming. The research of cor-
count absolute differences based on pixel gray value relation matching algorithm based on correlation coef-
which already exists, so SSDA is faster than the algo- ficient [J]. Signal Processing, 2003, 19(6): 531-534 (in
rithm this paper proposed. But the feature algorithm Chinese).
can overcome the influence produced by gray distortion. [7] Brown L G. A survey of image registration techniques
[J]. ACM Computing Surveys, 1992, 24: 325-376.
5 Conclusion [8] Bernea D I, Silverman H F. A class of algorithms
for fast digital registration [J]. IEEE Transactions on
To keep the matching error minimum, algorithm that Computers, 1972, 21(2): 179-186
this paper proposes makes full use of the edge informa- [9] Mi Chang-wei, Liu Xiao-li, Xu Ming-you. An ad-
tion, which can overcome error produced by distortion vanced algorithm based on SSDA [J]. Journal of Pro-
and ensure matching precision. jectiles, Rockets, Missiles and Guidance, 2004, 24(1):
Adopt a self-adaption value which indicates feature 85-87 (in Chinese).
points’ location correlation between subgraph and tem- [10] Li Jun-shan, Tan Yuan-yuan, Zhang Yuan-li. An im-
plate graph. This can further decrease the time con- proved SSDA [J]. Electronics Optics & Control, 2007,
sumption. 14(2): 66-68 (in Chinese).
[11] Luo Zhong-xuan, Liu Cheng-ming. Fast algorithm of
Since algorithm that this article proposes adopts a
image matching [J]. Journal of Computer-Aided De-
self-adaption value and coarse-to-fine scanning strategy, sign & Computer Graphics, 2005, 17(5): 966-970 (in
its matching speed is faster than that of MAD and NC. Chinese).
However, feature points that between template image [12] Tian Ying, Yuan Wei-qi. Application of the genetic
and subgraph regions need to be detected. Traditional algorithm in image processing [J]. Journal of Image
SSDA arithmetic just requires absolute value of gray and Graphics, 2007, 12(3): 389-396 (in Chinese).
difference which already exists. So matching speed of [13] Xiong Guo-qing, Yu Qi-feng. Fast matching algorithm
the algorithm this article proposes is slower than that for real-time tracking [J]. Journal of Computer-Aided
of traditional arithmetic. Design & Computer Graphics, 2002, 14(1): 41-43 (in
Chinese).
References [14] Yan Bo-jun, Zheng Lian, Wang Ke-yong. Fast target-
detecting algorithm based on invariant moment [J]. In-
[1] Ma G H, Wang L, Liu Q Q, et al. Research on im- frared Technology, 2001, 23(6): 8-12 (in Chinese).
age process and tracing of a welding robot [J]. Lecture [15] Wu Pei-jing, Chen Guang-meng. An improved SSDA
Notes in Electrical Engineering, 2010, 88: 253-259. in image registration [J]. Computer Engineering and
[2] Chen Xi-zhang, Chen Shan-ben. Recognition and po- Applications, 2005, 33: 76-78 (in Chinese).
sitioning of start welding position for arc welding robot [16] Chen Wei-bing, Shu Hui. Research on fast edge
[J]. Transactions of the China Welding Institution, matching algorithm [J]. Computer Engineering and
2009, 30(4): 18-20 (in Chinese). Design, 2004, 25(1): 130-132(in Chinese).