A modified Normalized Cross-Correlation algorithm

2021 IEEE 12th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) | 978-1-6654-0690-1/21/$31.00 ©2021 IEEE | DOI: 10.1109/UEMCON53757.2021.9666634

for Embedded Systems

Ismael Lopez Sanchez* Miguel Angel Rosas Galaviz* Damian Gomez Herrera* Dr. Luis Rizo Dominguez*
Embedded Systems Specialization Embedded Systems Specialization Embedded Systems Specialization IEEE Member
[email protected] [email protected] [email protected] [email protected]

Abstract—Problems related to image matching, such as image matching, such as processing time, performance, and
processing time, performance, and changes in rotation and scale changes in rotation and scale [4,5,6]. Moreover, these studies
are widely studied. However, creating simple solutions for image have overlooked the creation of a simple and affordable
matching that can be implemented in an embedded system has solution for micro and small-sized enterprises.
been overlooked. Therefore, this paper proposes a modified
Normalized Cross-Correlation algorithm for embedded systems
In contrast with large and medium-sized enterprises, micro
aimed at creating a low-cost solution for a cement tile application. and small-sized enterprises do not have the same possibilities
The algorithm relies on the concept of template matching and it to spend their budget in state-of-the-art technology. Thus, these
makes use of cross-correlation, zero-mean normalization, and companies seek straightforward and stand-alone solutions that
thresholding for its implementation. Experimental results show can be easily integrated in their manufacturing process without
that the algorithm is suitable for a cement tile application and representing a huge investment. Previous research has proposed
appropriate to detect defective cement tiles during an on-site the implementation of NCC algorithms in workstations,
installation or manufacturing process. Finally, it is shown that the computers, or even using MATLAB, a programming software
algorithm can act as a discriminator to identify quality non- platform designed for engineers and scientists [7,8]. These
conformance issues.
NCC algorithms require significant computational resources,
Keywords— Normalized Cross-Correlation (NCC) algorithm, which complicate their implementation in a single embedded
template, reference image, embedded system system [9].
An NCC algorithm has been selected due to its simplicity
I. INTRODUCTION and effectiveness when it is implemented in an embedded
Image processing methods have played a significant role in system. Therefore, this paper proposes the implementation of a
detecting defects to prevent consumers from purchasing modified NCC algorithm in an embedded system based on
defective products. One of these methods involves template image matching for a low-cost cement tile application.
matching techniques, which are widely used to compare images The rest of the paper is organized as follows. Section II
and evaluate if a product has been manufactured with the introduces the NCC fundamentals for computing the cross-
desired quality [1]. The Sum of Absolute Differences (SAD), correlation coefficients. The proposal for the modified NCC
the Sum of Squared Differences (SSD), and the Normalized algorithm is presented in section III. The algorithm
Cross-Correlation (NCC), are some of the template matching implementation is described in section IV. Lastly, the algorithm
algorithms that have been proposed to correlate two images [2]. validation and the results are given in section V.
According to [3], NCC algorithms are widely reported in the
II. NORMALIZED CROSS-CORRELATION (NCC)
literature; they are based on correlation measures that detect
similarities and they are characterized by their robustness and An NCC algorithm locates a template into a reference image
effectiveness. to calculate a correlation coefficient; typically, the template
In essence, NCC algorithms are used to shift and compare origin is overlapped with the origin of the reference image to set
an input image, also known as template, with windows of the the coordinates of both origins at (0, 0) as shown in Fig. 1. After
same size in a reference image so that different correlation computing the first correlation coefficient, the template is
shifted around the reference image to compute the rest of the
coefficients can be obtained. These correlation coefficients are
correlation coefficients to find the maximum value of the NCC
necessary to quantify the degree of similarity between the
function according to the following equation:
template and each image window. By measuring this, it is
possible to determine if the template is an acceptable match; ∑ ∑ , ∙ ,
hence, defective products can be segregated with the intention  𝑁𝐶𝐶 𝑥, 𝑦  
∑ ∑ , ∙ ∑ ∑ ,
of not compromising the quality of a product.
Although there have been numerous studies aimed at
improving image matching techniques based on NCC where x and y denote the coordinates of the reference image of
algorithms, they only focus on solving problems related to size M x N, x= 0…N-n and y= 0…M-m, and i and j are the

978-1-6654-0690-1/21/$31.00 ©2021 IEEE

*ITESO, Jesuit University of Guadalajara

0126
Authorized licensed use limited to: Yalova Universitesi. Downloaded on December 27,2023 at 18:01:07 UTC from IEEE Xplore. Restrictions apply.
shifting variables that represent the distance from the separate regions of interest within an image. Normally, a pixel
corresponding x and y position. Moreover, the numerator of (1) in an image is compared with a constant threshold and it is
denotes the cross-correlation between the template and the replaced with a black pixel if its value is less than the defined
reference image, while the two terms on the denominator constant or with a white pixel if its value is greater than that
represent the normalization of the reference image I, and of the constant, or vice versa. Thus, the purpose of a thresholding
template T. operation is to create a binary image. Since a binary image can
be easily related to a pass-fail criterion, thresholding supports
the definition of a similarity measure.
Although thresholding is a method that processes a grayscale
or color image into a black and white image, the definition of a
cutoff value is required to create a binary output. By leveraging
on the cross-correlation and the zero-mean normalization, the
implementation of the algorithm can be simplified. However,
instead of using the correlation between two different images,
the autocorrelation function (ACF) can be used,

Fig. 1. (a) Template and (b) Reference image.  𝐴𝐶𝐹 𝑥, 𝑦 ∑ ∑ 𝑇 𝑥 𝑖, 𝑦 𝑗 ∙ 𝑇 𝑖, 𝑗  

III. PROPOSED MODIFIED NCC ALGORITHM The autocorrelation function is an adequate method to
The proposed algorithm in this paper showed a better determine the cutoff value since the maximum correlation
performance when the images had the same reference; this was coefficient that can be attained from an image is the resulting
accomplished by sharing the same mean. Thus, a modification value of the dot product from the same image. Therefore,
of the traditional NCC algorithm with a zero-mean solving for (3) results in the maximum degree of similarity of
normalization, is proposed. The key elements of the algorithm any image. Consequently, a cutoff threshold can be established
can be listed as follows: cross-correlation, zero-mean by multiplying the autocorrelation coefficient by a constant,
normalization, and thresholding.
 𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 ∙ 𝐴𝐶𝐹 𝑥, 𝑦  
A. Cross-Correlation
The algorithm uses the numerator of (1) to calculate the dot where the constant lies between 0 and 1. The characterization of
product of two numeric arrays: the reference image and the the constant value depends on the intensity of the image.
template. In other words, an element-by-element multiplication
of the template and the reference image vectors is performed to These three elements were used to build the proposed
produce a vector of products and then, these vectors of products algorithm. While the cross-correlation and the zero-mean
are summed up to produce a scalar result, also known as the normalization are responsible for quantifying the degree of
correlation coefficient. similarity between the template and the reference image,
thresholding is the method responsible for dictating if the
B. Zero-Mean Normalization correlation coefficient value means that a satisfactory match has
Before the dot product can be used, the two numeric arrays been found.
must be normalized. One alternative to normalize a cross-
IV. IMPLEMENTATION OF THE PROPOSED NCC ALGORITHM
correlation function is to use the denominator of (1). This
method is particularly useful when it is implemented in a There are four significant aspects that were considered for
computer. However, executing this function in an embedded the implementation.
system represents a great challenge because of the numerical Firstly, the clear definition of the elements used in the
operations that must be performed. For this reason, the mean is numerator in (1) and (3), as well as relating them to a cement tile
subtracted from both numeric arrays, which is another method application for validation purposes, were essential to specify the
to accomplish a normalized function. Specifically, the mean of implementation approach. Typically, to obtain the correlation
the template pixels is subtracted from each of the elements coefficients, the template is shifted around the reference image
contained in the template numeric array; similarly, the reference as indicated in (1). Note that if T and I are interchanged, then the
image is normalized to zero mean. Hence, the equations to outcome of the cross-correlation (CC) is the same. Thus, in
calculate the mean are, equation (5), T and I were swapped as follows:
, ,
∑ , ∑ ,
 𝑇
,
and 𝐼 ̅
,
   𝐶𝐶 𝑥, 𝑦 ∑ ∑ 𝑇 𝑥 𝑖, 𝑦 𝑗 ∙ 𝐼 𝑖, 𝑗  
∙ ∙

to explicitly state that the reference image is shifted around the

C. Thresholding
template since this is a more convenient approach for the cement
Once the pixel values of the images are normalized and the tile application and, mathematically speaking, it yields the same
correlation coefficients are calculated, a criterion for the output. Consequently, (3) can be expressed as,
similarity measure must be determined. Thresholding is a simple
and efficient approach that is used in image processing to  𝐴𝐶𝐹 𝑥, 𝑦 ∑ ∑ 𝐼 𝑥 𝑖, 𝑦 𝑗 ∙ 𝐼 𝑖, 𝑗  

0127
Authorized licensed use limited to: Yalova Universitesi. Downloaded on December 27,2023 at 18:01:07 UTC from IEEE Xplore. Restrictions apply.
to illustrate that the threshold in (4) is obtained from the Start
reference image I.
Once the elements are properly defined, the boundaries for
moving the images must be established. As expressed in (5), Fig. Acquire template image
2 shows that the reference image is shifted around the template
taking into consideration the size of the image to delimit the
region where no calculations are required.
Convert template to grayscale
image, set Threshold

Reference image size ≥ No End

Template size

Yes
Timer start

Fig. 2. Shifting the reference image around the template.

Normalize images to zero
Secondly, determining the procedure to compute the mean
algorithm is essential since embedded systems are known for
their limited resources. Although this is a valid belief, it is no
longer the rule. Current technological developments have led Calculate normalized
embedded systems to use more than one microprocessor core, Parallel
ACF(x,y)
allowing tasks to be run in parallel. Therefore, the reference computing
image is shifted around the template by sliding it using four
threads as shown in Fig. 3.
Calculate NCC(x,y)

NCC(x,y) > No
Threshold
Yes
White Black
Pixel Pixel
Fig. 3. Parallel computing scheme considering 4 threads.
Next pixel > No
Thirdly, since an embedded system has memory limitations, Template size
an optimization for computing the mean was considered. Hence,
Yes
instead of using (2) – the traditional algebraic function for the Timer stop
mean, the equation that was used within the implementation is End
the following recurrence relation:
Fig. 4. Flow chart of the modified NCC algorithm.
 𝑇 𝑇 𝑇 𝑘 1  
V. EXPERIMENTAL RESULTS
where k is related to the iteration number. This section presents the results of the embedded system
implementation and testing using the modified NCC algorithm
Lastly, the threshold in (4) is set in order to guarantee the as described in Fig. 4. To demonstrate the effectiveness of the
quality of the cement tile. This number is obtained from the algorithm, the experiments were tested on real grayscale images
autocorrelation function and then it is compared to the outcome of a cement tile application in terms of a pass-fail criterion and
of the modified NCC algorithm; defining this approach was the processing time. Fig. 5(a), Fig. 5(b), Fig. 6(a), Fig. 6(b), and Fig.
key aspect for stipulating a pass-fail criterion. 8(a) show the reference images and templates that were used
The flow chart in Fig. 4 depicts the modified NCC algorithm during the algorithm validation. All images were taken with the
that was implemented in an embedded system using (5), (6), and same camera. The experiments were conducted on a
(7). RASPBERRY PI 3 Model B, processor Broadcom BCM2711,
Quad core Cortex-A72 (ARM v8), and 64-bit SoC @ 1.5GHz.
Fig. 5(a) shows the typical four-image pattern used during
the installation process of cement tiles. This means that four

0128
Authorized licensed use limited to: Yalova Universitesi. Downloaded on December 27,2023 at 18:01:07 UTC from IEEE Xplore. Restrictions apply.
cement tiles are placed in a specific sequence. The algorithm also evaluate a single piece. Therefore, this algorithm was
utilizes this four-image pattern as the template since this is the subjected to an additional test where a single-image pattern and
image that must be validated. Fig. 5(b) shows the reference the same template from Fig. 5(b), were used.
image that is shifted around Fig. 5(a) to determine if an
acceptable match can be found. In other words, the reference Fig. 7(a) depicts the template as a single-image pattern. Fig.
image is considered the golden sample that dictates the degree 7(b) shows the reference image. Note that Fig. 7(a) and Fig. 7(b)
of similarity that must be achieved by setting the threshold value are images of cement tiles that were manufactured at different
according to the autocorrelation function. Fig 5(c) shows the points in time. Fig. 7(c) shows the image processing output,
binary image that is obtained after executing the proposed which is a single white pixel that corresponds to the correlation
algorithm. The thresholding operation of the algorithm coefficient obtained from the algorithm. Hence, the modified
simplifies the visualization of the outcome by indicating in black NCC algorithm can also detect the degree of similarity of a
the region where no matches were found and in white the single-image pattern; specifically, Fig. 7(c) exemplifies a
matches that are acceptable. As shown in Fig. 5(c), the upper satisfactory match. Moreover, it was observed that the
right corner was replaced with white pixels demonstrating that processing time is related to the image size.
Fig. 5(b) was only found in that region. Fig. 5(d) shows the
cross-correlation plot of the outcome. Thus, the modified NCC
algorithm can identify the reference image within the template
and, therefore, this algorithm can also detect quality non-
conformance issues.

Fig. 7. (a) Template with correct single-image pattern (b) Square model for
reference image (c) Image proccessing output considering a specific
threshold.

An incorrect single-image pattern, Fig. 8(a), was used to test

that the algorithm can determine that no acceptable matches are
Fig. 5. (a) Template with four-image pattern (b) Square model for reference found. The results of Fig. 8 show that the modified NCC
image (c) Image proccessing output considering a specific threshold (d)
Cross-correlation outcome.
algorithm can also detect quality non-conformance issues on a
single-image pattern.
A different four-image pattern and distinct template were
used during the experiments to verify the robustness of the
algorithm. Similar satisfactory results as previously discussed
for Fig. 5 were attained. These results are shown in Fig. 6.

Fig. 8. (a) Template with incorrect single-image pattern (b) Square model for
reference image (c) Image proccessing output considering a specific
threshold.

In order to evaluate the time performance of the modified

algorithm, Table I reports the processing times for two different
Fig. 6. (a) Template with four-image pattern (b) Diagonal model for template sizes and three reference image sizes considering the
reference image (c) Image proccessing output considering a specific threshold
(d) Cross-correlation outcome. four-image pattern of Fig. 5 and Fig. 6.

The cement tiles are produced using a handmade

manufacturing process where only one tile can be produced at a
time. The modified NCC algorithm proposed in this paper can

0129
Authorized licensed use limited to: Yalova Universitesi. Downloaded on December 27,2023 at 18:01:07 UTC from IEEE Xplore. Restrictions apply.
 TABLE I. ALGORITHM RESULTS USING A FOUR-IMAGE PATTERN that the proposed algorithm is suitable for a cement tile
Sample Template Reference Execution application. Moreover, the execution times obtained in the
No. Size Image Size Time (s) embedded system implementation are appropriate for the
1. 480x480 100x100 36.624 manufacturing process.
2. 480x480 150x150 60.144 The proposed algorithm was tested under two different
scenarios – the four-image pattern and the single-image pattern
3. 480x480 230x230 83.069
– to determine an acceptable match. Consequently, when the
4. 320x240 100x100 7.471 threshold is set, it can be used as a discriminator of defective
cement tiles during the on-site installation or manufacturing
5. 320x240 150x150 8.819
process.
6. 320x240 230x230 1.381
Future research could include an algorithm to detect changes
in illumination and color variation. Similarly, edge detection
Table II shows the execution times for images of various algorithms can be developed on top of the current
sizes and including the single-image pattern of Fig. 7 and Fig. 8, implementation. Additionally, the proposed algorithm can be
which implies that the template and the reference image are of implemented in other boards.
the same size.
ACKNOWLEDGMENTS
TABLE II. ALGORITHM RESULTS USING A SINGLE-IMAGE PATTERN The authors thank the cement tile company, Studio Victoria,
Sample Template Reference Execution
for manufacturing the tiles used in this study.
No. Size Image Size Time (s)
REFERENCES
1. 100x100 100x100 0.007

2. 130x130 130x130 0.012 [1] S. Omachi and M. Omachi, “Fast Template Matching With Polynomials,”
IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2139–2149,
3. 150x150 150x150 0.014 Aug. 2007.
4. 180x180 180x180 0.011 [2] J. N. Sarvaiya, S. Patnaik, and S. Bombaywala, “Image Registration by
Template Matching Using Normalized Cross-Correlation,” in 2009
5. 200x200 200x200 0.011 International Conference on Advances in Computing, Control, and
Telecommunication Technologies, Dec. 2009, pp. 819–822.
6. 230x230 230x230 0.016
[3] M. B. Hisham, S. N. Yaakob, R. A. A. Raof, A. B. A. Nazren, and N. M.
W. Embedded, “Template Matching using Sum of Squared Difference
and Normalized Cross Correlation,” in 2015 IEEE Student Conference on
In accordance with other experiments reported in the Research and Development (SCOReD), Dec. 2015, pp. 100–104.
literature, reduced processing times are obtained with the [4] L. Di Stefano, S. Mattoccia, and M. Mola, “An efficient algorithm for
modified NCC algorithm. For instance, a study reports an exhaustive template matching based on normalized cross correlation,” in
experiment where an image of size 384 x 512 and a template of 12th International Conference on Image Analysis and Processing,
2003.Proceedings., Sep. 2003, pp. 322–327.
size 150 x150 was used resulting in a processing time of
346.9846 s [3]. Meanwhile, the execution time for the proposed [5] S.-D. Wei and S.-H. Lai, “Fast Template Matching Based on Normalized
Cross Correlation With Adaptive Multilevel Winner Update,” IEEE
algorithm with similar image sizes – 480 x 480 and 150x150 – Transactions on Image Processing, vol. 17, no. 11, pp. 2227–2235, Nov.
was 60.144 s. In general terms, the processing time was reduced 2008.
to 17% during the four-image pattern experiment. From the [6] F. Zhao, Q. Huang, and W. Gao, “Image Matching by Normalized Cross-
experimental results, it is observed that the processing time is Correlation,” in 2006 IEEE International Conference on Acoustics
proportional to the number of iterations when shifting the Speech and Signal Processing Proceedings, May 2006, vol. 2, pp. II–II.
reference image around the template. The execution time is also [7] Y. R. Rao, N. Prathapani, and E. Nagabhooshanam, “Application of
proportional to the number of pixels. Normalized Cross Correlation to Image Registration,” IJRET, vol. 03, no.
5, pp. 12–16, May 2014.
Furthermore, since the cement tile application relies on a [8] S. Mattoccia, F. Tombari, and L. Di Stefano, “Fast Full-Search Equivalent
handmade manufacturing process, it implies a slow production Template Matching by Enhanced Bounded Correlation,” IEEE
rate and the execution times attained from the modified NCC Transactions on Image Processing, vol. 17, no. 4, pp. 528–538, Apr.
2008.
algorithm do not impact negatively on the production line.
[9] Z. Yang, “Fast Template Matching Based on Normalized Cross
Correlation with Centroid Bounding,” in 2010 International Conference
VI. CONCLUSION on Measuring Technology and Mechatronics Automation, Mar. 2010, vol.
In this paper, a modified NCC algorithm was implemented 2, pp. 224–227.
in an embedded system using cross-correlation, zero-mean
normalization, and thresholding. The experimental results show

0130
Authorized licensed use limited to: Yalova Universitesi. Downloaded on December 27,2023 at 18:01:07 UTC from IEEE Xplore. Restrictions apply.