2944 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO.

7, JULY 2014

A Distributed Canny Edge Detector: Algorithm

and FPGA Implementation
Qian Xu, Srenivas Varadarajan, Chaitali Chakrabarti, Fellow, IEEE, and Lina J. Karam, Fellow, IEEE

Abstract The Canny edge detector is one of the most widely and has best performance [10]. Its superior performance is due
used edge detection algorithms due to its superior performance. to the fact that the Canny algorithm performs hysteresis thresh-
Unfortunately, not only is it computationally more intensive as olding which requires computing high and low thresholds
compared with other edge detection algorithms, but it also has
a higher latency because it is based on frame-level statistics. based on the entire image statistics. Unfortunately, this feature
In this paper, we propose a mechanism to implement the Canny makes the Canny edge detection algorithm not only more
algorithm at the block level without any loss in edge detection computationally complex as compared to other edge detection
performance compared with the original frame-level Canny algorithms, such as the Roberts and Sobel algorithms, but also
algorithm. Directly applying the original Canny algorithm at necessitates additional pre-processing computations to be done
the block-level leads to excessive edges in smooth regions and
to loss of significant edges in high-detailed regions since the on the entire image. As a result, a direct implementation of
original Canny computes the high and low thresholds based the Canny algorithm has high latency and cannot be employed
on the frame-level statistics. To solve this problem, we present in real-time applications.
a distributed Canny edge detection algorithm that adaptively Many implementations of the Canny algorithm have been
computes the edge detection thresholds based on the block type proposed on a wide list of hardware platforms. There
and the local distribution of the gradients in the image block.
In addition, the new algorithm uses a nonuniform gradient mag- is a set of work [1][3] on Deriche filters that have
nitude histogram to compute block-based hysteresis thresholds. been derived using Cannys criteria and implemented on
The resulting block-based algorithm has a significantly reduced ASIC-based platforms. The Canny-Deriche filter [1] is
latency and can be easily integrated with other block-based a network with four transputers that detect edges in a
image codecs. It is capable of supporting fast edge detection 256 256 image in 6s, far from the requirement for real-
of images and videos with high resolutions, including full-HD
since the latency is now a function of the block size instead of time applications. Although the design in [2] improved the
the frame size. In addition, quantitative conformance evaluations Canny-Deriche filter implementation of [1] and was able to
and subjective tests show that the edge detection performance of process 25 frames/s at 33 MHz, the used off-chip SRAM
the proposed algorithm is better than the original frame-based memories consist of Last-In First-Out (LIFO) stacks, which
algorithm, especially when noise is present in the images. Finally, increased the area overhead compared to [1]. Demigny pro-
this algorithm is implemented using a 32 computing engine
architecture and is synthesized on the Xilinx Virtex-5 FPGA. posed a new organization of the Canny-Deriche filter in [3],
The synthesized architecture takes only 0.721 ms (including the which reduces the memory size and the computation cost by a
SRAM READ / WRITE time and the computation time) to detect factor of two. However, the number of clock cycles per pixel
edges of 512 512 images in the USC SIPI database when of the implementation [3] varies with the size of the processed
clocked at 100 MHz and is faster than existing FPGA and GPU image, resulting in variable clock-cycles/pixel from one image
implementations.
size to another with increasing processing time as the image
Index Terms Distributed image processing, Canny edge size increases.
detector, high throughput, parallel processing, FPGA. There is another set of work [4][6] on mapping the Canny
edge detection algorithm onto FPGA-based platforms. The two
I. I NTRODUCTION
FPGA implementations in [4] and [5] translate the software

E DGE detection is the most common preprocessing step

in many image processing algorithms such as image
enhancement, image segmentation, tracking and image/video
design directly into VHDL or Verilog using system-level
hardware design tools, which results in a decreased timing per-
formance as shown later in Section 6 of this paper. The parallel
coding. Among the existing edge detection algorithms, the implementation in [6] operates on only 4 pixels in parallel,
Canny edge detector has remained a standard for many years resulting in an increase in the number of memory accesses
and in the processing time as compared to the proposed
Manuscript received December 10, 2012; revised June 20, 2013 and
November 7, 2013; accepted November 17, 2013. Date of publication algorithm. Furthermore, in order to reduce the computational
March 18, 2014; date of current version June 3, 2014. The associate editor complexity, all of these implementations compute the high and
coordinating the review of this manuscript and approving it for publication was low thresholds off-line and use the same fixed threshold values
Prof. Marios S. Pattichis.
The authors are with the School of Electrical, Computer and Energy for all the images, which result in a decreased edge detection
Engineering, Arizona State University, Tempe, AZ 85281 USA (e-mail: performance. In [7], a self-adapt threshold Canny algorithm
[email protected]; [email protected]; [email protected]; [email protected]). is proposed for a mobile robot system where a low and high
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org. threshold values are computed for each input image and an
Digital Object Identifier 10.1109/TIP.2014.2311656 implementation for an Altera Cyclone FPGA is presented.

1057-7149 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
XU et al.: DISTRIBUTED CANNY EDGE DETECTOR 2945

Recently, the General Purpose Graphic Processing Unit time and the computation time) to detect edges of 512 512
(GPGPU) has emerged as a powerful and accessible parallel images in the USC SIPI database when clocked at 100 MHz.
computing platform for image processing applications [8], [9]. A preliminary version of this work was presented in [13].
Studies of GPGPU accelerated Canny edge detection have The work presented herein does not only provide more elabo-
been presented [10][12]. All of these implementations are rate discussions and performance results but also provides an
frame-based and do not have good edge detection performance improved distributed Canny edge detector both at the algo-
since they use the same fixed pair of high and low threshold rithm and architecture levels. The algorithm is improved by
values for all images. Furthermore, as shown later in the paper, proposing a novel block-adaptive threshold selection procedure
their timing performance is inferior compared to the proposed that exploits the local image characteristics and is more robust
algorithm in spite of being operated at a very high clock to changes in block size as compared to [13]. While the
frequency. architecture presented in [13] was limited to a fixed image
In the original Canny method, the computation of the high size of 512 512, a fixed image block of 64 64, this paper
and low threshold values depends on the statistics of the whole presents a general FPGA-based pipelined architecture that
input image. However, most of the above existing implementa- can support any image size and block size. Furthermore, no
tions (e.g., [4][6], [10][12]) use the same fixed pair of high FPGA synthesis results and no comparison results with exist-
and low threshold values for all input images. This results ing techniques were presented in [13]. In this paper, FPGA
in a decreased edge detection performance as discussed later synthesis results, including the resource utilization, execution
in this paper. The non-parallel implementation [7] computes time, and comparison with existing FPGA implementations
the low and high threshold values for each input image. are presented.
This results in increased latency as compared to the existing The rest of the paper is organized as follows. Section 2 gives
implementations (e.g., [4][6], [10][12]). Furthermore, the a brief overview of the original Canny algorithm. Section 3
non-parallel implementations ([4][7]) result in a decreased presents the proposed distributed Canny edge detection
throughput as compared to the parallel implementations algorithm which includes the adaptive threshold selection
([6], [10][12]). The issue of increased latency and decreased algorithm and a non-uniform quantization method to compute
throughput is becoming more significant with the increasing the gradient magnitude histogram. Quantitative conformance
demand for large-size high-spatial resolution visual content as well as subjective testing results are presented in Section 4
(e.g., High-Definition and Ultra High-Definition). in order to illustrate the edge detection performance of the
Our focus is on reducing the latency and increasing the proposed distributed Canny algorithm as compared to the
throughput of the Canny edge detection algorithm so that it can original Canny algorithm for clean as well as noisy images.
be used in real-time processing applications. As a first step, the In addition, the effects of the gradient mask size and the
image can be partitioned into blocks and the Canny algorithm block size on the performance of the proposed distributed
can be applied to each of the blocks in parallel. Unfortunately, Canny edge detection scheme are discussed and illustrated in
directly applying the original Canny at a block-level would fail Section 4. The proposed hardware architecture and the FPGA
since it leads to excessive edges in smooth regions and loss implementation of the proposed algorithm are described in
of significant edges in high-detailed regions. In this paper, Section 5. The FPGA synthesis results and comparisons with
we propose an adaptive threshold selection algorithm which other implementations are presented in Section 6. Finally,
computes the high and low threshold for each block based on conclusions are presented in Section 7.
the type of block and the local distribution of pixel gradients
in the block. Each block can be processed simultaneously, thus
II. C ANNY E DGE D ETECTION A LGORITHM
reducing the latency significantly. Furthermore, this allows
the block-based Canny edge detector to be pipelined very Canny developed an approach to derive an optimal edge
easily with existing block-based codecs, thereby improving detector to deal with step edges corrupted by a white Gaussian
the timing performance of image/video processing systems. noise. The original Canny algorithm [14] consists of the
Most importantly, conducted conformance evaluations and following steps: 1) Calculating the horizontal gradient G x and
subjective tests show that, compared with the frame-based vertical gradient G y at each pixel location by convolving with
Canny edge detector, the proposed algorithm yields better edge gradient masks. 2) Computing the gradient magnitude G and
detection results for both clean and noisy images. direction G at each pixel location. 3) Applying Non-Maximal
The block-based Canny edge detection algorithm is mapped Suppression (NMS) to thin edges. This step involves comput-
onto an FPGA-based hardware architecture. The architecture ing the gradient direction at each pixel. If the pixels gradient
is flexible enough to handle different image sizes, block sizes direction is one of 8 possible main directions (0, 45, 90,
and gradient mask sizes. It consists of 32 computing engines 135, 180, 225, 270, 315, the gradient magnitude of this
configured into 8 groups with 4 engines per group. All 32 com- pixel is compared with two of its immediate neighbors along
puting engines work in parallel lending to a 32-fold decrease the gradient direction and the gradient magnitude is set to zero
in running time without any change in performance when if it does not correspond to a local maximum. For the gradient
compared with the frame-based algorithm. The architecture directions that do not coincide with one of the 8 possible main
has been synthesized on the Xilinx Virtex-5 FPGA. It occupies directions, an interpolation is done to compute the neighboring
64% of the total number of slices and 87% of the local gradients. 4) Computing high and low thresholds based on
memory, and takes 0.721ms (including the SRAM read/write the histogram of the gradient magnitude for the entire image.
2946 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

Fig. 1. Block diagram of the Canny edge detection algorithm.

The high threshold is computed such that a percentage P1

of the total pixels in the image would be classified as strong
edges. In other words, the high threshold corresponds to the
point at which the value of the gradient magnitude cumulative
distribution function (CDF) equals to 1-P1 . The low threshold Fig. 2. (a), (f) Original Lena and Houses images; (b), (g) zoomed-in blocks of
is computed as a percentage P2 of the high threshold. The the Lena and Houses images; (c), (h) edge maps of block images in (b) and (g)
values of P1 and P2 are typically set as 20% and 40%, obtained by applying the classical Canny edge detector to blocks separately;
(d), (i) edge maps using the Canny edge detector applied to the entire Lena
respectively [14], [15]. 5) Performing hysteresis thresholding image and Houses image; (e), (j) zoomed-in blocks of the edge maps (d) & (i).
to determine the edge map. If the gradient magnitude of a pixel
is greater than the high threshold, this pixel is considered as
a strong edge. If the gradient magnitude of a pixel is between block, extracted from the Lena image and Houses image.
the low threshold and high threshold, the pixel is labeled as Despite the fact that the smooth block [Fig. 2(b)] does not
a weak edge. Strong edges are interpreted as certain edges, contain any significant edges, applying the original Canny
and can be immediately included in the final edge images. algorithm to this block results in the detection of an excessive
Weak edges are included if and only if they are connected to number of edges as shown in Fig. 2(c). This is due to the
strong edges. fact that the Canny algorithm assumes that a percentage P1
A block diagram of the Canny edge detection algorithm [14] of the total pixels in the block are strong edges. On the other
is shown in Fig. 1. In this original Canny edge detection hand, for an edge block [Fig. 2(g)], the majority of pixels are
algorithm [14], the gradient calculation (Step 1) is performed edges. Since P1 is much smaller than the actual percentage
by using Finite-Inpulse Response (FIR) gradient masks of edge pixels in the edge block, it results in missing edges
designed to approximate the following 2D sampled versions as shown in Fig. 2(h). For comparison, Fig. 2(d), (e), (i),
of the partial derivatives of a Gaussian function: and (j) show the performance of the Canny edge detector
x x 2 +y2 2
 2

1 y 22
 when applied to the entire image. In latter case, the Canny
x2
Fx (x, y) = 2 e 2 = xe e (1) edge detector is able to achieve the expected edge detection
2
performance in the smooth regions [Fig. 2(e)] and in the edge
y x 2 +y 2 2
1 x 22
   
y regions [Fig. 2(j)].
Fy (x, y) = 2 e 2 2 = ye 2 e (2)
2 In order to improve the performance of the edge detection
where is the standard deviation of the Gaussian function. at the block level and achieve the same performance as the
The size of the gradient masks used by the Canny edge original frame-based Canny edge detector when this latter
detector is usually implemented as a function of the chosen , one is applied to the entire image, a distributed Canny edge
with larger values of yielding larger masks. However, the detection algorithm is proposed. A diagram of the proposed
best choice of is image-dependent and can be selected by the algorithm is shown in Fig. 3. In the proposed distributed
user based on knowledge of the present noise characteristics or version of the Canny algorithm, the input image is divided
the size of desired objects in the image [16]. The parameter into m m overlapping blocks and the blocks are processed
can also be set by a separate application that estimates the independent of each other. For an L L gradient mask,
noise and/or scale of objects in the image. The effect of the the m m overlapping blocks are obtained by first dividing
gradient mask size is illustrated in Section 4.1.1. the input image into n n non-overlapping blocks and then
extending each block by (L + 1)/2 pixels along the left,
right, top, and bottom boundaries, respectively. This results
III. P ROPOSED D ISTRIBUTED C ANNY E DGE in m m overlapping blocks, with m = n + L + 1. The
D ETECTION A LGORITHM non-overlapping n n blocks need to be extended in order to
As discussed in Section 2, the classical Canny edge detec- prevent edge artifacts and loss of edges at block boundaries
tion algorithm sets the high and low thresholds based on the while computing the gradients and due to the fact that the
distribution of the gradients at all the pixel locations of an NMS operation at boundary pixels requires the gradient values
image. Thus, directly applying the original Canny algorithm of the neighboring pixels of the considered boundary pixels
to a block of the image would fail to detect the desired edges in a block. Fig. 4 shows an example of non-overlapping block
since the statistics of a block may differ significantly from and its extended overlapping block version in the case when a
the statistics of the entire natural image. For example, Fig. 2 3 3 gradient mask. In order to perform NMS for the border
shows two types of blocks, namely smooth block and edge pixel (i , j ), the gradient information of the adjacent pixels
XU et al.: DISTRIBUTED CANNY EDGE DETECTOR 2947

Fig. 5. (a) Original 512 512 Lena image; b) uniform block; (c) uniform/
texture block; (d) texture block; (e) edge/texture block; (f) medium edge
block of the Lena image. Shown blocks are of size 64 64.
Fig. 3. Proposed distributed Canny edge detection algorithm.

Fig. 4. An example of the structure of an m m overlapping block, where

m = n + L + 1 for an L L(L = 3) gradient mask. and when the image is Fig. 6. Normalized gradient magnitude CDFs for the 512 512 Lena image
initially divided into n n non-overlapping blocks. and CDFs for the 64 64 uniform block, uniform/texture block, texture block,
edge/texture block, medium/edge block shown in Fig. 5.

(i 1, j 1), (i 1, j ), (i 1, j +1), (i, j 1), (i +1, j 1) are

needed. In order to compute the gradient of the adjacent pixels edge/texture, medium edge block for the 512512 Lena image
(i 1, j 1), (i 1, j ), (i 1, j + 1), (i, j 1), (i + 1, j 1) with a block size of 64 64. Note that there are no strong
for the 3 3 gradient mask, the block has to be extended by edge blocks for the Lena image.
2 (where (L 1)/2 + 1 = 2) pixels on all sides in order to In order not to degrade the performance of the original
generate a block of size (n + 4) (n+4). Thus, m equals to frame-based Canny algorithm when it is applied to a block,
n+4 for this example. Note that, for each block, only edges the high and low thresholds when computed at the block
in the central n n non-overlapping region are included in level should match thresholds that would have been obtained
the final edge map, where n = m L 1. Steps 1 to 3 for the entire image. The gradient magnitude CDFs of each
and Step 5 of the distributed Canny algorithm are the same block in Fig. 5 are shown in Fig. 6,1 along with the gradient-
as in the original Canny algorithm except that these are now magnitude CDF of the entire Lena image. According to the
applied at the block level. Step 4, which is the hysteresis high CDF of the entire Lena image, the high threshold should
and low thresholds calculation, is modified to enable parallel be selected as 1.8 corresponding to a P1 value (percentage
block-level processing without degrading the edge detection of pixels that should be classified as strong edges) of 0.2.
performance. However, if P1 is still set to 0.2 for the medium edge block,
As discussed above, exploiting the local block statistics can the local high threshold for that block would be about 5.2,
significantly improve the edge detection performance for the which is significantly different from the high threshold, 1.8,
distributed Canny edge detector. For this purpose, in order to that is obtained for the entire image. Such a setting will result
learn the statistics of local blocks, we use a training database in a loss of significant edges in the considered strong edge
consisting of natural images. For each image in the database, block. On the other hand, the local high thresholds for the
we first divide the image into nn non-overlapping blocks and uniform and uniform/texture block would be about 0.7, if P1
classify the blocks into six types, uniform, uniform/texture, is set to 0.2. Such a setting will lead to excessive edges in the
texture, edge/texture, medium edge, and strong edge block, uniform and uniform/texture blocks. From this analysis, it can
by adopting the block classification method of [17]. This be seen that, in order to keep a similar threshold for all block
classification method utilized the local variance of each pixel types, P1 should be selected differently for different types of
using a 3 3 window that is centered around the con- blocks by adapting its value to the local block content.
sidered pixel in order to label it as of type edge, texture, In order to determine, for each block type, the appropri-
or uniform pixel. Then, each block is classified based on the ate percentage value P1 that would result in high and low
total number of edge, texture, and uniform pixels in the con- threshold values similar to the ones that are obtained for the
sidered block. Fig. 5 shows an example where the 512 512
Lena image is divided into blocks, and each block is classified
according to this classification technique. Fig. 5(b)(f) provide, 1 In Fig. 6, the horizontal and vertical gradients are calculated by using 33
respectively, examples of uniform, uniform/texture, texture, gradient masks and thus m equals to 68.
2948 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

TABLE II
P1 VALUES FOR E ACH B LOCK T YPE W ITH D IFFERENT B LOCK S IZES

Fig. 7. P1 values for each block type.

TABLE I
S TANDARD D EVIATIONS OF P1 VALUES FOR E ACH B LOCK
T YPE FOR 64 64 B LOCK

entire image, a training set2 of 200 images is formed from

Fig. 8. (a)-(c) Different types of 64 64 blocks ((a) Edge/texture,
the Berkeley Segmentation Image Database [18]. For each (b) medium edge, (c) strong edge) and (d) corresponding gradient magnitude
image in the training database, the high threshold of the entire histograms.
image is first calculated. Then, the image is divided into blocks
and the blocks are classified into six block types as discussed
previously. Then, for each block type, the gradient magnitude In order to compute the high and low hysteresis thresholds,
CDF is computed and the corresponding CDF used to compute a finely quantized gradient magnitude histogram is usually
the P1 value such that the local high threshold of the blocks needed. Here, we employ the non-uniform quantizer, which
in this class is the same as the one for the entire image. Fig. 7 has been proposed by us in [13], to obtain the gradient magni-
shows the P1 values that are obtained for different block types, tude histogram for each block such that the high threshold can
each of size 64 64, for 200 512 512 images with varied be precisely calculated. As in [7] and [13], for the edge/texture,
content. It illustrates that the P1 values of each block type medium edge and strong edge block, it was observed that
are highly different from each other, except for the uniform the largest peak in the gradient magnitude histograms of
block and uniform/texture block types. Also, for a given block the Gaussian-smoothed images occurs near the origin and
type, the P1 values across all 200 images are quite similar for corresponds to low-frequency content, while edge pixels form
different block sizes. The final P1 value for a considered block a series of smaller peaks, where each peak corresponds to
type is computed as the average value of its corresponding set a class of edges having similar gradient magnitudes. Conse-
over all images and over all block sizes. For illustration, the quently, the high threshold should be selected between the
standard deviations of the obtained P1 values for each block largest peak and the second largest edge peak. Examples of
type are shown in Table I for 64 64 blocks. gradient magnitude histograms for an edge/texture, medium
To evaluate the robustness of the obtained P1 values with edge and strong edge block are shown in Fig. 8. Based on the
respect to the block size, the 512512 images are divided into above observation, we proposed a non-uniform quantizer to
fixed-size blocks, with the block size varying from 8 8 to discretize the gradient-magnitude histogram in [13]. Specifi-
256 256. Table II shows the P1 values that are obtained for cally, the quantizer needs to have more quantization levels in
each block type and for each block size. It should be noted that the region between the largest peak A and the second largest
the P1 values for uniform and uniform/texture blocks are equal peak B and few quantization levels in other parts. Fig. 9
to 0 for all block sizes, which indicates that the uniform and shows the schematic diagram of the proposed non-uniform
uniform/texture blocks can be combined into one block type, quantizer. The first reconstruction level (R1 ) is computed as
which we refer to as smooth block type. Also, this implies the average of the maximum value and minimum value of the
that there are no pixels that should be classified as edges in gradient magnitude in the considered block, and the second
a smooth block. Therefore, there is no need to perform edge reconstruction level (R2 ) is the average of the minimum
detection on smooth blocks, and this results in reduced compu- value of the gradient magnitude and the first reconstruction
tations, a feature that is exploited in the FPGA implementation. level. Accordingly, n reconstruction levels can be computed as
shown in [13], R1 = (mi n +max)/2 and Ri+1 = (mi n + Ri )/2
2 In order to accommodate various block sizes that are integer powers of 2, (i = 2, 3, . . . n), where mi n and max represent the minimum
the size of images in the dataset was extended from the original size of and maximum values of the gradient magnitude, respectively,
321 418 to 512 512. and Ri is the reconstruction level.
XU et al.: DISTRIBUTED CANNY EDGE DETECTOR 2949

Fig. 11. (a) Original Houses image; Edge map using the original Canny
algorithm (b) with a 9 9 ( = 1.4) gradient mask and (c) with a 3 3
( = 0.4) gradient mask.

what is the best choice for the mask size for different types of
images, and what is the smallest block size that can be used by
our proposed distributed Canny algorithm without sacrificing
Fig. 9. Reconstruction values and quantization levels. performance.
1) The Effect of Mask Size: As indicated in Section 2,
the size of the gradient mask is a function of the standard
deviation of the Gaussian filter, and the best choice of
is based on the image characteristics. Canny has shown
in [14] that the optimal operator for detecting step edges in
the presence of noise is the first derivative of the Gaussian
operator. As stated in Section 2, for the original Canny
algorithm as well as the proposed algorithm, this standard
deviation is a parameter that is typically set by the user based
on the knowledge of sensor noise characteristics. It can also be
set by a separate application that estimates the noise and/or
blur in the image. A large value of results in smoothing
and improves the edge detectors resilience to noise, but it
undermines the detectors ability to detect the location of true
edges. In contrast, a smaller mask size (corresponding to a
lower ) is better for detecting detailed textures and fine edges
but it decreases the edge detectors resilience to noise.
An L-point even-symmetric FIR Gaussian pulse-shaping
filter design can be obtained by truncating a sampled version of
the continuous-domain Gaussian filter of standard deviation .
The size L of the FIR Gaussian filter depends on the standard
deviation and can be determined as follows:

L = 2 L side + 1 (3)
  
L side = 2 (1) log(C T ) (4)

where C T represents the cut-off value in the spatial domain

of the continuous-domain Gaussian function and determines
the cut-off error. A smaller C T corresponds to a smaller
cut-off error. In our implementation, in order to make the
cut-off error small for all , C T is chosen to be a small
value (e.g., C T = 103 ). Fig. 11 shows that the relatively
smaller size 3 3 filter ( from 0.3 to 0.4) can detect the
Fig. 10. Pseudo-codes for the proposed (a) block classification and edges of a detailed texture image, while Fig. 12 shows that
(b) adaptive threshold selection scheme.
the larger 9 9 filter ( from 1.2 to 1.4) suppresses noise
more effectively. In addition, Fig. 13 shows that the 3 3 filter
The pseudo-code of the block classification technique is exhibits a better performance ingenerating the edge maps
in [17] and the proposed adaptive threshold selection algorithm of blurred images as compared to the 9 9 filter.
is shown in Fig. 10(a) and (b), respectively. 2) Block Size: To find out the smallest block size for which
the proposed Canny algorithm can detect all the psycho-
IV. MATLAB E XPERIMENTAL R ESULTS visually important edges, the perceptual visual quality of
the obtained edge maps was assessed using visual quality
A. Parametrical Analysis metrics. More specifically, the quality of edge maps obtained
The performance of the proposed algorithm is affected by by using different block sizes was analyzed by evaluating their
two parameters, the mask size and the block size. We discuss corresponding Pearsons correlation coefficient (PCC), which
2950 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

Fig. 12. (a) Houses image with Gaussian white noise (n = 0.01); Edge
map using the original Canny algorithm (b) with a 9 9 ( = 1.4) gradient Fig. 15. (a) 512 512 Salesman image; edge-maps of (b) the original
mask and (c) with a 3 3 ( = 0.4) gradient mask. Canny edge detector, and (c) the proposed algorithm with a non-overlapping
block size of 64 64, using a 3 3 gradient mask.

Fig. 13. (a) Gaussian Blurred Houses image (blur = 2); Edge map using
the original Canny algorithm (b) with a 9 9 ( = 1.4) gradient mask and Fig. 16. (a) 512 512 Fruit image; edge-maps of (b) the original Canny
(c) with a 3 3 ( = 0.4) gradient mask. edge detector, and (c) the proposed algorithm with a non-overlapping block
size of 64 64, using a 3 3 gradient mask.

Fig. 17. (a) 512 512 Houses image; edge-maps of (b) the original Canny
edge detector, and (c) the proposed algorithm with a non-overlapping block
size of 64 64, using a 3 3 gradient mask.

Fig. 14. Pearsons correlation coefficient reflecting the correlation between

the CPBD sharpness metric [19] and MOS as a function of block size and
mask size.

was used to measure how well state-of-the-art edge-based

sharpness metrics commensurate with the actual perceived
sharpness. For this purpose, the psycho-visual sharpness met- Fig. 18. (a) 512 512 Lena image; edge-maps of (b) the original Canny
edge detector, and (c) the proposed algorithm with a non-overlapping block
ric of [19] is used, which primarily depends on the edge size of 64 64, using a 3 3 gradient mask.
map to estimate the cumulative probability of detecting blur
(CPBD) in an image. This metric was shown in [19] to achieve
the best performance among existing sharpness/blur metrics. for the various block sizes, the increase or decrease in the PCC
It calculates the probability of detecting blur at each edge is insignificant. In other words, the PCC is robust to changes
in an image and then obtains a final quality score over the in block size. In conclusion, the proposed distributed Canny
entire image by evaluating the cumulative probability of blur algorithm can detect all the psycho-visually important edges
detection. Four images with varying image characteristics were for images with moderate noise and blur levels, similar to the
chosen. These were blurred using Gaussian masks with six original frame-based Canny algorithm.
different values (0.5, 1, 1.5, 2, 2.5, and 3) to generate a set
of 24 images. The images were displayed one after another
in a random order and subjects were asked to rate them on B. Edge Detection Performance Analysis
a 5-point scale of 1 (poor) to 5 (excellent) to form a Mean The edge detection performance of the proposed distributed
Opinion Score (MOS) by averaging the subjective scores for approach is analyzed by comparing the perceptual significance
each image. Fig. 14 depicts the effect of replacing the frame- of its resulting edge map with the one produced by the
based Canny edge detection with the proposed distributed original frame based Canny edge detector. Figs. 1518 show
Canny edge detection on the sharpness metric of [19], for the edge maps that are obtained by the original frame-based
masks of different sizes and for different block sizes. Note Canny edge detector and the proposed distributed Canny edge
that a 512 512 block corresponds to the original frame-based detection algorithm with 3 3 gradient masks and a non-
Canny edge detector. Fig. 14 shows that PCC has a maximal overlapping block size of 64 64 (n = 64; m = 68) for the
value when the block size is 64 64. It also illustrates that, 512 512 Salesman, Fruit, Houses, and Lena images,
XU et al.: DISTRIBUTED CANNY EDGE DETECTOR 2951

TABLE III
C ONFORMANCE E VALUATION

algorithm that were not detected by the proposed distributed

Canny edge detection algorithm also referred to as false
negatives) and P f a (percentage of edge pixels detected by
the proposed Canny edge detection algorithm that were not
detected by the original Canny edge detection algorithm, also
referred to as false positives), as proposed in [12], are used
for the conformance test. Here, the edge map generated by the
original frame-based Canny for the clean image is considered
as the reference for both the clean and noisy cases. This
is compared with the edge map generated by the proposed
algorithm for the same clean image, as well as the edge maps
generated by these two algorithms (original Canny and pro-
posed) for the noisy version of the same image. Experiments
Fig. 19. Comparison of the edge maps of noisy images by using the original were conducted using the USC SIPI Image Database [20] and
Canny edge detector and the proposed method: (a) images with Gaussian the Standard Test Image Database [21]. The results are shown
white noise (n = 0.01); edge-maps of (b) the original Canny edge detector,
and (c) the proposed algorithm with a non-overlapping block size of 64 64,
in Table III.
using a 9 9 gradient mask. For clean images, the detected edge pixel percentage (Pco )
was higher than 94.8%. This further validates our claim that
the proposed distributed algorithm detects almost all edges that
respectively. As shown in Figs. 15 and 16, the proposed are detected by the original Canny edge detection algorithm.
distributed Canny edge detection algorithm yields comparable For noisy images, as shown in Table III, the Pco of the
edge detection results as compared with the original frame- proposed distributed Canny algorithm is higher than that of
based Canny edge detector. From Figs. 17 and 18, it can be the original Canny algorithm, and the error (the sum of
seen that, in contrast to the original frame-based Canny, the Pnd and P f a ) of the proposed distributed Canny algorithm
proposed distributed Canny algorithm can detect edges in low is much lower than the original Canny detector. This validates
contrast regions since it is able to adapt to the local statistical our earlier observation that, for noisy image, the edge detection
distribution of gradient values in a block. Furthermore, Fig. 19 performance of the proposed distributed Canny algorithm is
shows the edge maps of images with white Gaussian noise better than the original frame-based Canny algorithm.
generated by the original frame-based Canny edge detector 2) Subjective Testing: Subjective tests were conducted by
and the proposed distributed Canny edge detection algorithm having human subjects evaluate the quality of the detected
with 3 3 gradient masks and a non-overlapping block size edge maps that are generated by the proposed algorithm and
of 64 64. Comparing the edge maps in Fig. 19(b) and (c) the original Canny for both clean and noisy images, without
with the clean edge maps in Figs. 1518, it can be seen that, the subjects knowing which algorithm produced which edge
the proposed Canny edge detection algorithm is more robust maps, using images from the SIPI Database [20] and the
to noise than the original frame-based Canny. Standard Test Image Database [21].
To further assess the performance of the proposed dis- The edge maps of clean images obtained by the original
tributed Canny algorithm, quantitative conformance evalua- Canny algorithm and the proposed algorithm were displayed
tions and subjective tests are performed. The conformance simultaneously for comparison together with the original
evaluations aim to evaluate the similarity between edges image, which was displayed in the middle. Fig. 20 shows a
detected by the original frame-based Canny algorithm and snapshot of the subjective test interface. This was done for
the proposed distributed Canny edge detection algorithm, both clean and noisy images. Experiments were conducted
while the subjective tests aim to validate whether the edge using a 19-inch DELL LCD monitor having a resolution of
detection performance of the proposed distributed Canny is 1024 1280. The average room luminance is measured to
better, worse, or similar to the original frame-based Canny as be 30 cd/m2 . The subjects were asked to rate the quality
perceived by subjects. of the displayed edge maps using a five-point scale with
1) Conformance Evaluation: In order to quantify the sim- scores 1 to 5 corresponding, respectively, to the rightmost edge
ilarity of two edge maps, three metrics, Pco (percentage of map is worse, slightly worse, same, slightly better,
edge pixels detected by both implementations) Pnd (percentage and better compared to the edge map shown on the left.
of edge pixels detected by the original Canny edge detection These scores were then automatically converted internally
2952 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

Fig. 22. Block diagram of the embedded system for the proposed algorithm.
Fig. 20. Snapshot of the performed subjective test comparing the edge maps
generated by the original Canny algorithm and the proposed algorithm.

Fig. 21. Subjective scale where scores 1, 2, 3, 4, & 5 correspond to worse,

slightly worse, same, slightly better, and better, with respect to the
original Canny algorithm.

by the interface into five-point scale scores from 1 to 5

corresponding, respectively, to the edge map produced by the
proposed distributed Canny algorithm is worse, slightly
worse, same, slightly better, and better compared to the
original Canny algorithm (see Fig. 21). Thus, a score higher
than 3 indicates that the generated edge map by the proposed
algorithm is better than the one obtained using the original
Fig. 23. The architecture of the proposed distributed Canny algorithm.
Canny algorithm and vice versa. The images are randomly
displayed. Each case is randomly repeated four times, with
the left and right images swapped in a random fashion, to external Static RAMs (SRAM) & memory controllers, and an
obtain better subjective response statistics [22]. Ten subjects intellectual property (IP) design for the proposed distributed
with normal to corrected-to-normal vision took the test. Two Canny detection algorithm. The embedded micro-controller
Mean Opinion Scores (MOS) were computed by averaging the coordinates the transfer of the image data from the host
subjective scores over the set of clean images and over the set computer (through the PCIe (or USB) controller, system local
of noisy images, respectively. The obtained MOS values were bus, and memory controller) to the SRAM; then from the
3.69 (slightly better than the original Canny) for the clean SRAM to the local memory in the FGPA for processing
image set and 4.6 (better than the original Canny) for the and finally storing back to the SRAM. Xilinx and Altera
noisy image set. Results from the subjective tests show clear offer extensive libraries of intellectual property (IP) in the
preference of the subjects for the edge maps that are generated form of embedded micro-controllers and peripherals controller
by the proposed algorithm for both clean and noisy images. [23], [24]. Therefore, in our design, we focused only on
These results indicate that the proposed algorithm results in a the implementation of the proposed algorithm on the Xilinx
superior performance as compared to the original frame-based Virtex-5 FPGA and the data communication with external
Canny algorithm. This is partly due to the capability of the SRAMs. These components have been highlighted in Fig. 22.
proposed algorithm to adapt to the local image characteristics In the rest of this section, we provide a high-level overview
since the P1 value is selected differently for different types of of the proposed IP design (Section 5.1) followed by details of
blocks by adapting its value to the local block content. each of the units in the computing engine (Sections 5.2). The
FPGA synthesis results are presented in Section 6.
V. FPGA I MPLEMENTATION OF THE P ROPOSED
D ISTRIBUTED C ANNY E DGE D ETECTION A LGORITHM A. Architecture Overview
In order to demonstrate the parallel efficiency of the pro- The proposed architecture, shown in Fig. 23, consists of
posed distributed Canny edge detection algorithm, we describe q processing units (PU) and external dual-port Static RAMs
an FPGA-based hardware implementation of the proposed (SRAMs). In Fig. 23, each PU consists of p computing engines
algorithm in this section. Fig. 22 gives a birds eye view of the (CE), where each CE processes an m m overlapping image
embedded system for implementing the distributed Canny edge block and generates the edges of an n n block, where
detection algorithm based on an FPGA platform. It is com- m = n + L + 1 for an L L gradient mask. The dataflow
posed of several components, including an embedded micro- through this architecture is as follows. For each PU, the SRAM
controller, a system bus, peripherals & peripheral controllers, controller fetches the input data from SRAM and stores them
XU et al.: DISTRIBUTED CANNY EDGE DETECTOR 2953

Fig. 24. High throughput memory interface.

Fig. 25. Block diagram of CE (computing engine for edge detection).

into the input local memory in the PU. The CEs read this
data, process them and store the edges into the output local The customized memory interface, shown in Fig. 24, has a
memory. Finally, the edges are written back to the SRAM one 2b-bit wide internal data-bus. In our application, the dual-
output value at a time from the output local memory. port SRAM, the memory interface and the local memories,
In order to increase the throughput, the SRAM external which connect with the SRAM interface, operate at the same
memory is organized into q memory banks, one bank per PU. frequency, which is f S R AM MHz.
Since only one b-bit data, corresponding to one pixel value,
can be read from a SRAM at a time, such an organization helps B. Computing Engine (CE)
multiple PUs to fetch data at the same time and facilitates As described before, each CE processes an m m over-
parallel processing by the PUs. For an image of size N N, lapping image block and generates the edges of an n n
each SRAM bank stores a tile of size N 2 /q image data, where non-overlapping block. The computations that take place in
the term tile refers to an image partition containing several CE can be broken down into the following five units: 1) block
non-overlapping blocks. The SRAM bank is dual-port so that classification, 2) vertical and horizontal gradient calculation as
the PUs can read and write at the same time. well as magnitude calculation, 3) directional non-maximum
In order to maximize the overlap between data read/write suppression, 4) high and low threshold calculation, and
and data processing, the local memory in each PU is imple- 5) thresholding with hysteresis. Each of these units is mapped
mented using dual port block RAM (BRAM) based ping- onto a hardware unit as shown in Fig. 25 and described in
pong buffers. Furthermore, in order that all the p CEs can the following subsections. The communication between each
access data at the same time, the local memory is organized component is also illustrated in Fig. 25 and will be described
into p banks. In this way, a total of pq overlapping blocks in detail in the following subsections.
can be processed by q groups of p CEs at the same time. Suppose the input image data has been stored in the external
The processing time for an N N image is thus reduced memory (SRAMs). For each PU, once the ping-pong buffers
approximately by a factor of pq. If there are enough hardware have loaded p m m overlapping image blocks, which we
resources to support more CEs and more PUs, the throughput refer to as a Group of Blocks (GOB), from the SRAM, all the
would increase proportionally. p CEs can access block data from ping-pong buffers at the
However, FPGAs are constrained by the size of on-chip same time. For each CE, the edge detection computation can
BRAM memory, number of slices, number of I/O pins; and start after n m m overlapping block is stored in CEs local
the maximum throughput achievable is a function of these memories. In addition, in order to compute the block type,
parameters. Since each CE processes an m m overlapping vertical gradient and horizontal gradient in parallel, the m m
image block, for a b-bit implementation, this would require overlapping block is stored in three local memories, marked
3 m m b bits to store the vertical and horizontal as local memory 1, 2 and 3 as shown in Figs. 26 and 27.
gradient components and the gradient magnitude, as discussed
1) Block Classification: As stated before, the m m over-
in Section 5.2. To enable parallel access of these blocks, there
lapping block is stored in the CEs local memory 1 and is
are three BRAMs of size m m b in each CE. In addition, used for determining the block type. The architecture for the
2 p m m b bits are needed for each of the ping-pong
block classification unit consists of two stages as shown in
buffers. Therefore, for each PU, 3 p m m b bits are
Fig. 26. Stage 1 performs pixel classification while stage 2
need for p CEs and 4 p m m b bits are needed for the performs block classification. For pixel classification, the local
input and output ping-pong buffers. This results in a total of
variance of each pixel is utilized and the variance is calculated
7 p m m b q = 7 pqm 2b bits for the FPGA memory. as follows:
Thus, if there are more CEs and/or larger sized block, more
9
FPGA memory is required. Similarly, if there are more PUs, 1
more I/O pins are required to communicate with the external var = (x i x)2 (5)
8
i=1
SRAM memory banks. Thus, the choice of p and q depends
on the FPGA memory resources and numbers of I/O pins. where x i is the pixel intensity and x is the mean value of
We do not consider the numbers of slices as a constraint since the 3 3 local neighborhood. Thus, the pixels in the 3 3
the number of available slices is much larger than required by windows are fetched from the local memory and stored in one
the proposed algorithm. FIFO buffer to compute the local variance. The computation
2954 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

Fig. 26. The architecture of the block classification unit.

Fig. 27. Gradient and Magnitude Calculation Unit: (a) architecture; (b) executing time.

is done using one adder, two accumulators, two multipliers high and low threshold calculation and thresholding with
and one square (right shift 3 bits to achieve multiplication hysteresis units. Otherwise, these units do not need to be
by 1/8). Next, the local variance is compared with Tu and activated and the edge map with all zero value pixels is stored
Te [17] in order to determine pixel type. Then two counters back into the SRAM. The P1 and EN are the outputs for
are used to get the total number of pixels for each pixel type. the block classification unit and are stored in the registers
The output of counter 1 gives C1, the number of uniform for thresholding calculation. The latency between the first
pixels, while the output of counter 2 gives C2, the number of input and the output P1 is m m +12 clock cycles and the
edge pixels. The block classification stage is initialized once total execution time for the block classification component is
the C1 and C2 values are available. C2 is compared with m m +13.
0 and the result is used as the enable signal of COMP 5, 2) Gradient and Magnitude Calculation: Since the gradient
COMP 6 and MUX 2. C1 and C2 are compared with different and magnitude calculation unit is independent of the block
values as shown in Fig. 10(a), and the outputs are used as classification unit, these two components can work in parallel.
the control signals of MUX 1 and MUX 2 to determine the In addition, the horizontal gradient and vertical gradient can
value of P1 . Finally, the P1 value is compared with 0 to also be computed in parallel. The m m overlapping block
produce the enable signal, marked as EN. If the P1 value is also stored in the CEs local memory 2 and 3 and is
is larger then 0, then EN signal enables gradient calculation, used as input to compute the horizontal and vertical gradient,
magnitude calculation, directional non-maximum suppression, respectively. The architecture for the gradient and magnitude
XU et al.: DISTRIBUTED CANNY EDGE DETECTOR 2955

calculation unit is shown in Fig. 27(a). It consists of three com-

putation parts (horizontal gradient calculation, vertical gradient
calculation and magnitude calculation) and one address and
time controller, which provides the address and control signals
to coordinate the computation. The execution timing of this
unit is shown in Fig. 27(b).
For the horizontal and vertical gradient calculation, the input
block image is convolved with the 2D horizontal and vertical
gradient masks, respectively; the size of these masks can be
chosen based on the features of the image. These 2D gradient
masks are separable, so the implementation of 2D convolution
is achieved by two 1D convolutions. To implement the 1D
convolution, we adopt Xilinxs FIR IP core, which provides
a highly parameterizable, area-efficient implementation that
utilizes the symmetry characteristics of the coefficient set [25].
Thus, each 2D convolution operation is implemented with two
1D FIR cores, resulting in a total of four 1-D FIR cores for
both the horizontal and vertical gradient computations. The
results of the rowwise 1-D convolutions for the horizontal
Fig. 28. Directional non maximum suppression unit: (a) mechanism;
and vertical gradient computations are stored back into local (b) architecture.
memory 2 and 3, respectively, and are used as input for
the columnwise 1D convolutions. Since the row and column
access times are the same for BRAM, the row-filtered block
image data does not need to be transposed before applying
the column filter. The FIR filter outputs are the input to the
magnitude calculation unit which computes the magnitude
of the gradient from the horizontal and vertical gradients
for each pixel. The magnitude computation consists of two
multipliers and one square-root computation module. The
square-root function is implemented using the CORDIC 4.0
IP core of the Xilinx core generator with the optimized
pipeline mode. In addition, the unsigned fraction data format
was used and the round mode was chosen as truncate. Finally,
the gradient magnitude is stored back into local memory 1
for further computation. The maximum and minimum values
of the magnitude, marked as mag max and mag mi n, are
the outputs of this unit and used as the input of the threshold
calculation unit. The execution timing for this unit is illustrated
in Fig. 27(b). Here, L F I R is the latency of Xilinx FIR IP core. Fig. 29. The architecture of thresholds calculation unit.
A Mealy Finite State Machine (FSM) is used in the timing
controller module since it requires less number of states and
is faster than a Moore FSM. along the direction of the gradient into the arithmetic unit.
3) Directional Non Maximum Suppression (NMS): The This arithmetic unit consists of one divider, two multipliers
horizontal and vertical gradient and the gradient magnitude and one adder, which are also implemented by the Xilinx
are fetched from local memory 2, 3 and 1, respectively; Math Functions IP cores. Finally, the output of the arithmetic
and used as input to the directional NMS unit. Fig. 28(a) unit is compared with the gradient magnitude of the center
and (b) show the mechanism and the architecture of the pixel. The gradient of the pixel that does not correspond
directional NMS, respectively. As described in Section 2, to a local maximum gradient magnitude is set to zero. The
this step involves computing the gradient direction at each final gradient magnitude after directional NMS [marked as
pixel. According to the sign and value of the horizontal Mag N M S(x, y) in Fig. 28(b)] is stored back into local
gradient G x and the vertical gradient G y , the direction can be memory 1 as the input for the hysteresis thresholding unit. The
determined; and thus the gradient magnitudes of four nearest latency between the first input and the first output is 20 clock
neighbors along the direction are selected to compute two cycles and the total execution time is m m +20.
intermediate gradients M1 and M2 as shown in Fig. 28(a). 4) Calculation of Thresholds: Since the gradient magni-
The corresponding architecture is shown in Fig. 28(b). The tude of each pixel after directional NMS is used as the
horizontal gradient component G x and the vertical gradi- input for the calculation of thresholds, this unit can be
ent component G y control the selector which forwards the pipelined with the directional NMS unit. Beside, the P1
magnitude [marked as M(x, y) in Fig. 28(b)] of neighbors value, which is determined by the block classification unit,
2956 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

Fig. 30. Pipelined architecture of thresholding with hysteresis.

Fig. 31. Execution time of the 512 512 image.

the mag max, and mag mi n, which are determined by

the gradient and magnitude calculation unit, are the inputs
for this unit. As discussed in Section 3, the 8-step non-
uniform quantizer is employed to obtain the discrete CDF;
and the high and low thresholds are calculated based on
this discrete CDF. The corresponding architecture is shown
in Fig. 29. The adders and shifters are used to obtain the Fig. 32. Comparison between the fixed-point Matlab software simulation
reconstruction levels Ri , and the comparators (1 to 8) and result and FPGA hardware simulation result for detecting edges of the
counters (1 to 8) are used to count the corresponding numbers 512 512 Houses image using a block size of 64 64 and a 3 3 gradient
mask: (a) edge map generated by the fixed-point MATLAB implementation;
of pixels, noted by No Pi xels Ri , which have a gradient (b) edge map generated by the FPGA implementation.
magnitude equal to or less than each reconstruction level Ri .
The P1 value, which is determined by the block classification
unit, is multiplied with the total number of pixels in the requirement of the Gaussian filter parameters, which is typi-
block. The obtained pixel number, noted by No Pi xels P1 , cally in the order of 0.001.
is compared with each No Pi xels Ri in order to select the To store grayscale images, we used the SRAM
level i , where No Pi xels Ri is closest to the No Pi xels P1 . (CY7C0832BV) [27]. This is a dual ported SRAM with
According to the selected level i , mag max, and mag mi n, 110 pins. The Xilinx Virtex-5 FPGA (XC5VSX240T) has
the arithmetical unit can compute the corresponding Ri , which 960 I/O pins and so, to satisfy the I/O pin constraint, the
is the high threshold T h H . Finally, the low threshold T h L is maximum number of PUs is 8 (q = 8). The local memory
computed as 40% of the high threshold. The latency of the on the FPGA for a block size of 64 64, which is needed
critical path is m m + 6 clock cycles. to support 8 PUs, is equal to 7 pqm 2b = 4046 p Kbits (see
5) Thresholding With Hysteresis: The gradient magnitude of Section 5.1), for q = 8, m = 68 (for a 64 64 block size and
each pixel after directional NMS is fetched from local mem- a 3 3 gradient mask size), and b = 16. Since the available
ory 1 and used as input to the thresholding unit. Meanwhile, memory resource on the FPGA is 18,576 Kbits, the p value
the T h H and T h L , which are determined by the threshold cal- using the memory constraint is determined to be 4. The
culation unit, are also the inputs for this unit. Fig. 30 illustrates p value could have also been constrained by the number of
the pipelined design of this stage, where f 1 represents a strong available slices. Since the number of slices for the considered
edge pixel while f2 represents a weak edge pixel. If any of FPGA is very large (37 440) and since each CE only utilizes
the neighbors of the current pixel is a strong edge pixel, the a small slice percentage (as shown later in Section 5.3.2),
center weak edge pixel is then considered as a strong edge the local memory resource in each PU constrains p, the
pixel; otherwise, it is considered as a background non-edge number of CEs in each PU, and not the numbers of slices.
pixel. The latency between the first input and the first output Taking all this into consideration, our design has q = 8 PUs
is 10 cycles and the total execution time for the hysteresis and each PU has p = 4 CEs. This design is coded in Verilog
thresholding unit is m m +10 cycles. and synthesized on a Xilinx Virtex-5 device (XC5VSX240T)
using the Xilinxs ISE software and verified using Modelsim.
VI. S YNTHESIS R ESULTS According to the Place and Route synthesis report, our
The proposed FPGA-based architecture can support multi- implementation can achieve an operating frequency of
ple image sizes and block sizes. To demonstrate the perfor- 250 MHz. But we choose 100 MHz to support a pipelined
mance of the proposed system, a Xilinx Virtex-5 FPGA [26] implementation of SRAM read/write and CE processing as
was used to process grayscale images with a block size of described later in Section 6.2.
64 64. The data width is 16 bits (Q8.7) with 8 bits to
represent the integer part since the maximum gray value of A. Resource Utilization
the image data is 255, and 7 bits to represent the fractional The FPGA resource utilization in each CE is listed in
part since the Gaussian filter parameters are decimals. Our Table IV, followed by the resource utilization for each PU
analysis shows that 7 bits are sufficient to meet the accuracy in Table V. The FIR IP core and the arithmetic functions are
XU et al.: DISTRIBUTED CANNY EDGE DETECTOR 2957

TABLE IV unit is pipelined with the directional NMS unit and the latency
R ESOURCE U TILIZATION ON XC5VSX240T FOR 1 CE of the NMS unit is 20 clock cycles. This is referred to as
T N M S in Fig. 31. TT C represents the latency of the thresholds
calculation stage and is equal to 4630 cycles, while TT H
represents the latency of the thresholding with hysteresis stage
and is equal to 4634 cycles. Table VII shows the latency for
each unit. Therefore, one CE takes TC E = TG R AD + TF I R +
T N M S + TT C + TT H = 18548 cycles.
TABLE V Each PU takes 18,496 cycles to load 4 68 68 overlapping
R ESOURCE U TILIZATION ON XC5VSX240T FOR 1 PU blocks from the SRAM into the local memory. It also takes
16,384 (4 64 64 non-overlapping blocks) cycles to write
final edge maps into SRAM. If SRAM operates at f S R AM , the
SRAM read time is 18,496/ f S R AM . The CE processing time
equals to 18,548/ f C E when CE is clocked at f C E . In order
TABLE VI to completely overlap communication with computation and
R ESOURCE U TILIZATION ON XC5VSX240T avoid any loss of performance due to communication with
FOR AN 8-PU A RCHITECTURE SRAM, given a fixed f S R AM , fC E should be selected such
that the processing time is approximately equal to the SRAM
read time (since the SRAM write time is less than the read
time). Thus, 18,496/ f S R AM = 18,548/ f C E , and the f C E can
be set to be 1.003 times higher than f S R AM .
The maximum speed of the employed SRAM device
(CY7C0832BV) is 133MHz. However, we choose the SRAM
TABLE VII clock rate as f S R AM = 100 MHz to allow for sufficient design
C LOCK C YCLES FOR E ACH U NIT margin. Thus, f C E 100 MHz, which is lower than the
maximum operating frequency (250 MHz) of the used FPGA
according to the synthesis report. The total computation period
for one CE is TC E = 18, 548/105 0.186 ms when clocked at
100 MHz. Thus, for a 512 512 image, the total computation
implemented using DSP48Es. The on-chip memory is imple- period is Tcom = 0.372ms; while the total execution time,
mented using BRAMs. Table VI summarizes the resource including the SRAM read/write time and the computation time,
utilization of the 8-PU architecture. It shows that the 8-PU is Tt ot al = (18, 496 + 16, 384)/1E 5 + 0.186 2 = 0.721 ms.
architecture occupies 64% of the slices and 87% of the BRAM The simulation results also show that, at a clock frequency of
memory. 100MHz, the execution time for processing 512 512 images
is 0.721ms for the images in the USC SIPI database.

B. Execution Time
Fig. 31 shows the pipeline implementation of SRAM C. FPGA Experiments and Results
read/write with the CE computation, where each SRAM bank 1) Conformance Tests: In order to validate the FPGA
stores a tile of size N 2 /q image data and each ping or pong generated results, two conformance tests are performed. One
buffer stores a group of blocks (GOB) of size p m m aims to evaluate the similarity between edges detected by
image data. Since our design has q = 8 PUs, one SRAM can the fixed-point FPGA and Matlab implementation of the
hold a tile of size 32,768 (64 64 8) image data for a distributed Canny edge detection algorithm. The other is to
512 512 image. In addition, for p = 4 CEs and for a 64 measure the similarity between edges detected by the fixed-
64 block size (n = 64; m = 68), the image data stored in the point FPGA and the 64-bit floating-point Matlab implemen-
SRAM result in two GOBs. These GOBs can be pipelined. tation of the distributed Canny edge detection algorithm. Our
As shown in Fig. 31, while GOB 2 is loaded to the ping-pong results for both tests were performed on the USC SIPI image
buffers, the CEs process GOB 1. Also, while GOB 1 is written database.
back into SRAM, the CEs process GOB 2 at the same time. For the first test, the difference between the edges detected
Such a pipelined design can increase throughput. by the fixed-point FPGA and Matlab implementations is
Fig. 31 also shows the computation time of each stage in calculated. Fig. 32 shows an example of the obtained fixed-
a CE during the processing of an m m overlapping block point Matlab and FPGA results for the Houses image using
(m = 68 for a 64 64 block and a 3 3 gradient mask). the proposed algorithm with a 64 64 block size and 3 3
As shown in Fig. 31, TBC , the time to classify the block gradient masks. The FPGA simulation result is obtained using
type, is less than TG R AD , the time for Gradient and Magnitude Modelsim and assumes the original image data has been stored
calculation, which equals to 9248 clock cycles. TF I R , the FIR in SRAMs. It can be seen that the two edge maps are the same.
filter computation latency equals to 8 clock cycles for a 3 3 Furthermore, the quantitative difference between the FPGA
FIR separable filter. The high and low thresholds calculation simulation result and the fixed-point Matlab simulation result
2958 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

TABLE VIII TABLE IX

RUNTIME C OMPARISON ON VARIOUS P LATFORMS C OMPUTATION T IME OF THE P ROPOSED D ISTRIBUTED C ANNY FPGA
FOR L ENA I MAGE I MPLEMENTATION U SING A N ON -OVERLAPING B LOCK S IZE OF
64 64 AND A 3 3 G RADIENT M ASK AND OF E XISTING
F RAME -BASED C ANNY FPGA I MPLEMENTATIONS

is zero. The same results were obtained for all 26 images in

dataset.
In order to verify similarity of the edges generated by the
fixed-point FPGA and 64-bit floating-point Matlab implemen-
tation of our proposed algorithm, the Pco , Pnd and P f a , met- TABLE X
rics as described in Section 4.2.1, were used. For all images, RUNTIMES OF THE GPGPU AND O UR 32-CE FPGA I MPLEMENTATIONS
the correctly detected edge pixel percentage (Pco ) was higher
than 98.2%. Furthermore, less than 1.8% of all edge pixels
were detected by only one of the detectors (Pnd and P f a ).
This leads to the conclusion that the hardware implementation
of our proposed algorithm can successfully detect significant
edges.
2) Performance Test: Furthermore, to show the efficiency
of the FPGA-based system, we compared the FPGA imple-
mentation of the original frame-based Canny edge detector
This illustrates the efficiency of the proposed distributed
with an assembly optimized CPU implementation of the
Canny edge detector.
original Canny edge detector available in Intels OpenCV
3) Comparison With Other Existing GPU and FPGA
library, which makes good use of thread parallelization and
Implementations: A long standing challenge for the use of
SSE instructions. Also, to show the efficiency of the FPGA
the Canny edge detection algorithm for real-time applications
implementation of our proposed distributed Canny algorithm,
is its high computational complexity. Several FPGA and GPU
the proposed FPGA distributed Canny implementation is com-
implementations have been proposed to meet the requirements
pared with the FPGA and CPU implementations of the original
of real-time applications. Comparisons between our proposed
frame-based Canny edge detector. The two candidate hardware
FPGA implementation and other existing FPGA and GPU
platforms were:
implementations are discussed below.
CPU: Intel Core i7-975, four 3.33G cores, 8192KB cache As far as we know, the existing FPGA-based approaches
and 12GB RAM; implemented the frame-based Canny algorithm and only the
FPGA: Xilinx SXT Virtex-5, 37,440 slices, 1,056 computation time for detecting edges are presented in papers
DSP48E slices and 18,576 Kbits BRAM. [4][6]. The computation time and resource utilization results
Runtime results are shown in Table VIII for the standard of these frame-based Canny FPGA implementations and
Lena image. The absolute runtimes of the two algorithms our distributed Canny FPGA implementation are shown in
were recorded and averaged over 100 times. Since the FPGA Table IX. Since these approaches are implemented for different
runtime does not include the image loading time from the host image sizes and operated at different clock frequencies, their
to the external SRAMs, for fair comparison, the loading time results have to be normalized for a fair comparison. The
from the HDD to the RAM for the CPU implementation is rightmost column of Table IX gives the normalized computing
also not included in the reported runtime. In general, the CPU time, where the normalization is done with respect to an image
loading time is shorter than the FPGA loading time since the of size 512 512 and an FPGA clock frequency of 100 MHz.
speed of the internal CPU bus is faster than when using an We see that the computation time of the proposed distributed
external bus such as PCI or PCI external. implementation is much faster than the existing implemen-
From Table VIII, it can be seen that the speed of the tations. Furthermore, the implementations [5][7] result in
FPGA implementation for the original frame-based Canny degraded performance compared to our implementation since
edge detection is faster than the frame-based CPU implemen- the thresholds are fixed to predetermined values, while our
tation even though, for the CPU implementation, the CPU is implementation computes the hysteresis thresholds adaptively
clocked 33 times faster. Furthermore, the FPGA implementa- based on the input image characteristics. In addition, com-
tion for the proposed distributed Canny edge detection is much pared to other frame-based FPGA implementations, the pro-
faster than the frame-based FPGA and CPU implementations. posed implementation enables a higher throughput due to its
XU et al.: DISTRIBUTED CANNY EDGE DETECTOR 2959

distributed nature, at the expense of more memory and slice R EFERENCES

usage. [1] R. Deriche, Using canny criteria to derive a recursively implemented
Also, the GPU-based Canny implementations in optimal edge detector, Int. J. Comput. Vis., vol. 1, no. 2, pp. 167187,
[10] and [12] are compared to our 32-CE FPGA Canny 1987.
[2] L. Torres, M. Robert, E. Bourennane, and M. Paindavoine, Implemen-
implementation. One NVidia GPGPU was used in [10] and tation of a recursive real time edge detector using retiming technique,
three NVidia GPGPUs were used in the recent GPU Canny in Proc. Asia South Pacific IFIP Int. Conf. Very Large Scale Integr.,
implementation [12]. The GPU configuration details are given 1995, pp. 811816.
[3] F. G. Lorca, L. Kessal, and D. Demigny, Efficient ASIC and FPGA
below: implementation of IIR filters for real time edge detection, in Proc.
GTX 80 [10]: NVidia GeForce 8800 GTX with IEEE ICIP, vol. 2. Oct. 1997, pp. 406409.
[4] D. V. Rao and M. Venkatesan, An efficient reconfigurable architecture
128 575-MHz cores and 768MB RAM. and implementation of edge detection algorithm using handle-C, in
GT 80 [12]: NVidia GeForce 8800 GT with 112 1.5-GHz Proc. IEEE Conf. ITCC, vol. 2. Apr. 2004, pp. 843847.
cores and 512MB RAM. [5] H. Neoh and A. Hazanchuck, Adaptive edge detection for real-time
video processing using FPGAs, Altera Corp., San Jose, CA, USA,
GT 200 [12]: NVidia Tesla C1060 with 240 1.3-GHz
Application Note, 2005.
cores and 4GB RAM; [6] C. Gentsos, C. Sotiropoulou, S. Nikolaidis, and N. Vassiliadis, Real-
Fermi [12]: NVidia Tesla C2050 with 448 1.15-GHz cores time canny edge detection parallel implementation for FPGAs, in Proc.
IEEE ICECS, Dec. 2010, pp. 499502.
and 3GB RAM. [7] W. He and K. Yuan, An improved canny edge detector and
Since the FPGA runtime does not include the image loading its realization on FPGA, in Proc. IEEE 7th WCICA, Jun. 2008,
pp. 65616564.
time from the host to the external SRAMs, the loading time [8] I. K. Park, N. Singhal, M. H. Lee, S. Cho, and C. W. Kim, Design
for GPU implementations was removed from the total runtime and performance evaluation of image processing algorithms on GPUs,
for fair computation. Table X illustrates the runtimes of the IEEE Trans. Parallel Distrib. Syst., vol. 22, no. 1, pp. 91104, Jan. 2011.
[9] J. D. Owens et al., A survey of general-purpose computation on
GPGPU implementations and our 32-CE FPGA implementa- graphics hardware, Comput. Graph. Forum, vol. 26, no. 1, pp. 80113,
tion (with a 64 64 block size and a 3 3 gradient mask) 2007.
for processing a 512 512 image. Even though all these [10] Y. Luo and R. Duraiswami, Canny edge detection on NVIDIA CUDA,
in Proc. IEEE CVPRW, Jun. 2008, pp. 18.
GPGPU implementations do not have the adaptive threshold [11] R. Palomar, J. M. Palomares, J. M. Castillo, J. Olivares, and
calculation stage, our 32-CE FPGA implementation is faster J. Gmez-Luna, Parallelizing and optimizing lip-canny using NVIDIA
than the GPGPU implementations even though the FPGA is CUDA, in Proc. IEA/AIE, Berlin, Germany, 2010, pp. 389398.
[12] L. H. A. Lourenco, Efficient implementation of canny edge detection
operated at a much slower frequency. filter for ITK using CUDA, in Proc. 13th Symp. Comput. Syst., 2012,
pp. 3340.
[13] Q. Xu, C. Chakrabarti, and L. J. Karam, A distributed Canny
VII. C ONCLUSION edge detector and its implementation on FPGA, in Proc. DSP/SPE),
The original Canny algorithm relies on frame-level statistics Jan. 2011, pp. 500505.
[14] J. F. Canny, A computation approach to edge detection, IEEE Trans.
to predict the high and low thresholds and thus has latency Pattern Anal. Mach. Intell., vol. 8, no. 6, pp. 769798, Nov. 1986.
proportional to the frame size. In order to reduce the large [15] S. Nercessian, A new class of edge detection algorithms with perfor-
latency and meet real-time requirements, we presented a novel mance measure, M.S. thesis, Dept. Electr. Eng., Tufts Univ., Medford,
MA, USA, May 2009.
distributed Canny edge detection algorithm which has the [16] P. Bao, L. Zhang, and X. Wu, Canny edge detection enhancement by
ability to compute edges of multiple blocks at the same time. scale multiplication, IEEE Trans. Pattern Anal. Mach. Intell., vol. 27,
To support this, an adaptive threshold selection method is no. 9, pp. 14851490, Sep. 2005.
[17] J. K. Su and R. M. Mersereau, Post-processing for artifact reduction in
proposed that predicts the high and low thresholds of the JPEG-compressed images, in Proc. IEEE ICASSP, vol. 3. May 1995,
entire image while only processing the pixels of an individual pp. 23632366.
block. This results in three benefits: 1) a significant reduction [18] P. Arbelaez, C. Fowlkes, and D. Martin. (2013). The Berkeley
Segmentation Dataset and Benchmark [Online]. Available:
in the latency; 2) better edge detection performance; 3) the http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/
possibility of pipelining the Canny edge detector with other [19] N. D. Narvekar and L. J. Karam, A no-reference image blur metric
block-based image codecs. In addition, a low complexity non- based on the cumulative probability of blur detection (CPBD), IEEE
Trans. Image Process., vol. 20, no. 9, pp. 26782683, Sep. 2011.
uniform quantized histogram calculation method is proposed [20] (2013). USC SIPI Image Database [Online]. Available:
to compute the block hysteresis thresholds. The proposed http://sipi.usc.edu/database/database.php?volume=misc
algorithm is scalable and has very high detection performance. [21] (2013). Standard Test Image Database [Online]. Available:
http://www.imageprocessingplace.com/
We show that our algorithm can detect all psycho-visually [22] L. V. Scharff, A. Hill, and A. J. Ahumada, Discriminability measures
important edges in the image for various block sizes. Finally, for predicting readability of text on textured backgrounds, Opt. Exp.,
the algorithm is mapped onto a Xilinx Virtex-5 FPGA platform vol. 6, no. 4, pp. 8191, Feb. 2000.
[23] (2013). Altera Available IP [Online]. Available: http://www.
and tested using ModelSim. The synthesized results show 64% altera.com/products/ip/processors/nios2/features/ni2-peripherals.html
slice utilization and 87% BRAM memory utilization. The pro- [24] (2013). Xilinx Available IP [Online]. Available: http://www.
posed FPGA implementation takes only 0.721ms (including xilinx.com/ise/embedded/edk/_ip.htm
[25] (2007). Xilinx IP LogiCore FIR Datasheet [Online]. Available: http://
the SRAM read/write time and the computation time) to detect www.xilinx.com/support/documentation/ip_documentation/fir_compiler
edges of 512 512 images in the USC SIPI database when _ds534.pdf
clocked at 100 MHz. Thus the proposed implementation is [26] (2012). Xilinx Vertrix-5 Family Datasheet [Online]. Available:
http://www.xilinx.com/support/documentation/data_sheets/ds100.pdf
capable of supporting fast real-time edge detection of images [27] (2012). SRAM Datasheet [Online]. Available: http://www.
and videos including those with full-HD content. cypress.com/?docID=31696
2960 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

Qian Xu received the B.E. degree in optoelectronics Lina J. Karam (F13) received the B.E. degree
engineering from the Huazhong University of Sci- in computer and communications engineering from
ence and Technology, Wuhan, China, and the M.E. the American University of Beirut, Beirut, Lebanon,
degree in electrical engineering from the Shanghai and the M.S. and Ph.D. degrees in electrical engi-
Institute of Technical Physics, Chinese Academy of neering from the Georgia Institute of Technology,
Sciences, China, in 2006 and 2009, respectively. Atlanta, GA, USA, in 1989, 1992, and 1995, respec-
She is pursuing the Ph.D. degree in electrical tively.
engineering, specializing in image processing and She is a Full Professor with the School of
analysis, with Arizona State University, Tempe. Her Electrical, Computer and Energy Engineering, Ari-
research interests include image processing, com- zona State University, Phoenix, AZ, USA, where
puter vision, pattern recognition, and algorithm- she directs the Image, Video, and Usability (IVU)
architecture codesign. Research Laboratory. Her industrial experience includes image and video
compression development at AT&T Bell Laboratories, Murray Hill, NJ,
USA, multidimensional data processing and visualization at Schlumberger,
and collaboration on computer vision, image/video processing, compres-
Srenivas Varadarajan received the B.E. degree in sion, and transmission projects with industries, including Intel, NTT,
electronics and communications engineering from Motorola/Freescale, General Dynamics, and NASA. She has authored more
than 100 technical publications, and she is a co-inventor on a number of
the PSG College of Technology, Coimbatore, India,
and the M.S. degree in electrical engineering from patents.
Arizona State University, in 2003 and 2009, respec- Dr. Karam was the recipient of the U.S. National Science Foundation
tively, where he is currently pursuing the Ph.D. CAREER Award, the NASA Technical Innovation Award, the 2012 Intel
Outstanding Researcher Award, and the Outstanding Faculty Award by the
degree in electrical engineering in the area of image.
His research interests include texture analysis and IEEE Phoenix Section in 2012. She has served on several journal editorial
synthesis, image and video compression, computer boards, several conference organization committees, and several IEEE techni-
cal committees. She served as the Technical Program Chair of the 2009 IEEE
vision, 3-D modeling, and embedded-software opti-
mizations for media processing algorithms. He has International Conference on Image Processing, the General Chair of the 2011
about seven years of Industrial experience in image and video processing IEEE International DSP/SPE Workshops, and the Lead Guest Editor of the
in companies, including Texas Instruments, Qualcomm Research, and Intel P ROCEEDINGS OF THE IEEE, P ERCEPTION -BASED M EDIA P ROCESSING
I SSUE (S EP. 2013). She has co-founded two international workshops (VPQM
Corporation.
and QoMEX). She is currently serving as the General Chair of the 2016
IEEE International Conference on Image Processing and a member of the
IEEE Signal Processing Societys Multimedia Signal Processing Technical
Committee and the IEEE Circuits and Systems Societys DSP Technical
Chaitali Chakrabarti received the B.Tech. degree Committee. She is a member of the Signal Processing, Circuits and Systems,
in electronics and electrical communication engi- and Communications societies of the IEEE.
neering from IIT Kharagpur, India, and the M.S.
and Ph.D. degrees in electrical engineering from
the University of Maryland, College Park, in 1984,
1986, and 1990, respectively. She is a Professor with
the School of Electrical Computer and Energy Engi-
neering, Arizona State University (ASU), Tempe.
Her research interests include all aspects of low-
power embedded systems design and VLSI archi-
tectures, and algorithms for signal processing, image
processing, and communications.
Dr. Chakrabarti was the recipient of the Best Paper Awards at SAMOS07,
MICRO08, SiPS10, and HPCA13. She is a Distinguished Alumni with the
Department of Electrical and Computer Engineering, University of Maryland.
She was the recipient of several teaching awards, including the Best Teacher
Award from the College of Engineering and Applied Sciences from ASU
in 1994, the Outstanding Educator Award from the IEEE Phoenix Section
in 2001, and the Ira A. Fulton Schools of Engineering Top 5% Faculty
Award in 2012. She served as the Technical Committee Chair of the DISPS
subcommittee, IEEE Signal Processing Society, from 2006 to 2007. She
is currently an Associate Editor of the Journal of VLSI Signal Processing
Systems and the IEEE T RANSACTIONS OF VLSI S YSTEMS .