Politecnico Di Torino Porto Institutional Repository
Politecnico Di Torino Porto Institutional Repository
I. I NTRODUCTION
Partial Differential Equation (PDE) based models have shown to be useful for image preprocessing
(multiscale analysis) [1], [2], [3], [4]. They can be classified in three main categories: a) linear isotropic
filters; b) nonlinear isotropic filters; c) nonlinear anisotropic filters. Linear isotropic filtering (heat equation)
was proved to be the only linear model that satisfies all the multiscale analysis requirements [2]. However
it presents the disadvantage of blurring important features in image processing, like edges. Nonlinear
isotropic filters (in particular the Perona-Malik filter [5]) were introduced for overcoming the limitations
of linear models and are based on the idea of adapting the diffusivity to the gradient of the evolving image.
They do not exhibit good performances in images containing noisy edges and coherent flow-like structures.
In such cases they can be replaced by anisotropic models [1], that have found significant applications in
medical imaging and computer-aided quality control [1]. The main drawback of PDE based techniques
is that from a computational point of view they are rather time-consuming and therefore they are not
suitable for real-time image processing.
In parallel with the mathematical studies on the application of nonlinear PDEs to image processing, in
the electrical engineering community a new paradigm of neural network, called cellular neural network
(CNN) ([6], [7], [8], [9]), was introduced and shown to be able to solve several complex computational
problems.
As far as the applications are concerned, the implementation of PDE-based algorithms onto CNNs
allows one to afford real-time image processing [10]. From a theoretical point of view, the relationship
between PDE and CNN dynamics has been investigated in several papers. In [11] it was shown that, in
general, CNN dynamics represents a broader class than PDEs.
In this manuscript a CNN model of partial differential equations for image multiscale analysis is
proposed. The model is based on a polynomial representation of the diffusivity function and defines a
paradigm of polynomial CNNs, which approximates a large class of nonlinear isotropic and anisotropic
PDEs, and also some non-PDE models [12]. The global dynamics of space-discrete polynomial CNN
models is analyzed and compared with the dynamic behavior of the corresponding space-continuous PDE
models. It is shown that in the isotropic case the two models are not topologically equivalent: in particular
discrete CNN models allow one to obtain the output image without stopping the image evolution after
a given time (scale). This property represents an advantage with respect to continuous PDE models and
could simplify some image preprocessing algorithms.
II. PDE BASED CNN MODELS
According to [1], [2], a gray-level image is defined as a real bounded function u0 (x, y) : R 2 R,
where u0 (x, y) denotes the gray level value at the point (x, y). Introducing an artificial time t, an image
u(t, x, y) is assumed to evolve, according to a PDE of the form:
u(t, x, y)
= F [u(t, x, y)]
t
u(0, x, y) = u0 (x, y)
(1)
where F : R R is an operator that typically depends on the image and on its first and second
order space derivatives and characterizes the given algorithm; u0 (x, y) represents the initial condition.
Hereinafter we assume that each image u(t, x, y) exhibits Neumann type space boundary conditions.
The solution of equation (1), denoted by Tt (u0 (x, y)) = u(t, x, y), represents the processed image at
the scale (time) t. As shown in [2], the sequence of images u(t, x, y), that is the images generated from
u0 (x, y) by the operator Tt (), performs a multiscale analysis satisfying some selected basic principles
(called axioms in [2]).
A. Isotropic diffusion based CNN models
It is well known that the simplest multiscale analysis is obtained by using linear isotropic diffusion,
i.e. by exploiting the following PDE (heat equation):
u
2u 2u
= div [u] = 2 + 2 .
(2)
t
x
y
To overcome the drawbacks of linear diffusion, Perona and Malik [5] proposed the following PDE
describing a nonlinear isotropic diffusion:
u
= div g(kuk2)u
(3)
t
where the original image u(0, x, y) = u0 (x, y) is used as initial condition, u and kuk denote the
image gradient and its magnitude respectively and g() represents the diffusivity function. The following
diffusivity functions were proposed by Perona and Malik [5]:
"
2 #1
kuk 2
kuk
g(kuk2) = e( k ) , g(kuk2) = 1 +
(4)
k
where k is a scalar parameter.
Finite difference approximations of Perona-Malik equation (3) have been investigated in several papers
([1], [13], and [14]).
Let us suppose that the continuous space domain is composed by N M points arranged on a regular
grid and let us denote the position of a point with two indexes (i, j); uij represents the pixel value,
obtained by sampling the image u(t, x, y) at the point (xi , yj ) = (h i, h j), where h is the grid size. It
turns out that among the many approaches for solving PDEs, the finite difference method is the most
suited to obtain a system of coupled ordinary differential equations, that can be mapped onto a CNN
structure.
3
1
f(z)
0.5
0
10
Fig. 1.
z
8
10
By using a finite difference approximation, a consistent semidiscrete version of the Perona-Malik model
(3) can be written as [1] (1 i N and 1 j M):
X gkl + gij
d uij (t)
=
(ukl uij )
dt
2 h2
(5)
(kl)Nij
where Nij = {(i 1, j), (i + 1, j), (i, j 1), (i, j + 1)} and gkl and gij are evaluated by approximating
the image gradient through the central difference around the points (xk , yl ) and (xi , yj ) respectively. As
e i,j k2 ) with
an example, gij = g(ku
2
2
u
u
u
u
i+1,j
i1,j
i,j+1
i,j1
2
e i,j k =
ku
+
.
(6)
2h
2h
In order to implement (5) in a general feasible polynomial CNN architecture, let us consider the
following basis function (see Fig. 1):
z
1 z
(7)
f (z) = 1 + 1 1
2 m
m
and let us approximate the g() functions (4) with the following expression:
g(z)
= (z) =
Q
X
cp f p (z)
(8)
p=1
It is worth observing that f (z) is different from zero only on the finite domain z (m, m); this
allows us to control the nonlinear diffusion process according to the threshold m. Hence, this property is
also valid for (z).
The parameter m depends on the admissible error eb between g() and () at the point z = m. This
allows us to rewrite m as m = k 2 (b
e), where (b
e) = ln(b
e) or (b
e) = ((b
e)1 1) for the functions
reported in (4).
The parameters c1 , ..., cQ , Q and m are chosen in such a way that (z) interpolates g(z) in a set of
Q uniform spaced points within the interval [m, m]. It follows that (z) and g(z) exhibit the same
properties, i.e. (z) is constrained to be a positive and bounded function (0 (z) 1); these conditions
ensure that:
gij
= ij =
Q
X
p=1
e i,j k2 )
cp f p (ku
(9)
By replacing gij and gkl in (5) with the approximate expressions ij and kl , we obtain a general,
nonlinear PDE based, polynomial CNN model (PCNNs):
d uij (t)
=
dt
(kl)Nij
kl (ukl uij )
(10)
(11)
and defines the following nonlinear template, whose structure is identical for each processing cell:
0
i1,j
0
0
i,j+1
i,j1
(12)
0
i+1,j
0
B. Anisotropic diffusion based CNN models
In nonlinear isotropic filters, the diffusion process only depends on the gradient magnitude. In certain
applications [1] better performances are achieved if the diffusivity is not identical in all the directions.
This requirement can be fulfilled, by introducing a suitable diffusion matrix, that changes the orientation
of the gradient.
In [1] the following anisotropic PDE equation was proposed:
u
= div [D u]
(13)
t
where D is a positive definite 2 2 matrix, depending on the gradient. It is easily derived that linear and
nonlinear isotropic diffusion can be obtained with D = I and D = g(kuk2)I respectively (where I
denotes the identity matrix).
Usually, the matrix D R 22 is chosen to be a function of u (see [1]):
D = S(kuk)S T
(14)
where (kuk) = diag(1 (kuk), 2(kuk)) is a diagonal matrix containing the eigenvalues 1 , 2
and S is a 2 2 matrix whose rows are the two corresponding eigenvectors v1 , v2 .
The main drawback of isotropic models (like Perona-Maliks) is that they might be not robust in presence
of noise. In order to remove the noise at the edges it is necessary to rotate the flux according to the edge
direction, in order to allow the diffusion along the edges and to prevent it perpendicularly. This can be
obtained by choosing the matrices S and reported below:
u
u
x
y
g(kuk2) 0
S =
=
(15)
0
1
u
u
y
x
that give rise to the following matrix D
D=
a(x, y) b(x, y)
b(x, y) c(x, y)
(16)
a(x, y) = g kuk
u
x
2
u
y
2
u u
g kuk2 1
x y
2 2
u
u
+
.
c(x, y) = g ku|2
y
x
b(x, y) =
(17)
A standard space discretization scheme for equation (13), when the matrix D shown in (16) is exploited,
allows one to obtain the following semi-discrete model:
X
d uij (t)
=
kl (ukl uij )
dt
(18)
bij
(kl)N
bi1,j + bi,j1
ai1,j + ai,j (bi1,j + bi,j+1 )
ci,j1 + ci,j
0
ci,j+1 + ci,j
(bi+1,j + bi,j1) ai+1,j + ai,j
bi+1,j + bi,j+1
In the above expression ai,j , bi,j and ci,j , represent the polynomial approximation of the functions
a(x, y), b(x, y) and c(x, y) respectively, according to the following expressions:
ai,j
bi,j
ci,j
2
2
ui+1,j ui1,j
ui,j+1 ui,j1
+
= i,j
2h
2h
ui+1,j ui1,j
ui,j+1 ui,j1
= (i,j 1)
2h
2h
2
2
ui,j+1 ui,j1
ui+1,j ui1,j
= i,j
+
.
2h
2h
(i, j)
(19)
(20)
Proof : By noting that the coefficient kl , shown in equation (11), is written as sum of nonnegative
terms, it can be easily obtained that kl = 0 for all (k, l) Nij and for all (i, j) if and only if:
e i,j k2 = 0 (i, j)
f p ku
(21)
e k,l k2 = 0
f p ku
(22)
The conditions above are verified if and only if the argument of the function f () (see equation (7)) is
greater than m, that is:
e i,j k2 m
ku
e k,lk2 m
ku
(i, j)
(23)
(24)
Q. E. D.
Note that if m is chosen in such a way that only one of the above constraints (19), (20) is verified, then
kl is always different from zero. This implies that both systems converge toward the only equilibrium
point represented by the average value of the input image.
In order to show the influence of m on PCNN dynamics, we consider the very simple case of onedimensional images composed by N pixels (denoted by ui , 1 i N). We denote with i = a generic
index and define the average value of the left and of the right side of the input image u0i (with respect to
index ) as follows:
1X 0
hul() i =
u ,
i=1 i
N
1 X 0
hur() i =
u
N i=+1 i
(25)
Owing to input image conditions, it is derived that only the following derivatives might be different
from zero at the initial instant t = 0:
0
0
(u+1
u0 )
(u )|t=0 = +1
0
(u +1 )|t=0 = 0 (u0 u+1
)
(26)
(27)
0
0
0
where the coefficients, defined in (11), 0 = 0 + +1
and +1
= +1
+ 0 are identical.
The necessary and sufficient conditions in order that (u )|t=0 and (u +1 )|t=0 be null is that both 0 and
0
+1 be zero, that is:
0
0 = 0 u01 u+1
2h m
0
0
+1
= 0 u0 u+2
2h m
(28)
0
It turns out that if the above conditions (28) are satisfied, then the
input image ui is an equilibrium
point. Since conditions (28) are equivalent to |hul() i hur() i| 2h m, this proves the first part of the
Proposition.
If |hul() i hur() i| < 2h m, then (26) and (27) are different from zero. In this case it can be readily
shown that the PCNN system described by (26) evolves to a solution given by the average value of the
input image.
Q. E. D.
Remark : Proposition 2 shows that the PCNN model exhibits an infinite set of equilibrium points
satisfying conditions (a) and (b), in addition to the set of equilibria (corresponding to the average value
of the input image) that also the discretized Perona-Malik model possesses. As a result the two models
are not topologically equivalent. The consequence of this different dynamic behavior is that the output
image of a PCNN model is obtained without stopping the system evolution after a given time (scale).
This could represent a significant advantage for implementation of image preprocessing algorithms, as
shown in the following example.
Example: Without losing any generality we assume that a Gaussian noise with zero mean value
and unknown variance is added to a given standard image (see Fig. 2). Similar considerations hold by
considering different noise models (e.g. speckle or salt and pepper noise).
The upper and the lower part of Fig. 3 show the dynamic behavior of the Perona-Malik (5) and of the
polynomial CNN model (10) respectively . In particular, a second order approximation of the exponential
diffusivity function is exploited (see equation (4)). The parameter m is set to 0.04 and the input graylevel image (bounded function between [1, +1], where 1 and +1 correspond to white and black pixels
respectively) with Gaussian noise, is given as initial condition. It is readily verified that the steady-state
output of the Perona-Malik model is the average value of the input image, while the PCNN converges to
an equilibrium point, corresponding to an image, where the noise has been removed.
It is worth remarking that in case of model (5), the correct scale at which the evolution should be
stopped, cannot be accurately estimated because no a-priori information is available for the Gaussian
noise variance.
In order to compare the two processes it is useful to compute the classical signal to noise ratio defined
as:
P
2
i,j (ui,j (t))
(29)
SNR(t) = 10 Log P
id
2
i,j (ui,j (0) ui,j (t))
where uid (0) is the original input image without noise and ui,j (t) is the processed image. It is well known
that this parameter can be used when the original image is available and the elaboration is performed on
Fig. 2.
Fig. 3. Top: Perona-Malik model. Bottom: Polynomial CNN. From left to right: Output images at the scales t = 0.1, t = 1, t = 10,
t = 100 and t = 500.
static images. Table I presents the SNR values evaluated at the scales corresponding to the images shown
in Fig. 3.
It is worth observing that the SNR, obtained by using the Perona-Malik model, becomes very low for
large values of the scale because the output image corresponds to the average value of the noisy input
image. On the other hand, by considering the PCNN model, the SNR tends to an higher constant value
for the reason that the output image is one equilibrium point of the system.
IV. N ON -PDE BASED CNN MODELS
The CNN models proposed in Section II are derived from nonlinear PDE based filtering processes.
Recently a novel non-PDE based approach has been proposed (see [12] and [15]), in order to deal with
images corrupted by non-Gaussian noise.
This model is described by the following equation (see [12]):
SNR
Perona-Malik model
PCNN
t = 0.1
23.73
23.81
t=1
23.27
23.00
t = 10
22.07
22.15
t = 100
19.31
20.70
t = 500
11.32
19.35
TABLE I
S IGNAL TO N OISE R ATIO EVALUATED AT
THE SCALES t
= 0.1, t = 1, t = 10, t = 100 AND t = 500 FOR THE P ERONA -M ALIK AND
PCNN MODELS .
THE
d uij (t)
= Ji,j1 (ui,j1 ui,j ) + Ji,j+1 (ui,j+1 ui,j ) +
dt
Ji1,j (ui1,j ui,j ) + Ji+1,j (ui+1,j ui,j )
(30)
where the coefficients Ji,j1, Ji,j+1, Ji1,j , and Ji+1,j represent the local variances defined as:
Ji,j1 =
(uk,l ui,j )2
(uk,l ui,j )2
(uk,l ui,j )2
(uk,l ui,j )2
l=j+1,|ki|1
Ji,j+1 =
l=j1,|ki|1
Ji1,j =
k=i+1,|lj|1
Ji+1,j =
k=i1,|lj|1
Vij
)
(1 e
Vt
V
ij
(1
e
)
Vt
V
ij
)
(1 e
Vt
V
ij
(1
e
)
Vt
Vt
+ 0.25 e
Vt
+ 0.25 e
Vt
+ 0.25 e
(31)
Vt
+ 0.25 e
Ji,j1 =
Ji,j+1 =
Ji1,j =
Ji+1,j =
l=j+1,|ki|1
k=i+1,|lj|1
l=j1,|ki|1
(32)
k=i1,|lj|1
where 1 () and 2 () are given by equation (8) and Vij has the following expression:
X
Vij
(uk,l ui,j )2 .
=
|ki|1,|lj|1
(33)
10
Finally, by posing 1 (Vij ) = 2 (Vij )/ and 2 (Vij ) = ij , the following nonlinear template, whose structure
is identical for each processing cell cij , is obtained:
0
i1,j
0
0
i,j+1
i,j1
(34)
0
i+1,j
0
where
i,j1
i,j+1
i1,j
i+1,j
ij
ij
ij
k=i+1,|lj|1
ij
l=j1,|ki|1
l=j+1,|ki|1
k=i1,|lj|1
V. C ONCLUSIONS
In this study we have proposed a class of CNN models for approximating isotropic (Perona-Malik)
and anisotropic diffusion equations, based on a polynomial approximation of the diffusivity function.
Such models present the advantage of defining a paradigm of polynomial CNNs, that is also useful for
approximating some non-PDE based models.
The dynamic of the original Perona-Malik model and that of the polynomial CNN have been compared.
It has been shown that polynomial CNNs have more than one stable equilibrium point and for each initial
image, the output image corresponds to one of these equilibria. This property (implying that the PeronaMalik and the polynomial CNN model are not topologically equivalent) could be exploited for some
image preprocessing tasks.
VI. ACKNOWLEDGMENT
This work was supported in part by Ministero dellIstruzione, dellUniversit`a e della Ricerca, within
the framework of the National Research Plan PRIN 2004 no. 2004092944 004 and the FIRB project
no. RBAU01LRKJ.
R EFERENCES
[1] J. Weickert, A review of nonlinear diffusion filtering, Lecture Notes in Computer Science, Springer, Berlin, vol. 1252, pp. 3-28, 1997.
[2] L. Alvarez, F. Guichard, P. L. Lions and J. M. Morel, Axioms and fundamental equations of image processing, Arch. Rational Mech.
Anal., vol. 123, pp. 199-257, 1993.
[3] A. P. Witkin, Scale-space filtering, Proc. Eighth Int. Joint Conf. on Artificial Intelligence (IJCAI83), vol. 2, pp. 1019-1022, 1983.
[4] F. Catte, P. L. Lions, J. M. Morel and T. Coll, Image selective smoothing and egde detection by nonlinear diffusion, SIAM Journal
on numerical analisys, vol. 29, pp. 182-193, 1992.
11
[5] P. Perona, J. Malik, Scale space and edge detection, using anisotropic diffusion, IEEE Trans. Pattern Anal. Mach. Intell., vol. 12,
pp. 629-639, 1990.
[6] L. O. Chua and L. Yang,Cellular neural networks: Theory, IEEE Trans. Circuits Syst. I, vol. 35, no. 10, pp. 1257-1272, Oct. 1988.
[7] L. O. Chua and L. Yang, Cellular neural networks: Applications, IEEE Transactions on Circuits and Systems, vol. 35, no. 10,
pp. 1273-1290, Oct. 1988.
[8] L. O. Chua and T. Roska, The CNN paradigm, IEEE Trans. Circuits Syst. I, vol. 40, no. 3, pp. 147-156, March 1993.
[9] L. O. Chua and T. Roska, Cellular neural networks and visual computing, Cambridge University Press, U.K., 2002
[10] L. O. Chua, T. Roska, T. Kozek, and A. Zarandy, CNN universal chips crank up the computing power, IEEE Circuits and Devices,
pp. 18-28, 1996.
[11] M. Gilli, T. Roska, L. O. Chua, P. P. Civalleri, CNN dynamics represents a broader class than PDEs, International Journal of
Bifurcation and Chaos, vol. 12, no. 10, pp. 2051-2068, 2002.
[12] C. Recekzky, CNN architectures for constrained diffusion based locally adaptative image processing, Int. J. Circuit Theory and
Applicat., vol. 30, pp. 313-348, 2002.
[13] C. Recekzky, T. Roska and A. Ushida, CNN-based difference-controlled adaptative nonlinear image fiters, Int. J. Circuit Theory and
Applicat., vol. 26, pp. 375-423, 1998.
[14] J. Weickert and B. Benhamouda,Nonlinear diffusion scale-spaces: From the continuous to the, discrete setting, in Proc. ICAOS96:
Images, Wavelets and PDEs, vol.219, M.-O. Berger et al., Eds. New York: Springer, pp. 111-118, 1996.
[15] T. Shiota N. Nitzberg, Non-linear image smoothing with edge and corner enhancement, Technical Report 90-2, Division of Applied
Sciences, Harvard University, Cambridge, MA.