Computers & Graphics 27 (2003) 529534

Technical section

An efcient algorithm for the enhancement of

JPEG-coded images$
G.A. Triantafyllidisa, M. Varnuskab, D. Sampsonc, D. Tzovarasc,
M.G. Strintzisa,c,*
a

Information Processing Laboratory, Electrical and Computer Engineering Department, Aristotle University of Thessaloniki,
Thessaloniki 54124, Greece
b
Computer Science Department, University of West Bohemia, Plzen, 30614 Czech Republic
c
Informatics and Telematics Institute, 1st Km Thermi-Panorama Road, Thermi-Thessaloniki 57001, Greece
Accepted 15 April 2003

Abstract
Despite its growing age, the JPEG is among the most popular choices as a standard compression scheme for
continuous-tone still images. In this paper, a novel technique is proposed to alleviate the blocking artifacts that usually
occur in JPEG coded images especially at low bit rates. The proposed algorithm consists of two stages: rstly, the AC
coefcients are estimated based on their observed probability distribution and secondly, a postprocessing scheme is
applied for blockiness removal, consisting of a region classication algorithm and a spatial adaptive ltering.
Experimental results demonstrate the efciency of the proposed method.
r 2003 Elsevier Ltd. All rights reserved.
Keywords: Blocking artifact reduction; DCT coefcient adjustment; Spatial adaptive ltering; MSDS

1. Introduction
The basic approach for the JPEG image compression
is fairly simple. The encoding process consists of
dividing the image into blocks, typically of size 8
8:
A block transform, typically the DCT, is applied to
these blocks, and the transform coefcients are individually quantized (scalar quantization). This block DCT
scheme takes advantage of the local spatial correlation
$
This work was supported by the EU project IST
OTELO.
*Corresponding author. Information Processing Laboratory,
Department of Electrical and Computer Engineering, Aristotelian University of Thessaloniki, Thessaloniki 54124, Greece.
Tel.: +30-31-99-6351; fax: +30-31-99-6398.
E-mail addresses: [email protected] (G.A. Triantafyllidis),
[email protected] (M. Varnuska), [email protected]
(D. Sampson), [email protected] (D. Tzovaras),
[email protected] (M.G. Strintzis).

property of images and also saves processing time [1].

Due to the performance of DCT to highly correlated
signals which is close to that of the (optimal) Karhunen
Loeve transform (KLT), and the availability of fast
software and hardware implementations, JPEG is very
popular in many image coding applications.
However, the individual processing of each block
induces visually annoying blocking effects, since the
correlation among spatially adjacent blocks is disregarded during coding, particularly when a high quantization parameter is used for achieving high compression
ratios. For example, a smooth change of luminance
across a border can result in a step in the decoded image
if neighboring samples fall into different quantization
intervals. Such so-called blocking artifacts, are often
very disturbing. The blocking effect in JPEG images can
be classied into three categories [2]: (a) Stair case noise
along the image edges, (b) grid noise in the monotone
areas, and (c) corner outliers in the corner points of the
8
8 DCT blocks.

0097-8493/03/$ - see front matter r 2003 Elsevier Ltd. All rights reserved.
doi:10.1016/S0097-8493(03)00088-8

530

G.A. Triantafyllidis et al. / Computers & Graphics 27 (2003) 529534

To reduce such artifacts, various low-pass ltering

schemes have been proposed. These techniques smooth
out the block boundary at the expense of unnecessary
blurring of the decoded image. The easiest approach
taken to tackle this problem is to low-pass the blocky
JPEG image [3]. This approach will reduce the effect of
high-frequency tendency but the resulting image will be
blurry and some details will be wiped out. In the
Ramamurthi algorithm [4], the ratio of intensity change
is compared to the average intensity using an appropriate threshold. This algorithm neutralizes the gradients caused by random speckle noise. A 2-D lter is used
to reduce the grid noise in the monotone area, whereas a
1-D lter is used to reduce the staircase noise along the
edges. However, the classier cannot discriminate
between an L-shaped edge segment and diagonal edge
segment.
Xiong et al. [5] use an overcomplete wavelet
representation to reduce the quantization effects of
block based DCT. In [6], the wavelet transform modulus
maxima (WTMM) representation is used for efcient
image deblocking. A spatially adaptive image recovery
algorithm was proposed in [7,8] based on the theory of
projections onto convex sets (POCS). In the POCSbased method, closed convex constraint sets are rst
dened that represent all the available data on the
original uncoded image. Alternating projections onto
these convex sets are then iteratively computed to
recover the original image from the coded image. POCS
is effective in eliminating blocking artifacts, but less
practical for real-time applications, since the iterative
procedure signicantly increases the computation complexity.
In this paper, a new method is proposed for the
reduction of blocking effects in JPEG. Initially, better
estimates of the reconstructed DCT coefcients are
obtained based on their observed probability distribution. Subsequently, a novel postprocessing scheme is
applied consisting of a region classication algorithm
and a spatial adaptive ltering for blockiness removal.
The type of ltering is determined on the basis of an
estimate of the local characteristics of the coded image.
The aforementioned two stages of the proposed algorithm are acting complementarily for the reduction of
blocking artifacts. The efcient performance of the
proposed algorithm is due rstly to the proposition that
the shape and the position of the lter kernel are
adjusted according to the characteristics of the local
image region and secondly to the employment of the
modied improved DCT coefcients by the postprocessing lter. This approach is shown to produce excellent
results in removing blocking artifacts.
The rest of this paper is organized as follows: In
Section 2, the mathematical analysis underlying the
concept for the distribution-based restoration of the
DCT coefcients is described. Section 3 presents in

detail the blocking artifact reduction algorithm by the

spatial adaptive ltering procedure. Experimental results
given in Section 4 evaluate visually and quantitatively
the performance of the proposed method, using the
MSDS metric. Finally, conclusions are drawn in
Section 5.

2. DCT coefcient adjustment

In the JPEG formulation, the input image is rst
divided into 8
8 blocks, and the two-dimensional DCT
of each block is calculated. The two dimensional DCT
can be obtained by performing column-wise onedimensional DCT followed by row-wise one-dimensional DCT. The 64 DCT coefcients in the spatial
block Bi;j are then determined by the following formula:
Fij u; v

7
7
X
X

cn;u cm;v fij n; m;

n0 m0

where

cn;u au cos 2n 1up=16; cm;v av cos 2m

1
if x 0 or ax 1 if xa0; u; v
1vp=16; ax p
2

0; y; 7; i 0; y; N=8  1; j 0; y; M=8  1; Fij u; v

are the DCT coefcients of the Bi;j block, fij n; m is the
luminance value of the pixel n; m of the Bi;j block, and
N
M are the dimensions of the image.
The two-dimensional DCT transformed output is
ordered so that the DC coefcient Fij 0; 0; is in the
upper left corner and the higher frequency coefcients
follow, depending on their distance from the DC
coefcient. The higher vertical frequencies are
represented by higher row numbers and higher horizontal frequencies are represented by higher column
numbers. A typical quantization-reconstruction process
of the DCT coefcients as described in JPEG [1], is
given by


Fij u; v
FijQ u; v round
and
Q u; v
FijR u; v FijQ u; vQu; v;

where Qu; v indicates the quantization width bin for

the given coefcient, FijQ u; v indicates the bin index in
which the coefcient Fij u; v falls and FijR u; v represents
the reconstructed quantized coefcient. Then, the
reconstructed pixel intensity is obtained using inverse
DCT.
The standard method for restoring the DCT coefcients is equivalent to replacing each coefcient by the
center of the quantization interval in which the original
coefcient falls. The distribution of the non-DC
coefcients for a given u; v peaks at zero and decreases
monotonically. For quantization intervals not including
zero, the distribution of the original coefcients is denser

G.A. Triantafyllidis et al. / Computers & Graphics 27 (2003) 529534

at the end of the interval closer to zero. The mean of the

distribution is the minimum mean squared error
reconstructor. For simplicity, we model the distribution
of absolute amplitudes as exponential with mean m [11].
The mean of FijQ u; v yields an estimation for m: Then, a
DCT quantized coefcient is restored by
e1=m
if FijQ u; v > 0;
1  e1=m
F#ijQ u; v FijQ u; v if FijQ u; v 0;

F#ijQ u; v FijQ u; v  0:5 m 

e
F#ijQ u; v FijQ u; v 0:5  m
;
1  e1=m
Q
if Fij u; vo0:

Let Px; y denote the image intensity value at point

x; y: The image gradient rP Gx ; Gy T is computed
as follows:
Px 1; y  Px  1; y
and
2
Px; y 1  Px; y  1
:

Gx x; y

Gy x; y
4

The WSMM is then dened as [13]

W
2

1=m

531

6 x;yAM
M
4
P

wb x; yGx2 x; y
wb x; yGx x; yGy x; y

x;yAM
M

wb x; yGx x; yGy x; y

x;yAM
M

x;yAM
M

wb x; yGy2 x; y

3
7
5;

3
5

3. Postprocessing using spatial adaptive ltering

Following the description of the main picture quality
impairments due to low bit rate JPEG compression
scheme and after the biased reconstruction of the DCT
coefcients, the design requirements of the proposed
efcient postprocessing method can be outlined as
follows. The compressed image is initially segmented
into different regions which correspond to areas that
suffer from different types of degradation. Hence, a
classier which distinguishes the smooth regions from
the detailed ones need to be employed.
Provided that the above step of region classication
has been implemented successfully, an adaptive ltering
technique takes into consideration the characteristics of
the quantization noise in different areas of the coded
image. Specically:
*

Staircase noise along the edges must be removed, so

that edges appear continuous and sharp. This
requires smoothing along, but not across, the edge
direction to avoid blurring.
False contours visible in the areas of slowly varying
intensity should be eliminated. This requires smoothing of the intensity changes that occur between
adjacent blocks.
The disturbing blocking effect should also be
removed from textured areas, in a manner that
preserves any high detail that survived during the
coding process.

The proposed method employs a useful tool for the

description of local image features, known as windowed
second moment matrix (WSMM) [12], whose elements
are the locally smoothed functions of the image
derivatives. Information provided by the WSMM
has been utilized for a number of computer vision
tasks, such as the detection of feature points and the
extraction of shape information from texture
areas [12].

where M
M is the size of the analysis block, that is,

the area from where information is obtained for the
calculation of the WSMM components A W 0; 0;
B W 1; 1 and C W 0; 1 W 1; 0: The function
wb x; y is a symmetric and normalized window function
which is used for local smoothing of image features. This
is necessary due to the sensitivity to noise of the rst
order derivatives of image intensity. A natural choice for
wb x; y is a Gaussian function of the form wb x; y
2
2
2
ex y =2t : The WSMM coefcients are then employed
to perform the tasks required for the removal of
annoying artifacts from the compressed images, namely,
region classication and ltering.
3.1. Region classification
A simple algorithm based on the values of the
WSMM coefcients is used to distinguish the low-detail
areas from the high-detail ones in the coded image. The
quantities A and B; dened in (5), contain information
about the local edge strength [14]. It is expected that a
large value of either of them will be due to the presence
of a jagged edge or image texture. This is especially true
when the area of smoothing, dened by parameter M; is
chosen to have size similar to that of the coding block.
Thus, region classication is performed by comparing
the values of A and B against a predetermined threshold
T [14]. If at least one of them exceeds T; then the
corresponding picture element is assumed to belong to a
high-detail area of the image; otherwise, it is assumed to
lie in a uniform low-detail area. The ltering strategy
is then decided according to the outcome of the classication.
3.2. Filtering of high-detail areas
The purpose of the postprocessing in high-detail areas
is twofold: to eliminate the visible noise from the edges
and to preserve any detail that remains in the texture
blocks. A new adaptive ltering operation which satises

G.A. Triantafyllidis et al. / Computers & Graphics 27 (2003) 529534

532

both requirements is proposed in this paper. The process

computes for each pixel, a Gaussian-type kernel which is
shaped and displaced according to locally smoothed
image gradient functions [15]. The lter kernel Kx; y is
dened as follows:
2
2
1
2
#
#
#
Kx; y eAxDx 2CxDx yDy ByDy =2s ;
6
S
where S is a normalization factor dened by [14]
X
2
2
2
#
#
#
eAuDu 2CuDu vDv BvDv =2s
7
S
u;vAN
N

and N is the side length of the square truncation mask

that contains the kernel, s is the standard deviation of
# B# and C# are the
the variable Gaussian kernel, A;
coefcients of the WSMM normalized by division by
A B to reduce image dependency problems [15], and
Dx ; Dy determine the magnitude of the kernel displacement along each of the principal axes. Dx ; Dy are then
dened as follows [16]:
N
Vx
Dx q and
2 m2 V 2 V 2
x

Dy

Vy
N
q;
2 m2 V 2 V 2
x

where m is an attenuation constant and

X
1
wb x; yGx2 x; yx
Vx
A B x;yAM
M
Gx x; yGy x; yy;
Vy

1
AB

wb x; yGy2 x; yy

x;yAM
M

Gx x; yGy x; yx:

The three terms in the exponent adapt the shape of

the kernel in such a way that its longest axis is parallel to
the locally dominant edge direction [16]. The function of
the displacement components, which are also obtained
from gradient calculations, is to displace the main
weights of the kernel away from edges. The effect is that,
when the centre is close to an intensity edge, the main
contribution to the convolution originates from the
region that includes the centre and not from the other
side of the boundary. Consequently, the sharpness of the
edge is preserved.
This lter satises all the requirements for processing
detailed areas of the reconstructed image. First, near an
intensity edge the lter kernel is oriented along the edge
which is important in order to avoid blurring. Secondly,
the fact that the main weights of the kernel are displaced
in or out of the edge (depending on the relative position
of the point which is ltered) ensures the continuity and
the sharpness of the edge. Finally, the presence of strong
intensity discontinuities in different directions results in

the reduction of weights lying away from the centre of

the lter kernel. The latter ensures that image details are
preserved in textured areas.
3.3. Filtering of low-detail areas
For low-detail image areas, the adaptive lter
described above is not appropriate. This is due to the
fact that intensity changes in the boundaries of
neighbouring blocks will be wrongly perceived as
intensity discontinuities. In such areas, simple smallscale Gaussian ltering can smooth the boundaries
between coding blocks, as required for achieving better
visual quality, without introducing any noticeable
defects.

4. Experimental results
In order to measure and evaluate the performance of
our approach for blocking artifact reduction, commonly
used metrics, such as the mean square error or signal to
noise ratio, were not employed since they involve pixels
of the entire image and not pixels near the block
boundaries. Instead, a metric called mean squared
difference of slope (MSDS) [9,10] per block is preferred
to be used for the evaluation of the proposed technique.
It involves the intensity gradient (slope) of the pixels
close to the boundary of two blocks. Specically, it is
based on the empirical observation that quantization of
the DCT coefcients of two neighboring blocks increases the MSDS between the neighboring pixels on
their boundaries.
To better understand this metric, consider an 8
8
block f of the input image and a block w vertically
adjacent to f : If the coefcients of the adjacent blocks
are coarsely quantized, a difference in the intensity
gradient across the block boundary is expected. This
abrupt change in intensity gradient across the block
boundaries of the original unquantized image is rather
unlikely, because most parts of most natural images can
be considered to be smoothly varying and their edges are
unlikely to line up with block boundaries. Thus, the
MSDS between the blocks f and w is ew
P
7
2
m0 d1 m  d2 m where d1 m is the intensity slope
across the boundary between the f and w blocks, dened
by d1 m f m; 0  wm; 7 and d2 m is the average
between the intensity slope of f and w blocks close to
their boundaries, dened by d2 m wm; 7 
wm; 6=2 f m; 1  f m; 0=2: The ideas in the
above discussion are applicable to both horizontal and
vertical neighboring blocks. Specically, if blocks w; e
denote the blocks horizontally adjacent to f ; and blocks
s; n present the blocks vertically adjacent to f ; then, the
MSDS which involves both horizontal and vertical
adjacent blocks (hereafter, MSDS1 ) is given by

G.A. Triantafyllidis et al. / Computers & Graphics 27 (2003) 529534

Table 1
MSDSt per block for various test images
Original image

Bit per pixel

(bpp)

Lenna
512
512
MSDSt 1608

0.41
0.31
0.20

3108
3828
5113

2968
3299
4776

2719
2988
4439

Peppers
512
512
MSDSt 2341

0.43
0.32
0.21

2426
3003
3921

2212
2621
3631

2163
2505
3314

Boat
512
512
MSDSt 4393

0.49
0.33
0.23

6195
7339
8609

5972
6848
7893

5602
6547
7424

A: MSDSt of the nonsmoothed reconstructed image; B: MSDSt

of reconstructed image processed by method of [2]; C: MSDSt
of reconstructed image processed by the proposed method.

(a)

533

MSDS1 ew ee es en ; where ee ; es and en are

dened similarly.
We now involve also the four diagonally adjacent
blocks. If nw is a block diagonally adjacent to f ; then, we
dene: enw g1  g2 2 where g1 f 0; 0  nw7; 7 and
g2 nw7; 7  nw6; 6=2 f 1; 1  f 0; 0=2: If nw;
ne; sw and se are the four blocks diagonally adjacent to
f ; the MSDS involving only the diagonally adjacent
blocks (hereafter, MSDS2 ) is MSDS2 enw ene
esw ese where ene ; esw and ese are dened in a similar
way. Thus, the total MSDS (hereafter, MSDSt ) considered in this paper, involving the intensity slopes of all
the adjacent blocks is MSDSt MSDS1 MSDS2 :
Table 1 shows the image name, its size, the MSDSt of
the original image (all in the rst column), the coding
rate (bits per pixel) and the MSDSt per image block for
the cases of (i) the nonsmoothed reconstructed image,
(ii) the reconstructed image processed by method of [2]

(b)

Fig. 1. (a) A portion of the JPEG coded image Lenna at 0:24 bpp: (b) The same portion of Lenna after applying the proposed method
for blockiness reduction.

(a)

(b)

Fig. 2. (a) A portion of the JPEG coded image Peppers at 0:24 bpp: (b) The same portion of Peppers after applying the proposed
method for blockiness reduction.

534

G.A. Triantafyllidis et al. / Computers & Graphics 27 (2003) 529534

and (iii) the reconstructed image processed by the

proposed algorithm. As expected, in JPEG coded images
the value of MSDSt per block increases compared to the
original images, due to quantization. Our approach
shows a signicant reduction of the MSDSt and clearly
outperforms the method proposed in [2].
A visual illustration of the performance of our
method, showing the JPEG reconstructed portions of
Lenna and Peppers images and the corresponding
reconstructed portions of the images processed by the
proposed method is shown in Figs. 1 and 2. These
gures illustrate the efciency of the proposed method.

5. Conclusions
Decompressed images obtained from compressed
JPEG streams, often contain bothersome blocking
artifacts which constitute a serious bottleneck for many
important visual communication applications. This
paper proposes a novel algorithm aiming to reduce such
blocking artifacts. In our approach, better estimates of
the reconstructed DCT coefcients are obtained based
on their observed probability distribution. Subsequently, a novel postprocessing procedure consisting of
high and low detail region classication and a spatial
adaptive ltering is applied for the removal of blocking
artifacts. The type of ltering is determined based on an
estimation of the local characteristics of the coded
image. The criterion used for the evaluation of the
proposed method is the MSDS (i.e., the MSDSt form).
Experimental evaluation of the performance of the
proposed technique clearly demonstrated its ability to
detect and alleviate blocking artifacts effectively.

References
[1] Pennebaker WB, Mitchel JL. JPEG still image data
compression standard. New York: Van Nostrand Reinhold, 1993.
[2] Lee YL, Kim HC, Park HW. Blocking effect reduction of
JPEG images by signal adaptive ltering. IEEE Transactions on Image Processing 1998;7:22934.
[3] Reeve HC, Lim JS. Reduction of blocking artifacts in
image coding. Optical Engineering 1984;23:347.

[4] Ramamurthi B, Gersho A. Nonlinear space-variant postprocessing of block coded images. IEEE Transactions on
Acoustics Speech, and Signal Processing 1986;34:125868.
[5] Xiong Z, Orchard MT, Zhang YQ. A deblocking
algorithm for JPEG compressed images using overcomplete wavelet representations. IEEE Transactions on
Circuits and Systems for Video Technology 1999;7(2):
4337.
[6] Hsung T-C, Lun DPK, Siu W-C. A deblocking technique
for block-transform compressed image using wavelet
transform modulus maxima. IEEE Transactions on Image
Processing 1998;7(10):148896.
[7] Yang Y, Galatsanos NP, Katsaggelos AK. Projectionbased spatially adaptive reconstruction of block-transform
compressed images. IEEE Transactions on Image Processing 1995;4:896908.
[8] Paek H, Kim R-C, Lee S-U. A DCT-based spatially
adaptive postprocessing technique to reduce the blocking
artifacts in transform coded images. IEEE Transactions on
Circuits Systems for Video Technology 2000;10(1):3641.
[9] Minami S, Zakhor A. An optimization approach for
removing blocking effects in transform coding. IEEE
Transactions on Circuits Systems for Video Technology
1995;5:7482.
[10] Triantafyllidis GA, Tzovaras D, Strintzis MG. Blocking
artifact reduction in frequency domain. Proceedings of
2001 IEEE International Conference on Image Processing.
Thessaloniki, October 2001.
[11] Ahumada AJ, Horng R. Smoothing DCT compression
artifacts. SID Digest, 1994.
[12] Lindeberg T, Garding J. Shape from texture from a multiscale perspective. In: Proceedings of the Fourth International Conference on Computer Vision, Berlin, Germany,
May 1993. p. 68391.
[13] Forstner W, Gulch E. A fast operator for detection and
precise location of distinct points, corners and centres of
circular features. In: Proceedings on ISPRS Intercommission Workshop, Interlaken, Switzerland, 1987. p. 281304.
[14] Sampson D, Papadimitriou D, Chamzas C. Postprocessing
of block-coded images at low bitrates. Proceedings of 1996
IEEE International Conference on Image Processing,
Lausanne, Switzerland, September 1996.
[15] Sampson D, Papadimitriou D, Ghanbari M, Dennis T. A
method for enhancing the picture quality for low bit rate
block coded images. In: Proceedings of IEE International
Conference on Image Processing and its Applications,
IPA-95, July 1995. p. 404.
[16] Nitzberg M, Shiota T. Non-linear image ltering with edge
and corner enhancement. IEEE Transactions on Pattern
Analysis and Machine Intelligence 1992;14(8):82633.