High Performance De-interlacing Algorithm Based on Region Adaptive Interpolation

Filter

*Yang, Yang ** Chen, Xiangdong ***Wang, Jin ****Jeong, Jechang

Department of Electronics and Computer Engineering,

Hanyang University, 17 Haengdang-dong, Seongdong-gu, Seoul, Korea
E-mail: [email protected]

Abstract
In order to convert interlaced video into progressive scanning format, this paper proposed a high performance de-interlacing
algorithm based on region adaptive interpolation filter design. Specifically, usage of the 6-tap filter is only for the most complex
region, but for the smooth and regular edge region, much more correlated filter such as 2-tap or 4-tap filter should be used instead.
According to the experimental results, the proposed algorithm has achieved noticeably good performance.
Keywords: De-interlacing, filter design, directional interpolation.

1. Introduction 2. The Existing Methods

De-interlacing is the conversion of video from interlaced format to A. ELA
progressive format [1]. The interlaced format is economical to save
transmission bandwidth and system costs. However, it has defects The ELA method is a traditional edge-based method which
such as line crawling and inter-line twitter in pictures with fine details. considering three directional correlations between two adjacent
The goal of de-interlacing is to reconstruct the missing lines of pixels interlacing lines. In this method, a 3x3 window is shown in Fig.1 in
in interlaced images, reduce aliasing, and increase the vertical which detects vertical, diagonal, and antidiagonal directional
resolution of the images. correlations. The directional correlation measurement C(k) is
In order to overcome these problems, many de-interlacing
techniques have been proposed during the last two decades. These
techniques can be roughly divided into two categories:
inter-field(temporal) and intra-field(spatial) algorithms. Actually, the
inter-field algorithm shows better performance than intra-field
algorithm, but it requires a mass of calculation and memory for
implementation, so it is not suitable for real-time communication.
Consequently, the intra-field algorithm is widely being applied.
The most typical method among intra-filed algorithm is
Edge-based Line Average(ELA)[3]. This method extracts edge determined by equation (1).
information first and then determines its direction to use appropriate
pixels for interpolation. However, it just calculates average values Fig.1 3x3 window in ELA method
between lines after simple edge direction extraction, and it is much
sensitive to small variations of pixel values and uses incorrect edge
information. To resolve these problems, plentiful improved methods C(k ) | P(i  1, j  k )  P(i  1, j  k ) |, k  1,0,1 (1)
which based on ELA are proposed, such as Efficient Edge-based Line
Average(EELA)[4] and Modified Edge-based Line Average(MELA) Among C(-1), C(0), and C(1), the lowest value shows the highest
[5]. Even though these methods show a little bit better performance correlation, which is determined as equation (2).
than ELA, there still has been limitation in complex edge region.
Based on this point, another algorithm, fixed directional interpolation C (m)  arg min(C (k )), k  1,0,1 (2)
filter(FDIF) [2], which apply sinc interpolation filter and weighted
distance scheme has been presented. This algorithm can highly Then, the current pixel is interpolated by equation (3).
improve the performance of de-interlacing images, especially for the
images which consist of much complex edge information. But for the P(i, j )  [ P(i  1, j  m)  P(i  1, j  m)] / 2 (3)
smooth region, this algorithm looks too onerous that it may cost a
good deal of time to achieve a nearly result.
To surmount this issue, the paper presents a high performance B. EELA
de-interlacing algorithm which is based on region adaptive
interpolation filter design. The outline of the paper is as follows. Even though ELA provides a good result in the region where the
Section 2 introduces the previous methods such as ELA, EELA, edge can be correctly estimated, it still gives a poor performance in
MELA. And the proposed method is recommended in details during high spatial frequency region due to the edge detection errors. So,
Section 3. In Section 4, experimental results are presented and Section several methods have been proposed, such as EELA, which
5 gives the conclusion. introduces two more directional correlations are defined shown in
Fig.2.

200
 D. FDIF

This method employs a sinc interpolation filter, which is a 1-D

6-tap interpolation filter with fixed coefficients. Furthermore, it
applies an adaptive weighting scheme according to the distance
between the current pixel to be interpolated and neighboring pixels.
Before adopting the sinc interpolation filter, the edge direction should
be determined using MELA method. The 11x11 window in FDIF
Fig.2 3x3 window in EELA method method is shown in Fig.3.

R | P(i  1, j  1)  P(i  1, j ) |  | P(i  1, j )  P(i  1, j  1) | (4)

L | P(i  1, j )  P(i  1, j  1) |  | P(i  1, j  1)  P(i  1, j ) | (5)

Equation (4) and (5) denote the differences between the two edge
directions, and then compare them to select candidate pixels which
are used to interpolate. There are three cases for consideration as
follows. Here, C(-1), C(0) and C(1) are defined as equation (1).
Case1: R<L

P(i, j )  [ P(i  1, j )  P(i  1, j )] / 2, C (0)  C (1);


P(i, j )  [ P(i  1, j  1)  P(i  1, j  1)] / 2, C (0)  C (1).

Case2: R=L
Fig.3 11x11 window in FDIF method
Apply ELA method.
The three cases in FDIF method is described as follows. By the
Case3: R>L
way, C(-1), C(0) and C(1) are still determined by equation (1).
P(i, j )  [ P(i  1, j )  P(i  1, j )] / 2, C (0)  C (1);
  C (1) 5
C (0) 5

P(i, j )  [ P(i  1, j  1)  P(i  1, j  1)] / 2, C (0)  C (1).  P(i, j )  C (0)  C (1)   h(m)  R(m)  C (0)  C (1)   h(m)  V (m),
 m 0 m 0

 R  min{ R, L,V } & &C (1)  C (0);

C. MELA 
 P(i, j )  C (1) 5
C (0) 5
  h ( m)  L ( m)    h(m)  V (m),
 C (1)  C (0) m0 C (1)  C (0) m0
The EELA method recommends two directional measurements in  L  min{ R, L, V } & &C (1)  C (0);
order to obtain accurate edge directions as much as possible, but it 
 5

 P(i, j )   h(m) V (m),

cannot provide a good PSNR result both in thin edges and textured
regions. The MELA method can get better performance than ELA and
 m 0

Otherwise.
EELA, due to the three directional detections. The extraction of edge
direction in MELA is very similar as EELA, but one more direction is
added as follows.
The coefficients of sinc function are h={3,-17,78,78,-17,3}/128.
Where R(m), L(m) and V(m) denote the antidiagonal, diagonal and
R  [| P(i  1, j  1)  P(i  1, j ) |  | P(i  1, j )  P(i  1, j  1) |] / 2 (6) vertical direction pixels, respectively, which also can be found in
Fig.3.
L  [| P(i  1, j )  P(i  1, j  1) |  | P(i  1, j  1)  P(i  1, j ) |] / 2 (7)
3. The Proposed Method
V  [| P(i  1, j  1)  P(i  1, j  1) |  | P(i  1, j )  P(i  1, j ) |  (8)
| P(i  1, j  1)  P(i  1, j  1) |] / 3 FDIF algorithm presents a better performance than the other
existing algorithms, particularly for the images which containing
Equation (6), (7) and (8) explain the three directional correlations, much complex edge information. When the current pixel which
there are also three cases for application. located in irregular region should be interpolated, the window size
can be enlarged to enhance the correlation between the current pixel
 P(i, j )  [ P(i  1, j  1)  P(i  1, j )  P(i  1, j )  P(i  1, j  1)] / 4, and the neighboring pixels, and it can approximately estimate the
 R  min{ R, L, V } & &C (1)  C (0); target as much as possible. However, for the smooth and regular
 region, a smaller window size is enough to satisfy the directional
 P(i, j )  [ P(i  1, j )  P(i  1, j  1)  P(i  1, j  1)  P(i  1, j )] / 4,
 correlation requirements. Meanwhile due to most of an image with
 L  min{ R, L, V } & &C (1)  C (0); less complex directional correlation, for the smooth region, Line
 P(i, j )  [ P(i  1, j )  P(i  1, j )] / 2, Average(LA) algorithm is a good choice. In our proposed method,

Otherwise. with the purpose of reducing the CPU time, dividing the image into
three regions by equation (9) primarily.
Where, C(-1), C(0) and C(1) also come from equation (1).

201
 D  [| P(i  1, j  1)  P(i  1, j  1) |  | P(i  1, j )  P(i  1, j ) |  (9)  3

| P(i  1, j  1)  P(i  1, j  1) |] / 3  X  ( h2 (m)  R(m)  4) / 8

 m 0

 3

And then, the threshold values are chosen experimentally. Based on Y  ( h2 (m)  L(m)  4) / 8
the two threshold values, the whole image can be segmented into  m 0
 3
three regions, such as smooth region, regular edge region and Z  ( h2 (m)  V (m)  4) / 8
irregular or texture region. The three cases are described in details as  m 0

follows.
Case1: D<=th1 Case3: D>th2
This region can be determined as smooth area, and a 2-tap fixed This region may be much complex edge area, and a 6-tap fixed
interpolation filter with coefficients h1=(1,1)/2 is applied. The same Winer filter with coefficients h3=(1,-5,20,20,-5,1)/32 to be used. A
as Fig.4 represents. 11x11 window should be employed for de-interlacing. And the figure
is same as Fig.3.
In this situation, the edge extraction method and the conditions are
same as Case2, but for the X, Y, Z equations, there is a little bit
difference between them by the coefficients of the 6-tap fixed Winer
filter.

 5

Fig.4 2-tap filter interpolation for smooth region  X  ( h3 (m)  R(m)  16) / 32
 m 0

 5
The interpolation equation is shown as follows, and here the Y  ( h3 (m)  L(m)  16) / 32
notation V(m) denotes the pixels which using to be interpolated, it is  m 0
 5
Z  ( h3 (m)  V (m)  16) / 32
also described in Fig.4.
1  m 0
P(i, j )   h1 (m)  V (m)
m 0

Case2: D>th1 && D<=th2

4. Experimental Results
This region may contains of regular edge information, and we
apply 4-tap fixed Winer filter with coefficients h2=(-1,5,5,-1)/8 to In this section, we present experimental results from region
interpolate the missing pixels. A 7x7 window is used in this condition adaptive interpolation filter design. Experimentally, we set the
as shown in Fig.5. The interpolation formulas are described below. threshold of D as th1=10, th2=15.
And R(m), V(m), L(m) also denote the pixels which should be applied Here, we compare the PSNR values for still sequences between
for interpolation, meanwhile C(-1), C(0) and C(1) like we previous different de-interlacing methods in Table 1. The proposed algorithm
mentioned that still come from equation (1). increases 0.18 DB in average than FDIF method. Meanwhile, due to
the intersected region, the proposed method can reduce the CPU time
as much as 78.57%, which is shown in Table 2.
Fig. 6(a) shows the original image of Airplane with the size
512x512. Fig. 6(b) is the partial image of (a) during the red
rectangular. Fig. 6(c)-(g) are the de-interlacing results by the existing
algorithms. The result of the proposed algorithm is described in Fig.
6(h).

Tabel 1 PSNR Comparison for Still Sequences (DB)

Method Airplane Jets Lena Peppers Toys Average
LA 34.25 39.12 37.67 33.80 33.35 35.64
ELA 33.00 38.98 36.03 34.10 32.47 34.92
EELA 33.32 39.10 36.97 34.18 33.03 35.32
MELA 34.12 39.32 37.96 34.15 33.34 35.78
FDIF 34.70 39.43 38.19 33.85 33.71 35.98
Proposed 34.81 39.64 38.34 34.10 33.93 36.16
Fig. 5 4-tap filter interpolation for regular edge region
Tabel 2 Average Results of CPU Time for Still Sequences(s/frame)
 C (1) C (0) Method Airplane Jets Lena Peppers Toys Average
 P(i, j )  C (0)  C (1)  X  C (0)  C (1)  Z , LA 0.01 0.03 0.01 0.02 0.01 0.02

 R  min{ R, L, V } & &C (1)  C (0); ELA 0.06 0.14 0.06 0.06 0.06 0.08
 C (1)
EELA 0.07 0.20 0.07 0.08 0.07 0.10
 P(i, j )  Y 
C (0)
 Z,
 MELA 0.30 1.11 0.31 0.32 0.31 0.47
 C ( 1)  C ( 0) C ( 1 )  C (0)
FDIF 0.51 2.02 0.55 0.55 0.55 0.84
 L  min{ R, L, V } & &C (1)  C (0); Proposed 0.16 0.30 0.15 0.15 0.13 0.18

 P(i, j )  Z ,
Otherwise.

And the X, Y, Z equations are defined as follows.

202
 vol. 39, no.8, 2101-2105, 2000.
[5] W. Kim, S. Jin and J.Jeong, “Novel intra deinterlacing algorithm
using content adaptive interpolation,” Proc. IEEE, Transaction
on Consumer Electronics, vol. 53, no. 3, pp. 1036-1043, 2007.
[6] S. F. Lin, Y. L. Chang and L. G. Chen, “Motion Adaptive
De-interlacing with Horizontal Motion Detection and ELA with
Median,” Proc. International Symposium on Circuits and System,
Bangkok, Thailand, pp.696-699, 2003.
(a) (b) (c) [7] Y. Y. Jung, B. T. Choi, Y. J. Park and S. J. Ko, “An Effective
De-interlacing Technique Using Motion Compensation
Interpolation,” Proc. IEEE, Transaction on Consumer
Electronics, vol. 46, no. 3, pp. 460-466, 2001.
[8] A. M. Tourapis, O.C. Au and M. L. Liou, “Advanced
De-interlacing Techniques with the Use of Zonal Based
Algorithm,” Proc. Visual Communications and Image
Processing, San Jose, CA, pp.949-958, Jan.2001.
[9] Y. L. Chang, S. F. Lin, C. Y. Chen, and L. G. Chen, “Video
(d) (e) (f)
De-interlacing by Adaptive 4-field Global/Local Motion
Compensated Approach,” Proc. IEEE Trans. Circuits Syst.
Video Technol., vol. 15, no. 12, pp. 1569–1582, Dec. 2005.
[10] T. S. Chong, O. C. Au, T. W. Chan and W. S. Chau, “A
Spatial-Temporal De-interlacing Algorithm,” Proc. IEEE Int.
Conf. Multimedia Expo, pp. 249–252, Jul. 2005.
[11] H. Yoo and J. Jeong, “Direction-oriented Interpolation and Its
Application to De-interlacing,” Proc.IEEE Trans. Consum.
(g) (h) Electron., vol. 48, no. 4, pp. 954–962, Nov. 2002.
Fig. 6 Original image and several de-interlacing results. (a) Original [12] M. K. Park, M. G. Kang, K. Nam and S. G. Oh, “New Edge
image of Airplane 512x512. (b) Original partial image. De-interlacing Dependent De-interlacing Algorithm Based on Horizontal Edge
results of (c) LA, (d) ELA, (e) EELA, (f) MELA, (g) FDIF, (h) the Pattern,” Proc.IEEE Trans. Consum. Electron., vol. 49, no. 4, pp.
proposed method. 1508–1512, Nov. 2003.

5. Conclusion
This paper presents a high performance de-interlacing algorithm
based on the region adaptive filter design. The proposed method first
separates the smooth, regular edge and complex edge region by
simple correlation between 6 pixels during a 3x3 window, and then
using the 2-tap, 4-tap and 6-tap filters for the three regions
respectively. Because most of an image is less complex area, so the
region segmentation can highly increase the efficiency. Experimental
results show the proposed algorithm has a better performance than the
previous algorithms.

6. Acknowledgments

This research was supported by the MKE(The Ministry of

Knowledge Economy), Korea, under the ITRC(Information
Technology Research Center) support program supervised by the
NIPA(National IT Industry Promotion Agency)
(NIPA-2011-C1090-1100-0010).

7. References

[1] G. de Haan and E. B. Bellers, “Deinterlacing-An overview,” Proc.

IEEE, vol. 86, no. 9,pp. 1839-1857, Sep. 1998.
[2] S. Hong, S. Park, J. Jang and J. Jeong, “Deinterlacing algorithm
using fixed directional interpolation filter and adaptive distance
weighting scheme,” Proc. SPIE Optical Engineering, vol. 50,
no.6, June 2011.
[3] C. J. Kuo, C. Liao and C. C. Lin, “Adaptive interpolation
technique for scanning rate conversion,” IEEE Trans. Circuits
and Syst. Video Technol. vol. 6, no. 3, 317-321, 1996.
[4] T. Chen, H. R. Wu and Z. H. Yu, “Efficient deinterlacing
algorithm using edge-based line average interpolation,” Opt. Eng.

203