Image Processing on ECG Chart for ECG Signal Recovery

TW Shen, TF Laio

Tzu Chi University, Hualien, Taiwan

paper charts, web-based and DICOM-based ECG

Abstract screenshots, and PDF files to the digital time database for
Medical imaging plays an indispensable role on data extraction, compression, archive, and storage.
medical informatics. Most of imaging processing Moreover, other issues, such as ECG biometrics,
technologies is focused on the identification of the physiological system modelling, simulation and computer
locations of diseases on MRI, CT, PET, and SPECT aid diagnosis (CAD), may also be concerned to help
images. However, only a few researches focused on one biomedical engineers for developing tools. For examples,
dimension signal recovery or reconstruction of electronic a sudden cardiac death (SCD) simulator can be built by
signals. Spatial and frequency methods were provided to using Labivew program with a D/A converter to evaluate
process on colour or gray-level electrocardiogram a portable ECG embedded system [9] by using real
charts, including threshold segmentation and 2D Fourier hospital paper charts. However, only several researches
transform techniques. Finally, the performance was [3-7] focused on one dimension signal recovery or
evaluated by percentage of root mean square (PRD) for reconstruction of electrical signals from clinical paper
calculating waveform similarity. The results show that imaging.
the spatial method offered better performance than the To retrieval ECG signal from scanned images,
frequency method, especially if the RGB colours in chart according to previous researches, Zhang et al. [3]
are kept. Based on our investigation, the linear obtained the ECG restoration by applied a sequential
interpolation method provided the best interpolation with image processes (histogram filtering, gradient image and
less overall PRD. grid pattern extraction). Mitra et al. [6] generated digital
time database from paper ECG records. Their method
includes binarization, background separation, the
1. Introduction thinning algorithm, and raw data extraction. After ECG
Medical imaging plays an indispensable role on data extraction, they analyzed ischemic subjects by using
medical informatics. Nowadays, most of imaging 2D Fourier transform method.
processing technologies is focused on the enhancement In general, the percentage of root mean square (PRD)
and identification of the locations of diseases on MRI, is a very common index to evaluate the error of ECG
CT, PET, and SPECT images [1] to assist the physicians data compression. It is used to evaluate on different
on medical diagnosis. More and more hospitals have image extraction methods, which has not yet been done
introduced picture archiving and communication systems in previous researches. Hence, the aim of this research
(PACS) to the storage, retrieval, distribution and is to recognize ECG phantom on charts or images, to
presentation of images, and the images are most recovery raw ECG signal, and to evaluate the
commonly stored as DICOM (Digital Imaging and performances of two different imaging processing skills.
Communications in Medicine) format. A full PACS also
interfaces with hospital information system (HIS), which 2. Methodology
is a comprehensive, integrated information system for
hospital management and patient care [2]. This research proposed two ECG signal retrieval
Even integration with HIS and PACS systems, the techniques based on spatial and frequency oriented
technique of electrocardiogram (ECG) signal extraction methods. The spatial-oriented method mainly applied
from paper-written ECG image developed since 1987 [3- histogram segmentation, missing pixel replenishment,
8] is still needed to preserve the large accumulation of and interpolation. The frequency-oriented method
ECG paper charts in some counties [3]. Some systems majorly applied 2D Fourier transform, 2D lowpass filter,
were developed which can transfer wave data recorded on 2D median filter, missing pixel replenishment, and
paper to a digital time data [3–7]. In addition, the image interpolation. In order to evaluate two systems fairly, a
processing techniques are able to further transfer the ECG standardized database for both methods is essential.

ISSN 0276−6574 725 Computers in Cardiology 2009;36:725−728.

2.1. Experiment Database resolution is 990 x 124, the interpolation is applied to fill
the number of points in x-axis up to 5000 points (10 sec.*
The MIT/BIH Sudden Cardiac Death Holter Database 500 sps). Finally, five interpolation functions and their
is a collection of long-term ECG recordings of patients PRDs are calculated. The equation (1) defined PRD
who experienced SCD during the recordings. The function which describes the difference between the
database contains a total of 23 complete half-hour Holter reconstructed and original signal as follows,
recordings which become available to researchers [10-
11]. The database not only provides ECG signal at 500
 n 2
 ∑ [ xorg (i ) − xrec (i )] 
Hz sampling rate, but the PhysioNet website
demonstrates the ECG image (PNG format) that contains …(1)
PRD =  i =1 n 
10 second ECG. The web-ECG image database was used
to develop our ECG extraction system and those images


∑i =1
[ x org (i )] 2 

were trimmed to meet dimension at 990 x 124. These
arrhythmia images include tachyarrhythmia, tachycardia where n is the number of samples and xorg and xrec are
(VT), ventricular flutter (VFL), ventricular fibrillation samples of the 1D original and reconstructed data
(VFib), or atrial fibrillation. For this research, the ten- sequences. Figure 2 shows a processing example from an
second ECG image (involved before and after VF onset original color image (figure 2a) to a 1D recovered ECG
time or randomly selected ten-second arrhythmia for no signal (figure 2d).
VF onset cases) were kept in our database in order to
exam both types of normal sinus and arrhythmia
waveforms. Finally, the recovery results were compared
with machine recording data with 500Hz sampling rate.

2.2. Spatial-oriented method (a)

(b)

(c)

(d)
Figure 1. The flow diagram of spatial-oriented method
Figure 2. The procedures of spatial-oriented method: (a)
In spatial-oriented process (figure 1), a color image an original color image; (b) a gray tone image after
may directly eliminate the red component to remove selecting a suitable threshold on histogram; (c) missing
gridlines or convert to a gray or binary tone image after pixel replenishment and interpolation; (d) a recovered
selecting a suitable threshold with the help of histogram and calibrated ECG signal.
analysis. A suitable threshold selection after histogram
analysis can mostly remove the background and gridlines
of ECG charts. Then, the computer program searches 2.3. Frequency-oriented method
background-removed images for missing points on the The basic idea of the frequency-oriented method is to
columns without any ECG pixel points. The missing separate the ECG image into three components in the
point has been replenished by the darkness point of the image (2D) frequency sense, including (1) high frequency
same column on pervious gray scale image. If the same gridlines [1], (2) an ECG waveform (mainly in low
column has more than one pixel, the thinning algorithm is frequency), and (3) noise (mainly salt and pepper noise).
applied. Next, the co-ordinate of each black pixel was Hence, 2D Fourier transform (2D FT) plays an important
extracted and then the values were calibrated according to roles in this method, which is applied on a color, gray-
the co-ordinate system of the chart. Because of the image scale, or binary ECG image by using equation (2).

726
 ∞ ∞ transfers frequency domain to time domain and the result
F (u, v) = ∑ ∑ f [m, n]e − j 2π ( umx0 + vny0 )
…(2) shows in figure 6a. Then the median filter is applied to
m = −∞ n = −∞ remove salt and pepper noise as shown in figure 6b. Also
where x0 and y0 are the spatial intervals between a suitable threshold selection removes the background
consecutive signal samples in the x and y directions, and gridlines (figure 6c). Then, follow the procedure as
respectively. pervious mentioned. The computer program searches for
missing points and replenished by darkness points (figure
6d). Because this method provides more scattering noise,
the dark pixel distances between adjacent columns are
computed. If the same column has more than one pixel
and the distance is over or below thresholds, then the
pixel is eliminated (figure 6e). Next, the co-ordinate of
each black pixel was extracted and then the values were
calibrated according to the co-ordinate system of the
chart (figure 6f).

Figure 3. The flow diagram of frequency-oriented method

(a)
In frequency-oriented process (figure 3), 2D FT
projects the image from spatial to frequency domain. In
the frequency domain, an elliptic-shaped low-pass (LP) (b)
filter was applied on the frequency response map to omit
the high frequency component as shown in figure 4. The
center of frequency response map presents the low-
frequency component and the outside elliptic-cycle (c)
describes the high-frequency component. Figure 5 shows
the three dimension plot on the LP filter with crisp
threshold on the filter edge. The formula of the LP filter (d)
is written as equation (3).

(e)

Figure 4. An elliptic-shaped LP filter applied on

rectangular-shaped frequency response map
(f)
Figure 6. The procedures of frequency-oriented method:
(a) an image applied 2D FT, LP, and IFT; (b) an image
applied a median filter; (c) background elimination (d)
missing pixel replenishment; (e) extra pixels elimination
and interpolation; (f) a recovered and calibrated ECG
signal.
Figure 5. A 3D LP filter plot

(u − xcenter ) 2 (v − ycenter ) 2 3. Results

+ < r …(3)
wide * 0.4 length * 0.5 Our proposed methods processed on color or gray-
level electrocardiogram charts. To summary, first,
where u, v are the coordinates in the frequency domain. threshold segmentation or 2D Fourier transform were
applied to remove the chart grids and marker lines.
The coordinate ( xcenter , y center ) is the origin of the
Second, column searching on missing pixels or dots was
frequency response map. The r value is suggested as 50. utilized, which were refilled by image matching and pixel
After filtering, the inverse Fourier transform (IFT) interpolation. Then, the 2D image was converted to 1D

727
signal by using five interpolation methods (cubic, dispensable interpolation. In addition, according to this
v5cubic, pchip, spline, linear, and nearest) that rebuild investigation, the linear interpolation is the best
sampling frequency to 500 sps. The example results are interpolation method with less overall PRD.
plotted in figure 7 & 8.
400

300

200

100

0
Acknowledgements
V
m

-100

-200

-300

-400

-500
Thanks assistant professor Sheng-Fang Huang in
department of medical informatics of Tzu Chi University
-600
1500 1600 1700 1800 1900 2000
Number of points

Figure 7. Spatial-oriented method – the recovery signal who provides some technical suggestions. This work was
(red) versus the original signal (blue) supported by the Taiwan National Science Council under
400

300

200
Grant 98-2221-E-320-002.
100

0
V
m

-100

-200

-300

-400

-500

-600
1400 1500 1600 1700
Number of points
1800 1900 References
Figure 8. Frequency-oriented method – the recovery [1] Gonzalez RC, Woods R.E, Digital Image Processing, 3rd
signal (red) versus the original signal (blue) Ed., Addison-Wesley, Reading, MA, Longman, New York,
2000.
Finally, the performance was evaluated by percentage [2] Wang, C., K. Ohe, et al., An ECG storage and retrieval
of root mean square for calculating waveform similarity. system embedded in client server HIS utilizing object-
Twenty-three ten-second charts (about 990 by 124 pixels) oriented DB, J Med Syst., 1996: 20(1): 35.43.
were processed and compared with original ECG data for [3] Zhang ZN, Zhang H, and Zhuang TG, One-dimensional
performance evaluation. The spatial and frequency signal extraction of paper-written ECG image and its
archiving, SPIE, 1987:845:419.422.
methods provided each best result on average 45.46%
[4] Tezuka A., A study on transformation system of wave data
and 54.33% PRD by using linear interpolation. The detail recorded on paper into digital time data, Rec. Electr.
results are listed on Table 1. Commun. Eng. Conversazione 1989:57:343.
Table 1. Average PRD (%) on six different interpolation [5] Wideman LE, Freeman GL, A to D conversion from paper
records with a desktop scanner and a microcomputer,
methods Comput. Biomed. Res., 1989:22: 393.
Average PRD (%) [6] Mitra S, Mitra M, Chaudhuri BB. Generation of digital
on six different interpolation methods time database from paper ECG records and Fourier
transform-based analysis for disease identification.
Spatial –based method 2D FFT based method Computers in biology and medicine. 2004 Oct:
cubic 45.92 % cubic 55.07 % 34(7):551.60.
v5cubic 46.07 % v5cubic 55.39 % [7] F. Cheng, R. Gao, Y. Wei, Tracing algorithms of ECG
pchip 45.91 % pchip 55.04 % tracks from scanned image, J. Biomed. Eng., 2001:18: 306.
[8] O. Pahlm, L. Sornmo, Data processing of exercise ECG,
spline 46.09 % spline 55.71 %
IEEE Trans Biomed. BME 1987:34: 158.
linear 45.46 % linear 54.33 % [9] Lin TH, Ko TF, Tsao YT, and Shen TW, A Mutiple-
nearest 46.79 % nearest 57.07 % Functional ECG Simulator Based on LABVIEW Program,
2006 International Symposium on Biomedical
Engineering, Taiwan, 2006: 10421.
4. Discussions and conclusions [10] Shen TW, Shan HP, Lin CH, Ou YL, Detection and
Prediction of Sudden Cardiac Death (SCD) for Personal
Overall, our image processing methods successfully Healthcare, 29th Annual International Conference of the
converted 2D ECG images into one-lead ECG signals to IEEE Engineering in Medicine and Biology Society, 2007:
make further CAD analysis and modelling simulation 2575.2578.
possible. The results show that the spatial method offered [11] Shen TW, Lin CC, Shen HP, Ou YL, and Lin CH. A
better performance than the frequency method, especially personal Sudden Cardiac Death (SCD) detector based on
if the RGB colours in chart are kept. Unlike spatial ECG biometric technology, World Congress on Med. Phys.
method, 2D Fourier transform method contains more salt- and Biomed. Eng. 2006:1141.1144.
and-pepper noise on chart images, as well as scattering
Address for correspondence
noise. Our recovered signals also include amplitude
Tsu-Wang Shen, Ph.D.
displacement and inconsistent waveform phase shifting. Department of Medical Informatics
The above factors cause the higher average PRD. Tzu-Chi University
However, for improving the system performance, high 701, Sec. 3, Jhongyang Rd., Hualien, 97004, Taiwan
resolution images is a necessary condition to reduce [email protected]

728