PROCEEDINGS OF SPIE

SPIEDigitalLibrary.org/conference-proceedings-of-spie

Comparison of two methods for

evaluating image quality of chest
radiographs

Patrik Sund, Susanne Kheddache, Lars Mansson, Magnus

Bath, Ulf Tylen

Patrik Sund, Susanne Kheddache, Lars Gunnar Mansson, Magnus Bath, Ulf
Tylen, "Comparison of two methods for evaluating image quality of chest
radiographs," Proc. SPIE 3977, Medical Imaging 2000: Physics of Medical
Imaging, (25 April 2000); doi: 10.1117/12.384518

Event: Medical Imaging 2000, 2000, San Diego, CA, United States

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 05 Jan 2023 Terms of Use: https://www.spiedigitallibrary.org/terms-of-use

 Evaluation of image quality of a new CCD-based system for chest
imaging
Patrik sd'*, Magnus Bath", Susanne KheddacheC, U1fTy1én' andLars Gunnar MãnSSOfla

Departments of aMedical Physics and Biomedical Engineering and CRadiology

Sahlgrenska University Hospital, Goteborg, Sweden
bDepament of Radiation Physics, Goteborg University, GOteborg, Sweden

ABSTRACT

The Imix radiography system (Oy Imix Ab, Finland) consists of an intensifying screen, optics, and a CCD camera. An
upgrade ofthis system (Imix 2000) with a red-emitting screen and new optics has recently been released. The image quality
of Imix (original version), Imix 2000, and two storage-phosphor systems, Fuji FCR 9501 and Agfa ADC7O was evaluated in
physical terms (DQE) and with visual grading of the visibility of anatomical structures in clinical images (141 kV). PA
chest images of 50 healthy volunteers were evaluated by experienced radiologists. All images were evaluated on Siemens
Simomed monitors, using the European Quality Criteria. The maximum DQE values for Imix, Imix 2000, Agfa and Fuji
were 1 1%, 14%, 17% and 19%, respectively (141 kV, 5 Gy). Using the visual grading, the observers rated the systems in
the following descending order: Fuji, Imix 2000, Agfa, and Imix. Thus, the upgrade to Imix 2000 resulted in higher DQE
values and a significant improvement in clinical image quality. The visual grading agrees reasonably well with the DQE
results; however, Imix 2000 received a better score than what could be expected from the DQE measurements.

Keywords: CCD Technique, Chest Imaging, Digital Radiography, DQE, Image Quality, Visual Grading Analysis

1. INTRODUCTION
Among the various types of detection principles for digital radiography, the CCD technique has been used for
mammography' but is not commonly used for general radiography. The CCD technique in radiography is applied by
detecting the light emitted from an intensifying screen. The optical coupling can be performed by mirrors and/or lenses with
or without fiber optics.
ACQUSITIOW CO41RO1
COPJVER SlAt Ot
COMPUTEFt

Figure 1: Imix principle. Courtesy of Oy Imix Ab.

oy Imix Ab in Tampere, Finland, has specialized in developing and producing CCD based x-ray detectors for medical use.
They currently manufacture two types of detectors, a chest x-ray unit and a universal detector, the principle is shown in
Figure 1. The first generation of detectors is named Imix and the latest version Imix 2000 — in USA it is formerly known as
Trex 4000M, a name which is no longer in use. The first generation of Imix was released in the mid 90's and has an
intensifying screen made of Gd2O2S:Th. This screen is emitting light in the green part of the spectrum while the CCD is
*
Correspondence: Email: patrik.sundradfys.gu.se; Telephone: +46 31 3424023; Fax: +4631 822493

In Medical Imaging 2000: Physics of Medical Imaging, James T. Dobbins III, John M. Boone,
Editors, Proceedings of SPIE Vol. 3977 (2000) • 1605-7422/00/$1 5.00
437

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 05 Jan 2023

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 more sensitive to red light. The latest version ofthis system, Imix 2000, is therefore equipped with a screen of Gd2O2S:Eu3
which is red light-emitting to better match the sensitivity of the CCD. The emission spectrum of this screen essentially has
two peaks, 625 nm and 705 nm. At 40 keV, the efficiency of converting x-ray energy to light energy is approximately 15%.
For every 40 keV x-ray quantum absorbed, about 3000 light-photons of energy 2eV are created. In Imix 2000, also new
optics and improvements of the image processing software (ContextVision®). The matrix size is 2000 x 2000 pixels
covering an input screen ofsize 400 x 400 mm. The A/D conversion is performed with 16 bits.

This work aims at evaluating and describing the imaging properties ofthe chest x-ray detectors Imix and Imix 2000. For the
evaluation, both physical and observer performance tools were used. Comparisons are made with two commercially
available storage phosphor systems.

2. METHODS
The performance of Imix and Imix 2000 was compared with two storage phosphor systems, Fuji FCR 9501, with image
plates of generation V, and Agfa ADC7O with image plates of type MD1O. Imix, Imix 2000 and Fuji FCR 9501 are all
dedicated chest stands, but since Agfa utilizes image plates in portable cassettes, a regular chest stand, Siemens Vertix-E,
was used for the Agfa exposures. The read out procedure was performed in standard resolution mode for all systems. The
pixel size was 200 im for all systems except Agfa (170 tm).

The imaging properties of the systems were evaluated by two different methods. The visual image quality was determined
by employing European Quality Criteria2 on a set of images of healthy volunteers. Detective Quantum Efficiency (DQE)
and Signal-To-Noise Ratio (SNR) over the image surface were used to describe the SNR transfer characteristics of the
systems.

2.1 Visual Image Quality

PA chest images of 50 healthy volunteers were collected with all four systems. Due to practical reasons, one x-ray tube was
used for Imix, Imix2000 and Agfa while another was used for Fuji. However, both x-ray tubes were adjusted to have the
same half-value layer (HVL): 5.5 mm Al at 80 kV. The tube voltage was 141 kV for all systems, a beam quality which is
commonly used for chest x-ray examinations in Sweden. With all systems, automatic exposure control (AEC) was used in
the following way: The AEC on the Vertix stand was calibrated to give a correct chest exposure for a 200-speed screen-film
system. For this exposure, the entrance air kerma (K) to a CDRH chest phantom "LucAl"3 was measured using a solid-
state detector (Solidose 200, RTI Electronics, Mölndal, Sweden). All ABCs were then set to give this K value with the
same phantom. Entrance air kerma to all volunteers was measured with the same solid-state detector.

All images were stored digitally and soft-copy reporting was used in the evaluation process. Two Siemens Simomed
monitors with a maximum brightness of 160 cd/rn2 were used in a room with a background illuminance less than 40 lux. Six
radiologists were involved, five ofwhich were specialists in thoracic radiology.

The visual image quality was rated by employing European Quality Criteria, modified according to previous findings4, see
Table 1 and Figure 2. These criteria are based on the visibility of normal anatomy and are believed to be able also to
describe the ability ofthe system to reproduce pathological lesions.

Table 1: Modified European Quality Criteria for chest imaging used in this study.
Sharp visualization: Anatomical details are clearly defined (details clear).
Visualization: Characteristic features are detectable but details are not filly reproduced
(featuresjust visible).
1. Sharp visualization ofthe vessels seen 3 cm from the pleural margin.
(Peripheral vessels).
2. Sharp visualization of the vessels seen en face in the central area.
(Central vessels).
3. Visualization of the carina with main bronchi. (Carina).
4. Visualization of the thoracic vertebrae behind the heart.
(Thoracic vertebrae).
5. Sharp visualization of the pleural margin. (Pleural margin). Figure 2: Position of quality criteria.

438

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 05 Jan 2023

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 For the visual image quality evaluation, Visual Grading Analysis (VGA) was used in two variants: absolute and relative
rating. In the absolute rating, each image was viewed individually and each quality criterion was rated on a scale from 1 to 4
as follows:
Absolute rating: Meaning
1: Not visible
2: Poorly reproduced
3: Adequately reproduced
4: Very well reproduced

For a given system an Image Criteria Score (ICS) was calculated as

Ics = °=' =' c=1

OxIxC
t (1)

where is the absolute ratmg for a particular observer (o), image (i) and criterion (c). 0, I and C are the number of
observers, images and criteria, respectively.

For the relative rating, each image was compared to a reference image. Images from four systems can be paired in six
different combinations. In order to reduce bias and learning effects of the observers, the reference image was chosen
randomly for each pair. Also, in this way the risk of choosing the best or the worst image as the reference is eliminated. The
observers were not informed of the origin of the images in each pair.

The relative rating scale used was:

Relative rating: Meaning
The structure in the image is
-2: much worse reproduced
- 1: worse reproduced
0.• equally reproduced
+1: better reproduced
+2: much better reproduced
than in the reference image.

For a given system a Visual Grading Analysis Score (VGAS) was calculated as

VGAS = °=' =' =' (2)

OxlxC
where is the relative rating for a particular observer (o), image (i) and criterion (c). 0, I and C are described above.
Since the actual relative rating is dependent on the quality ofthe reference image, a randomization ofreference images as
used here may cause biased results. In order to avoid possible asymmetry in the number oftimes a given system is chosen as
reference, both images in the evaluated pair are actually given a rating; the observer rates the image compared to the
reference image and in the subsequent calculation process the reference image itself is given the same score with the
opposite sign. Thus, is calculated with a positive sign ifthe image was compared to the reference image and with a
negative sign for the reference image itself. It is assumed that the grading for each pair would be ofthe same magnitude but
with opposite sign ifthe compared and reference image changed places. With this kind ofsymmetry the relative score for a
system will be compared with the average score ofall systems.

The significance of differences between different systems was calculated using the analysis of variance (ANOVA) in
conjunction with a method for multiple comparisons in order to reduce the risk of random significance, the Newman-Keuls
test5 (cz=O.05). The multiple significance tests performed in the Newman-Keuls method offer a good protection against
significant differences between imaging systems appearing "by chance". The statistical analysis was made using the
software Statistica® Release 5.1.

439

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 05 Jan 2023

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 2.2 Physical Image Quality
The calculation ofDQE follows, to a large extent, the method described by Dobbins6.

The DQE can be calculated as

MTF(u)2
DQE(u D) = (3)
NNPS(u, D) x SNR(D)2

where u and D are spatial frequency and dose respectively, MTF is the Modulation Transfer Function and NNPS is the
Normalized Noise Power Spectrum, i.e. NPS divided by the large area signal in the image used for the NPS calculation.
SNR is the Signal-to-Noise Ratio ofthe incoming radiation.

For the MTF determination, an approximately 10 tm wide slit was used. The slit edges were made of 2 mm thick tungsten
separated by aluminum foil. The slit was tilted slightly compared with the pixel matrix to obtain the pre-sampling MiT that
was used in the DQE calculations. This method is described by Fujita et a!7. The grid was always removed so that the slit
could be positioned as close as possible to the image-producing surface.

The NPS was calculated as the 2-dimensional Fourier transform of the noise image. Flat-field images at K values between
0.5 tGy and 100 j.tGy at the entrance surface of the image detector were used for the calculation as well as for the
linearization of all images. In every image, the central 10242 pixels were used. This area was divided into 64 squares
("snapshots") ofsize 1282 and the NPS was calculated as the average ofthe NPS from each ofthese small areas. In this way,
NPS can be determined with greater precision. Furthermore, the effect of any background trends is reduced by subtracting a
low-order polynomial function from every snapshot. To obtain a 1-dimensional NPS, a few lines around (not on) the central
axes were used.

Another use of the flat-field images is to evaluate the SNR over the image surface. The SNR from several snapshot areas
can be calculated to see the noise behavior in different parts of the image. The SNR is calculated as the average value
divided by the standard deviation in each snapshot.

The spectrum of the incoming radiation was obtained from a spectrum-simulating program, based on tables from IPEM
report no. 788, and the energy-weighted SNR was calculated as

SNR2 = (Jq(E) xE dE)z (4)

dE
Jq(E)xE2

where q(E) is the number of photons with energy E.

All flat-field images were exposed with a tube voltage of 141 kV and a total filtration of 19 mm Al which is close to the
beam quality at the detector surface when exposing patients. The slit images used for MTF calculations were, however,
exposed at 70 kVp because of too high x-ray penetration of the slit edges at higher tube voltages.

440

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 05 Jan 2023

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 3. RESULTS
3.1 Visual Image Quality
Results from the VGA with absolute and relative ratings can be found in Figures 3 and 4, respectively. Both VGA methods
basically show the same result: Imix 2000 was rated as the best system for reproducing the peripheral and central vessels
(criteria 1 and 2) while Fuji 9501 was rated best for criteria 3 to 5. The first generation of Imix was rated worst for all
criteria. It is worth noticing that the rated order of the tested systems is the same for all criteria in the absolute rating as itis
in the relative rating. The differences between systems were generally statistically significant. In Figures 3 and 4, horizontal
solid lines join those systems that could not be statistically separated from each other.

Absolute rating

4,0 -
3,5

2 3,0.
2,5 Fuj 9501
0
C.) Irnx 2000
Agfa ADC7O
CkThX

U) 1,0.
'C
0,5.
0,0.
Peripheral Central Canna Thoracic Pleural Average of
vessels vessels vertebrae margin all criteria

Criterion

Figure 3: Absolute rating for four imaging modalities and five image quality criteria.
Horizontal bars indicate that no statistical difference was found. A score of 3.0 indicates
that the image is adequately reproduced, see Table 1.

Relative rating

2,00

1,50
Cl)
<1,00

1:0.I11k FL iL
C)
100
-1,50
-2,00
Peripheral Central Carina Thoracic Pleural Average of
vessels vessels vertebrae margin all criteria
Criterion

Figure 4: Relative rating for four imaging modalities and five image quality criteria. The
zero level is the average score of all systems tested. Horizontal solid lines indicate that no
statistical difference was found.

441

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 05 Jan 2023

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 3.2 Physical Image Quality
The MTF curves up to the Nyquist frequencies for the four different systems are shown in Figure 5. The MiT was in all
cases measured in the center of the image. A slight improvement (approximately 20%) for Imix 2000 as compared to Imix is
demonstrated. However, the MTF characteristics for the two storage phosphor systems are considerably better. Note the
substantial difference between Fuji 9501 and Agfa ADC7O to the advantage of Fuji 9501.

MTF

1,0

/
0,9
0,8
0,7
0,6 - —Imix2000
—tmix
0,5
0,4 =AgfaADC7O
0,3 A
0,2
0,1
0,0
0,0 0,5 1,0 1,5 2,0 2,5 3,0
Spatial frequency (1/mm)

Figure 5: MTF curves for four imaging modalities as a function of frequency. Tube
voltage and total filtration were 70 kV and 19 mm Al, respectively.

In Figure 6, DQE as a function of spatial frequency is shown for the four imaging modalities at a K value of 5 tGy at the
entrance surface of the image detector. The maximum DQE values, if extrapolated to zero frequency, for Imix, Imix 2000,
Agfa ADC7O and Fuji 9501 were 1 1%, 14%, 17% and 19%, respectively. Although the improvement ofDQE for Imix 2000
as compared to Imix is considerable, the two storage phosphor systems are distinctly better than both Imix systems in terms
of DQE. At frequencies up to approximately 1.5 mnf' the two storage phosphor systems have similar DQE properties but at
higher frequencies the Fuji FCR 9501 is the better one.

DQE at 5 1Gy

20%
18%
16%
14%
12%
LU
a 10%
8%
6%
4%
2%
0%
0,0 0,5 1,0 1,5 2,0 2,5 3,0

Spatial frequency (1/mm)

Figure 6: DQE-values for four imaging modalities as a function offrequency at 5 j.tGy.

Tube voltage and total filtration were 141 kV and 19 mm Al, respectively.

442

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 05 Jan 2023

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 A comparison between SNR surfaces for Imix and Imix 2000 over the entire 400 x 400 mm2 image is shown in Figure 7.
Each point on the surface represents the SNR at the corresponding location in the image. Imix 2000 is improved both in
terms ofabsolute SNR levels and vignetting towards the edges ofthe image.

Figure 7: SNR surfaces for Imix and Imix 2000 respectively. Z-axis represents SNR, X- and Y-axis represents pixel
location. The doses to the image entrance planes were approximately 4 p.tGy.

4. DISCUSSION
Since a homogenous phantom was used for the calibration of the AEC systems, differences in actual patient exposures can
occur due to e.g. different chamber positions for the different systems. The average entrance air kerma to the patients was
95 piGy for Fuji, 102 j.tGy for Imix, 103 iGy for Imix 2000 and 76 iGy for Agfa. Thus, the dose for three of the systems
was quite similar, but the position of the AEC chambers at the Vertix chest stand (used for the Agfa images) was such that
the clinical exposures resulted in lower entrance doses. The effect caused by this difference in dose is difficult to predict but
the results for Agfa in the visual grading might have been somewhat better if the images would have been produced at the
same dose level as the other systems.

Access to unprocessed data from the digitizer is a prerequisite for determination of DQE. Agfa ADC7O, developed in the
early 90's when the need for a large memory capacity was not realized, suffers from truncation of high frequencies due to
memory shortage. This has probably no visual effect on patient images but it makes it impossible to determine the MTF
from Agfa ADC7O slit images. The Agfa image plates where therefore read out in a new digitizer, Agfa ADC Solo. Since
the ADC Solo is developed recently, the DQE values presented for Agfa ADC7O in Figure 6 may be somewhat
overestimated.

The main modality for chest imaging at Sahlgrenska University Hospital, where this study took place, is Fuji FCR 9501.
Agfa ADC7O is used mainly for orthopedic imaging but also for bedside chest examinations. The image processing
parameters for these systems have been constantly adjusted for many years. The imaging modalities from Oy Imix are new
to all involved radiologists and the processing software has been adjusted to meet our radiologist's demands for a
considerably shorter time. Although the radiologists made every possible effort to read the images without preconceived
opinions regarding the overall appearance ofthe images, there is a potential risk that they were biased in this case.

The optical coupling in a CCD-based system is dependent on its design, in the way that the optical demagnification and the
quality of lenses and mirrors determine the extent of the quantum sink. In CCD-based systems like Imix, there is a risk of
loosing so many quanta that the quantum efficiency drops below that of the screen itself. If so, the DQE will decrease
accordingly. The new optical design in Imix 2000 obviously has improved the light collection efficiency compared to Imix
(cf. Figure 7).

Since the DQE is a function of both dose and spatial frequency, it is difficult to make any correlation between this quantity
and the visual image quality, which is also a very complex quantity. It is however clear that Fuji has the highest DQE while
the first version of Imix has the lowest and that the same result is true for the visual grading. While the outcome of the

443

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 05 Jan 2023

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 visual grading shows that the image quality for Imix 2000 is at about the same level as for Fuji, this is not the case forthe
DQE results, where there is a large difference between the two. Obviously, it is possible to produce x-ray images of good
quality at normal (200 speed) dose levels with image detectors with rather poor SNR transfer characteristics. However, the
potential for considerable dose savings with such a detector will be limited.

AlthoughDQE is a valuable quantity for describing the imaging properties of an image detector, the subjective image
quality is influenced by the image processing software and its settings. Therefore, an unequivocal relation between the
physical properties and the subjective image quality cannot always be expected. It is encouraging that, despite the use of
two complex methods for describing image quality, the correlation between the outcome of the methods was reasonably
good.

5. CONCLUSIONS
The upgrade from Imix to Imix 2000 resulted in both a higher DQE and a large improvement in the visual image quality.
Also, the SNR in the peripheral parts ofthe image, especially in the corners, was considerably improved. Despite the rather
poor SNR transfer characteristics (DQE), the anatomical details, as described by the quality criteria, were in general rated as
"adequately reproduced".

ACKNOWLEDGEMENTS
The authors would like to thank the following radiologists, technicians and engineers for their participation in the study:
Agneta Flinck, Johan Formgren, Bengt Gottfridsson and Rauni Rossi Norriund for reading the images; Margareta Widell
and Lena BjOrneld for taking care ofthe x-ray exposures and numerous other practical matters; Anders Karisson for writing
the software for the soft-copy evaluation. Many persons at Imix, Fuji and Agfa have been co-operative by helping us with
equipment and valuable advises.

REFERENCES
1. S. Thunberg, H. Sklebitz, B. Ekdahl, L. BätZ, A. Lundin, H. MOller, F. Fleischmann, G. Kreider and T. Weidner,
"OPDIMA: Large-area CCD-based X-ray image sensor for spot imaging and biopsy control in mammography",
Proceedings ofSPIE, Physics ofmedical imaging, J. M. Boone, J. T. Dobbins III, Volume 3659, pp. 150-158, SPIE,
Bellingham, 1999.

2. European Commission, European Guidelines on Quality Criteriafor Diagnostic Radiographic Images, EUR 16260
EN, Brussels, 1996.

3. International Commission on Radiation Units and Measurement, ICRU report 48: Phantoms and computational models
in therapy, diagnosis, andprotection, ICRU, Bethesda, 1992.

4. B. Lanhede, L. G. Mânsson, S. Kheddache, M. Widell, L. Bjömeld, P. Sund, A. Almén, A. Tingberg, J. Besjakov, S.

Mattsson, M. Zankl, W. Panzer and C. Herrmann, "The influence of different technique factors on image quality for
chest radiographs — application ofthe recent CEC image quality criteria" (to be published).

5. R. G. Miller Jr, Simultaneous statistical inference, 2nd ed. Springer-Verlag, New York, 1980

6. J. T. Dobbins III, "DQE(f) of four generations of computed radiography acquisition devices", Med Phys. 22, pp. 158 1-
1593, 1995.

7. H. Fujita, D. Y. Tsai, T. Itoh, K. Doi, J. Morishita, K. Ueda, and A. Ohtsuka, IEEE Transactions on Medical Imaging
11, pp. 34-39, 1992.

8. IPEM Report No.78, Catalogue of Diagnostic X-ray Spectra and Other Data, The Institute of Physics and Engineering
in Medicine, York, 1997 (CD-ROM)

444

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 05 Jan 2023

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use