The Fifth Color Imaging Conference:

The Fifth
Color
Color
Science,
Imaging
Systems,
TheConference:
Fifthand
Color
Applications
Color
Imaging
Science,
Conference:
Systems,
Color
and Scien
Applications Copyright 1997, IS&T

Today’s Image Capturing Needs: Going

beyond Color Management
Chris Tuijn, Wim Cliquet
Agfa-Gevaert N.V., GS/EPS/R&D
B-2640 Mortsel, Belgium

Abstract A generic approach to automatic image

correction
One of the main concerns in both desktop and pre-press
environments is reliable color reproduction. This problem First, we will introduce a formal model that defines a general
is addressed by the color management systems which are framework for automatic image correction. Both spatial
aiming at the production of so-called facsimile color. In image corrections (such as unsharp masking, de-screening,
order to use color management systems, one should know noise removal etc.) as well as color corrections (such as
very well what the color space of the digital representation tonal corrections, selective color corrections) or a
combination (such as the correction of colored patterns) can
of the source image is. If this knowledge is not available,
be considered. Basically, any automatic image correction
the CMS work-flows cannot be followed and more scheme can be split up in 4 steps. The first step is the most
intelligent adaptive color correction techniques are required. difficult one and deals with the general problem analysis.
Even if the source of the images and the scanning Following steps deal with image correction as such.
equipment is well-known, people often want to reproduce
their originals "better". In order to produce more appealing
images, so-called color editing is required. This kind of Step 1: problem analysis by studying a test s e t
editing includes range adjustments, tonal adjustments, of images
saturation enhancement, global and selective color First, the problem needs to be defined. Often, this is done
transformations etc. In order to increase productivity, these based on a number of images (In)n:1..K to be corrected and the
color corrections should be carried out automatically. manually corrected images (I'n)n:1..K . A classification of the
The main goal of automatic color correction techniques test set can be realized by studying both the original set and
the corrected set. This classification can be formalized by a
thus consists of bringing the original images (the source of
set of parameters (α n)n:1..L which can be calculated for each
which might not be known) into a well-known calibrated source image. The parameter set can be used to derive a set
RGB space such that the reproduction of the images is of M corrections (Γn)n:1..M such that for each image In in the
appealing to the viewer. In order to achieve these goals, the test set :
images have to be analyzed and reference points have to be
detected. M

This paper is organized as follows. In the first section, Ο Γi ( In ) ≅ In′

i =1
we will introduce a general purpose model for automatic
image correction. The general techniques exposed in that The idea, of course, is that the composition of the
section will be illustrated by several case studies in the transformations should produce good results on arbitrary
following sections. In the first case study, we will introduce images as well. The problem is to find a number of
an automatic tonal correction which has been used in the characterizing parameters which contain enough information
newspaper business for black and white images. The second to classify the images and to generate good transformations.
case study will briefly describe adaptive techniques which There is no general method to describe how to define those
have been used in order to convert negatives to a well- parameters. There is a danger in both using too many
calibrated positive RGB space. The complexity of this parameters as well as in using not enough of them. The
technique is relatively low since it only involves a global extreme cases are either using the empty set of parameters or
color correction through the indication of a neutral point taking all input pixels into consideration. Clearly none of
(which is equivalent to the specification of a global cast). In these approaches makes much sense. If only color (non-
the third case study, we will deal with the general problem spatial) corrections are to be applied, using a down-sampled
of automatic image correction of color images from version of an input image might turn out to be useful.
unknown sources. In the last section, we will summarize the Other characterizing parameters are f.i.:
obtained results and indicate topics for future research.

203
The Fifth Color Imaging Conference:
The Fifth
Color
Color
Science,
Imaging
Systems,
TheConference:
Fifthand
Color
Applications
Color
Imaging
Science,
Conference:
Systems,
Color
and Scien
Applications Copyright 1997, IS&T

• one-dimensional histograms We now present a few case studies that illustrate our
• multi-dimensional histograms approach.
• frequency analysis (using either classical FFT
techniques, windowed FFT techniques or multi- Case study 1: tonal correction of
resolution analysis) black/white images for the newspaper
• filtered versions of a complete image or smaller regions business
etc.
In contrast with the traditional offset presses, the
Also parameters that are special to the problems to be newspaper printing has specific constraints and, therefore,
addressed can be used. We hereby think, e.g., of parameters special gamut mapping techniques are required in order to
such as the hue of the lightest and darkest point, average produce good reproductions. In particular, the total ink limit
colors of regions of interest showing skin, snow, grass, sky constraints are very important and should be used as a
etc. driving force behind the gamut mapping. Although the
The type of the corrections to be applied in order to solutions to this problem are far from trivial, they already
generate the output images can be determined either have been studied in the past.
automatically from the corrected test set or by discussing Often, skilled scanner operators apply subtle tonal
the applied corrections with the skilled scanner operator. In changes on top of the color management transforms
the past, we have been doing experiments to determine the described above. These tonal changes appear to depend
parameters of well-defined parameterized transformations heavily on the type of the original. In order to get a better
automatically using neural networks. The learning set feeling of what kind of corrections usually are applied, we
consists of K elements (one for each image) mapping the asked an experienced scanner operator to collect a number of
analysis parameters (∝n)n:1..L to the parameterized representative grey-scale images and to correct those images.
representation of the transforms (Γn)n:1..M . This experiment resulted in a database of images and their
corrected versions. It turned out that the corrections could be
In 
→ I 'n reduced to a global remapping of the intensity values. Most
often, a small contrast enhancement or reduction was applied
⇓ together with some corrections in the highlights or shadows.
α1 ( I n ) ... α L( I n ) → Γ1 ( In ) ... ΓM ( I n ) More sophisticated spatial filters to create local contrast
changes (such as dodging) were not considered.
Although reasonable results were obtained at that time, The test set that we got back from our skilled scanner
often better results can be obtained using classical methods operator consists of 30 images some of which:
from numerical analysis based on linear and non-linear
regression techniques [4,5]. • are tonally good;
The following steps (2,3 and 4) describe how a • are overexposed;
particular image will be transformed. • are underexposed;
• have flat shadow parts;
Step 2: detection of well-known parameters • have a high contrast in the shadows;
Now, we need to calculate the characterizing set of • have a low contrast in the highlights;
parameters as described above. In this step, we basically • have a too high contrast in the highlights;
determine which class(es) the given image belongs to. • have flat mid-tones;
Often, heuristic techniques (based on fuzzy logic) are • have too much contrast in the mid-tones;
applied; rule-based systems or Prolog-based search engines • etc.
can be used in this context as well.
All of the images in the test set have one or more of the
Step 3: generating color transforms based o n
these parameters characterics listed above. Most of the items in the list can
This step calculates the image transforms based on the be derived by looking at the histogram or cumulative
characterizing set of parameters as returned in Step 2. Now, distribution function (CDF). By plotting all CDF’s in a
the image correction can be applied to the source image. diagram, we get a fairly good idea of what types of originals
we can expect. Most of the CDF’s are positioned in a kind
Step 4: learning mode of hysteresis shape.
The user is able to request small adjustments to the In figure 1, we show a few CDF curves and how they
(automatically) proposed image transforms. The modified can be interpreted. The horizontal axis represents the dot
parameters are then fed back to the algorithm and used later percentages; the vertical axis represents the accumulated
on. We advise an additional option to enable/disable the frequency percentages. The curves can be interpreted as
learning mode.
follows :

204
The Fifth Color Imaging Conference:
The Fifth
Color
Color
Science,
Imaging
Systems,
TheConference:
Fifthand
Color
Applications
Color
Imaging
Science,
Conference:
Systems,
Color
and Scien
Applications Copyright 1997, IS&T

100
• curve 1 is the CDF of an image which will be 90
perceived as overall too dark; 80
• curve 2 is the CDF of an image with no mid-tones; 70
Fm CO
• curve 3 is the CDF of an image with a uniform 60 Fm CH
histogram; 50 Fm CN

• curve 4 is the CDF of an image which has enhanced 40 Fm CL

30 Fm CU
mid-tones at the cost of little highlight and shadow areas;
20
and,
10
• curve 5 is the CDF of an image which will be 0
perceived as overall too light. 0 20 40 60 80 100
Dotpercentage
100
90
Figure 2: Corrrection for the different families
80
70 We now are ready to start with the actual correction of
Curve 1
60
Curve 2
any given image. First of all, we determine the family to
50 Curve 3 which the image belongs. This is done by means of a
40 Curve 4 distance function ∆. Good results can be obtained by using a
30 Curve 5
sum of squares of differences in a limited number of points.
20
10
If F denotes the set of families f, CDF(f) the cumulative
0 distribution function of f, Fm(f) the proposed correction as
0 20 40 60 80 100 in Figure 2, and ∆ a distance function in a function space as
D otpercentage
described previously, then the tonal correction TF to be
applied on an image g can be calculated as follows:
Figure 1: CDF sample curves

∑ ∆(CDF(f), CDF(g))
Fm ( f )
A straightforward method to improve the CDF would be to
apply a so-called histogram equalization (as pointed out in f∈F
TF(g) =
∑ ∆(CDF(f), CDF( g ))
[1]). It turns out that, although the tonally compressed areas 1
come out much better after histogram equalization, the result
f∈F
often looks artificial and is unacceptable. Therefore, we
propose to use the test set created by the skilled operator. Additional spatial corrections can be carried out to sharpen
After careful investigation of the test set, the test images can the images; the parameters for these USM filters can also be
be divided in 5 different families: determined automatically but are not covered in this article.

• Family CO, containing 5 images : underexposed images Case study 2: adaptive color corrections
without contrast problems; for negative to positive conversion
• Family CH, containing 10 images : overexposed
images without contrast problems; The problem of scanning negative film and converting the
• Family CN, containing 5 images : images with normal negative signal into a well-known positive RGB space has
been addressed extensively in the past [8,10,11,12]. The
exposure; main problem consists of calculating appropriate inversion
• Family CL , containing 9 images : images with tables in order to convert from negative to positive; these
contrast problems (usually located near the mid-tones); inversion curves are based on the characteristic film curves
and, of the negative film as perceived by the scanner as shown in
• Family CU, containing 1 image : images with Figure 3 (see also [9]). It turns out, however, that the
characteristic film curves do not only depend on the type of
extremely high contrast and lacking mid-tones.
film, but can vary from batch to batch and are also heavily
influenced by the development of the negative film. On top
The families were derived by analyzing the problem and of these problems, the circumstances under which the picture
looking at the proposed correction. After averaging the was taken also influence the final result (shutter speed,
curves proposed by the expert user, following corrections opening, type of camera, light conditions etc.).
could be derived for each family:

205
The Fifth Color Imaging Conference:
The Fifth
Color
Color
Science,
Imaging
Systems,
TheConference:
Fifthand
Color
Applications
Imaging
Color Science,
Conference:
Systems,
Color
andScien
Applications Copyright 1997, IS&T

2 .5
result in the so-called TFS curves3 which define the adapted
characteristic film curves for this specific film.
2 Now the frame to be converted is analyzed further in
order to get rid of local color casts due to, e.g., the
1 .5 Re d
illumination or other environmental parameters (such as
Gre e n flash-light). The final correction is obtained as a weighted
Bl u e average of the general mean densities, the densities of the
1

near-neutrals and the TFS curve.

The last parameter to be determined is related to the
0 .5
dynamic range and indicates a mid-tone neutral. Parameters
which are taken into account to determine this mid-point are:
0
0 0 .2 0 .4 0 .6 0 .8 1 1 .2 1 .4 1 .6 1 .8 2 2 .2 2 .4 2 .6 2 .8 3 3 .2 3 .4 3 .6 3 .8 4
• the minimum density of the frame;
• the maximum densities of concentric zones to find out
Figure 3: characteristic curves of Agfa HDC 100 film as seen b y
Agfa's DuoScan
where the main object of the frame resides;
• the densities of particular areas of interest showing skin,
The above arguments suggest that it is almost white areas (skiing), etc.;
impossible to take all these parameters into account. We • corrections based on the classification of scenes
can, however, get a reasonable idea of the characteristic (portrait, landscape, ...)
curves of a particular negative film by considering a test • the orientation of the frame;
wedge (made on the same type of film, captured by the same • ...
scanner) and use these curves as a basis to determine future
corrections. In figure 3, the characteristic film curves of an The algorithm above thus results in three densities
Agfa HDC 100 film1 are shown as measured by the Agfa which are used to calculate the inversion curves. Basically,
DuoScan2. The X-axis contains the exposing intensities on these densities will translate the characteristic curves
a logarithmic scale; the Y-axis contains the densities of the horizontally such that said densities (on Y axis) originate
developed negative wedge in DuoScan scanner space. In order from a triplet with equal values, i.e., neutral light (on the X
to get rid of measuring errors, we use a parameterized curve axis). The dynamic range of the selected frame (exposure
(as described in [6]); the parameters of these curves are latitude) is also taken into account when calculating the
determined in a least square sense [4,5]. inversion curves. For further detail, we refer to [8].
The adaptive corrections to these curves need to be The user has the option to specify small changes to the
obtained by analyzing the actual negative film strip and the proposed parameters. In Agfa’s FotoLook 3.04 scanning
specific picture on this strip which we want to scan and software, this can be done by picking a color cast in a
convert. The corrections we have in mind here are defined as hue/saturation color-wheel. The proposed changes can be
corrections on the characteristic film curves and are thus stored in a database and be used later on to influence the
inherently global. Local or so-called selective color system's behavior.
corrections will not be considered but better results can be Summarizing, we conclude that we basically follow the
obtained by applying additional multi-dimensional color same work-flow as described in the first section:
corrections on the inverted image to produce more reliable • Step 1: general problem analysis;
color (see [11,12]). In the traditional photo-finishing labs, • Step 2: analysis of the original (the film strip and one
however, the only parameters that can be influenced to specific frame on this strip), resulting in a histogram of
generate the print on photo paper are the exposure near neutrals and many other parameters resulting in
parameters of the light-source in red, green and blue (which three characterizing parameters;
are, of course, global corrections). • Step 3: calculation of the color correction based on the
These global correction parameters can be specified as three parameters; and,
red, green and blue densities in the scanner input space. The • Step 4: small corrections to characterizing parameters
meaning of this density triplet is that a density point of the suggested by user and learning facility.
negative film should be mapped to a neutral point of a given
intensity in the inverted (positive) space. As such, the Case study 3: automatic correction of
density triplet influences both tonal and color balance. color images from an unknown source
The color balance correction is based on a statistical
analysis of our film strip and involves the calculation of a The general problem of automatic image correction of
histogram of the near-neutrals. This information is images from unknown sources is very complex. The
combined with statistical information on the average complexity is not only caused by the inherent, technical
characteristic film curves of a representative set of various problems related to the recognition of objects, patterns etc.
films (of different vendors). All these data are weighted and but also stems from the fact that the proposed corrections

206
The Fifth Color Imaging Conference:
The Fifth
Color
Color
Science,
Imaging
Systems,
TheConference:
Fifthand
Color
Applications
Color
Imaging
Science,
Conference:
Systems,
Color
and Scien
Applications Copyright 1997, IS&T

will always be very subjective. In this section, we will not This type of correction requires a study of the spatial
cover our solution to automatic image correction in detail characteristics of the image by means of Fourier or wavelet
but we will rather outline some ideas we have been pursuing analysis [3].
and which seem of fundamental importance. In particular,
we will summarize the corrections we have been using for To conclude, I would like to emphasize that the
image improvement in general and some of the indicators obtained results must be interpreted in a standard color space
that allowed the determination of the parameters for the that has been determined up front. The classifying
transformations. parameters as well as the proposed corrections depend on the
choice of this exchange space. Candidates for such a space
Global color corrections: are monitor spaces having a specific gamma value. The
Global corrections can be defined as color corrections chromaticities of of this space should be wide enough to
which are applied to one or more color components span a reasonable gamut. Preferably, it should be specified
independently of each other. They can be implemented as an ICC profile (cf. [2]). Once the images are in this
using one-dimensional tonal LUT’s (look-up tables). The exchange space, standard CMS techniques can be used to
most important global color corrections are: transform the images from this space to any other space.
• dynamic range adjustments: often, the digital
image is not using its full range of values. This Conclusions
suggests that the dynamic range should be stretched to
the allowed values. This should be carried out carefully, In this document, we described a global approach to
however, based on a classification the original. automatic image correction. Although substantial results
Otherwise, very saturated colors or pastel tints might be have been reached so far, it is obvious that the general
ruined. problem of correcting colored images automatically will
• tonal corrections: a more general tonal correction is never have a completely satisfying solution and therefore
needed if the original appears to be overall too light or will need on-going attention and improvements. The main
too dark. One way to determine this is to study the technique which has been introduced here is essentially based
CDF as discussed in Case 1. This method can be on a statistical analysis of a test set of images using a
number of parameters which describe important features
improved by also taking into consideration the spatial
allowing to make some kind of classification. After the
activity in certain areas, the main idea being that a analysis of the test set, a relationship is established between
spatially active area often needs tonal enhancement. the classifying parameters and the corrections proposed by an
• color cast removal: for color cast removal, experienced user. In operational mode, the algorithm will
analogous techniques to Case 2 can be used. calculate the image specific parameters and come up with a
correction as proposed by the expert. In addition, the end-
Local color corrections: user will be able to apply minor changes to the proposed
Sometimes, particular areas in the color space require further corrections; the system will learn about the subjective
enhancement. In order to determine such color corrections changes proposed by a particular user and will correct its
(which are also known as selective color corrections), we future behavior accordingly.
need to detect reference objects in the original. This The inference engine which establishes a relationship
detection will be based on both spatial and color content. between the input parameters and image correction is
Areas which are considered to be very important are, e.g., essentially a rule-based system. It might be worthwhile to
areas containing skin, sky, meadows etc. study the usability of other formalisms to establish this
The correction then consists of mapping the color of the relationship such as, .e.g, neural networks or Prolog-based
detected area to a configurable color using a selective color engines.
transformation. The learning facilities (cf. Step 4 above) The quality of the obtained results is heavily dependent
consist of both influencing the detection of the special on the classifying parameters. Much of the future work will
categories as well as the determination of the target colors. be concentrated on trying to come up with new classifying
parameters and more sophisticated corrections. For spatial
Spatial corrections:
On top of color corrections, spatial corrections might be corrections, we can make use of wavelets and the associated
needed for further enhancement. We hereby think of: multi-resolution analysis; for color corrections, further
analysis of 3-D histograms can give better insight in what
• sharpening; type of original scene we are dealing with. An even more
• noise removal and removal of other artifacts (such as advanced solution might be based on a 5-dimensional
artifacts from lossy compression schemes); spatial/color analysis; this is, e.g., necessary to deal with
• scratch removal; colored patterns of cloth.
• dodging;
• ...

207
The Fifth Color Imaging Conference:
The Fifth
Color
Color
Science,
Imaging
Systems,
TheConference:
Fifthand
Color
Applications
Color
Imaging
Science,
Conference:
Systems,
Color
and Scien
Applications Copyright 1997, IS&T

10. Tuijn C., "The Influence of Precompensation Curves

References on Multidimensional Color Modeling", Proceedings of
IS&T/SPIE's Symposium on Electronic Imaging: Science
1 . Gonzalez R.C. and Wintz P., “Digital Image and Technology, Color Imaging : Device-Independent
Processing”, 2nd Edition, Addison -Wesley Publishing Color, Color Hard Copy, and Graphic Arts, Volume
Company, Reading MA, 1987 2658, San Jose CA, 1996, pp. 326-333

2 . International Color Consortium, ICC Profile Format 11. Tuijn C., "An Adaptive Approach to Negative
Specification, Version 3.3, http://www.color.org, Scanning", Proceedings of SPIE/Europto’s Conference
November 1996 on Imaging Sciences and Display Technologies, Berlin
FRG, 1996, pp. 281-290
3 . Kaiser G., “A Friendly Guide to Wavelets”, Birkhäuser,
Boston, 1994 12. Tuijn C., "Scanning Color Negatives", Proceedings of
IS&T/SID Fourth Color Imaging Conference: Color
4 . Lawson C.L. and Hanson R., "Solving Least Squares Science, Systems and Applications, Scottsdale AZ,
Problems", Prentice Hall, Englewood Cliffs NJ, 1974 1996, pp. 33-38

5. Marquardt D.W., Journal of the Society for Industrial 1 Agfa HDC 100 is a registered trademark of Agfa-
and Applied Mathematics, vol. 11, pp. 431-441, 1963 Gevaert N.V.
2 Agfa DuoScan is a registered trademark of Agfa-
6. Pytela O., Majer J., Journal of Imaging Science, vol.
35.6, pp. 362-364, "Quantitative Interpretation of Gevaert N.V.
Sensitometric Curves of Photographic Materials", 1991 3 TFS stands for Total Film Scanning and refers to
the fact that, in order to determine the correction
7. Stroebel L., Compton J., Current I. and Zakia R., parameters for a particular frame of the negative
"Photographic Materials and Processes", Focal Press, film roll, the entire strip (i.e., the total film) is
Boston, 1986 scanned first. TFS is a registered trademark of
Agfa-Gevaert N.V.
8. Tuijn C., "Input Calibration for Negative Originals",
4 Agfa FotoLook 3.0 is a registered trademark of
Proceedings of IS&T/SPIE's Symposium on Electronic
Imaging: Science and Technology, Device-Independent Agfa-Gevaert N.V.
Color Imaging II, Volume 2414, San Jose CA, 1995, pp.
76-83

9. Tuijn C., "Hardware and Firmware Requirements for the

Ideal Negative Scanner", Agfa-Gevaert N.V., Internal
Memo, Mortsel, 1996

208