TUTORIAL

Color conversion matrices in digital cameras: a tutorial

D. Andrew Rowlands*
Independent Consultant, Cambridge, United Kingdom

Abstract. Color conversion matrices and chromatic adaptation transforms (CATs) are of central
importance when converting a scene captured by a digital camera in the camera raw space into a
color image suitable for display using an output-referred color space. In this article, the nature of
a typical camera raw space is investigated, including its gamut and reference white. Various color
conversion strategies that are used in practice are subsequently derived and examined. The strat-
egy used by internal image-processing engines of traditional digital cameras is shown to be based
upon color rotation matrices accompanied by raw channel multipliers, in contrast to the approach
used by smartphones and commercial raw converters, which is typically based upon characteri-
zation matrices accompanied by conventional CATs. Several advantages of the approach used by
traditional digital cameras are discussed. The connections with the color conversion methods of
the DCRaw open-source raw converter and the Adobe digital negative converter are also exam-
ined, along with the nature of the Adobe color and forward matrices. © 2020 Society of Photo-
Optical Instrumentation Engineers (SPIE) [DOI: 10.1117/1.OE.59.11.110801]

Keywords: color matrix; white balance; chromatic adaptation; gamut; raw conversion; Adobe
digital negative.
Paper 20201001T received Aug. 24, 2020; accepted for publication Oct. 27, 2020; published
online Nov. 17, 2020.

1 Introduction
Consider converting a scene captured by a digital camera in the camera raw space into a digital
image suitable for display using an output-referred color space. At the very least, there are two
issues of fundamental importance that must be addressed when attempting to correctly reproduce
the appearance of color. The first is that the response functions of digital cameras differ from
those of the human visual system (HVS). A widely used approach to this issue is to consider
color spaces as vector spaces and to account for the differences in response by introducing a
color conversion matrix. A type of color conversion matrix that is commonly encountered is the
3 × 3 characterization matrix T that defines the linear relationship between the camera raw space
and the CIE XYZ reference color space:
2 3 2 3
X R
EQ-TARGET;temp:intralink-;e001;116;287

6 7 6 7
4 Y 5 ≈ T 4 G 5: (1)
Z B

In general, camera raw spaces are not colorimetric, so the above conversion is approximate. The
relationship can be optimized for a given illuminant by minimizing the color error. Significantly,
this means that the optimum T depends upon the nature of the scene illuminant,1,2 including its
white point (WP). The characterization methodology for determining the optimum T is described
in Sec. 2.4, along with an illustration of how T should be normalized in practice.
The second issue that must be addressed is the perception of the scene illumination WP.
Although the various adaptation mechanisms employed by the HVS are complex and not fully
understood, it is thought that the HVS naturally uses a chromatic adaptation mechanism to adjust
its perception of the scene illumination WP to achieve color constancy under varying lighting
conditions.3,4 Since digital camera sensors do not naturally adapt in this manner, incorrect white

*Address all correspondence to D. Andrew Rowlands, [email protected]

Optical Engineering 110801-1 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

balance (WB) will arise when the WP of the scene illumination differs from the reference white
of the output-referred color space used to encode the output image produced by the camera.
As demonstrated in Sec. 3, digital cameras must attempt to emulate this chromatic adaptation
mechanism by utilizing an appropriate chromatic adaptation transform (CAT).
As discussed in Sec. 4, modern smartphones and commercial raw converters typically cal-
culate the optimum characterization matrix T by interpolating between two preset characterization
matrices according to an estimate of the scene illumination WP, and the CAT is implemented after
applying T. In traditional digital cameras, the color conversion is typically reformulated in terms
of raw channel multipliers and color rotation matrices R. This approach offers several advantages,
as discussed in Sec. 5. A similar but computationally simpler approach is used by the DCRaw
open-source raw converter, as discussed in Sec. 6. The open-source Adobe® digital negative
(DNG) converter offers two color conversion methods, and the nature of the Adobe color matrices
and forward matrices are discussed in Sec. 7. Finally, conclusions are drawn in Sec. 8.

2 Camera Raw Space

2.1 Gamut
The camera raw space for a given camera model arises from its set of spectral responsivity func-
tions or camera response functions:
eλ
Ri ðλÞ ¼ QEi ðλÞ ; (2)
hc
EQ-TARGET;temp:intralink-;e002;116;489

where e is the elemental charge, λ is the wavelength, h is Planck’s constant, and c is the speed of
light. The external quantum efficiency for mosaic i is defined by

QEi ðλÞ ¼ T CFA;i ðλÞηðλÞTðλÞFF;

EQ-TARGET;temp:intralink-;e003;116;423 (3)

where T CFA;i is the color filter array (CFA) transmittance function for mosaic i, ηðλÞ is the charge
collection efficiency or internal quantum efficiency of a photoelement, and TðλÞ is the SiO2 ∕Si
interface transmittance function.5 The fill factor is defined by FF ¼ Adet ∕Ap, where Adet is the
photosensitive detection area at a photosite and Ap is the photosite area. The spectral passband of
the camera should ideally correspond to the visible spectrum, so an infra-red blocking filter is
required.
Analogous to the eye-cone response functions of the HVS, which can be interpreted as speci-
fying the amounts of the eye cone primaries that the eye uses to sense color at a given λ, the
camera response functions can be interpreted as specifying amounts of the camera raw space
primaries at each λ. For example, the measured Nikon D700 camera response functions are
shown in Fig. 1. However, a camera raw space is colorimetric only if the Luther-Ives condition
is satisfied,7–9 meaning that the camera response functions must be an exact linear transformation
of the eye-cone response functions, which are indirectly represented as a linear transformation
from the CIE color-matching functions for the standard observer.
Although the eye-cone response functions are suited to capturing detail using the simple lens
of the human eye, digital cameras use compound lenses that have been corrected for chromatic
aberration. Consequently, camera response functions are designed with other considerations in
mind.10,11 For example, better signal-to-noise performance is achieved by reducing the overlap of
the response functions, which corresponds to a characterization matrix with smaller off-diagonal
elements.10–12 Indeed, minor color errors can be traded for better signal-to-noise performance.10–13
On the other hand, increased correlation in the wavelength dimension can improve the perfor-
mance of the color demosaicing procedure.14 Due to such trade-offs along with filter manufac-
turing constraints, camera response functions are not exact linear transformations of the eye-cone
response functions in practice. Consequently, camera raw spaces are not colorimetric, so cameras
exhibit metameric error. Metamers are different spectral power distributions (SPDs) that are per-
ceived by the HVS to be the same color when viewed under exactly the same conditions.
Cameras that exhibit metameric error produce different color responses to these metamers.

Optical Engineering 110801-2 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

Fig. 1 Camera response functions for the Nikon D700 camera. The peak spectral responsivity
has been normalized to unity. Data sourced from Ref. 6.

Camera metameric error can be determined experimentally and quantified by the digital still
camera sensitivity metamerism index (DSC/SMI).8,15
Figure 2 shows the spectral locus of the HVS on the xy chromaticity diagram, which is a 2D
projection of the CIE XYZ color space that describes the relative proportions of the tristimulus
values. Note that the spectral locus itself is horseshoe-shaped rather than triangular due to the fact

Fig. 2 Gamut of the camera raw space for the Nikon D700 (light-blue shaded area) plotted on the
x y chromaticity diagram. The gamut is not a perfect triangle since the Luther-Ives condition is
violated, which also explains why certain regions are pushed outside the triangle accessible to
the CIE XYZ color space defined by the primaries located at (0,0), (0,1), and (1,0). The boundary
of the horseshoe-shaped gray shaded area defines the spectral locus of the HVS. Saturation
decreases with inward distance from the spectral locus. For comparison purposes, the (triangular)
gamuts of several standard output-referred color spaces are indicated.

Optical Engineering 110801-3 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

that overlap of the eye-cone response functions prevents the eye cones from being independently
stimulated, so the chromaticities corresponding to ðx; yÞ chromaticity coordinates positioned out-
side the spectral locus are invisible or imaginary as they are more saturated than pure spectrum
colors. The gamut of the camera raw space for the Nikon D700 camera is also plotted on Fig. 2
and compared with several standard output-referred color spaces, namely sRGB,16 Adobe®
RGB,17 and ProPhoto RGB.18 Due to the positions of the camera raw space primaries on the
xy chromaticity diagram, certain regions of the camera raw space gamut do not reach the spectral
locus of the HVS as these regions lie outside the triangular shape accessible to additive linear
combinations of the three primaries. Furthermore, a notable consequence of camera metameric
error is that the camera raw space gamut is warped away from the triangular shape accessible to
the additive linear combinations of the three primaries. Certain regions are even pushed outside of
the triangle accessible to the CIE XYZ color space.19 See Ref. 19 for additional examples.
To determine the gamut of a camera raw space, the first step is to measure the camera
response functions using a monochromator at a discrete set of wavelengths according to method
A of the ISO 17321-1 standard.15 For each wavelength, the camera response functions yield
raw RGB relative tristimulus values in the camera raw space. The second step is to convert
RGB into relative CIE XYZ values by applying a characterization matrix that satisfies
Eq. (1). Subsequently, the ðx; yÞ chromaticity coordinates corresponding to the spectral locus
of the camera raw space can be calculated using the usual formulas, x ¼ X∕ðX þ Y þ ZÞ and
y ¼ Y∕ðX þ Y þ ZÞ.
Since a given characterization matrix is optimized for use with the characterization illuminant,
i.e., the scene illumination used to perform the characterization, another consequence of camera
metameric error is that the camera raw space gamut may vary according to the characterization
matrix applied. The gamut of the Nikon D700 camera raw space shown in Fig. 2 was obtained
using a characterization matrix optimized for CIE illuminant D65. Figure 3 shows how the gamut
changes when a characterization matrix optimized for CIE illuminant A is applied instead.

Fig. 3 Same as Fig. 2 except that a characterization matrix optimized for CIE illuminant A was
used to obtain the camera raw space gamut rather than a characterization matrix optimized for
CIE illuminant D65.

Optical Engineering 110801-4 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

2.2 Raw Values

Color values in a camera raw space are expressed in terms of digital raw values for each raw color
channel, which are analogous to tristimulus values in CIE color spaces. For a CFA that uses three
types of color filters such as a Bayer CFA,20 the raw values expressed using output-referred units,
i.e., data/digital numbers (DN) or analog-to-digital units, belong to the following set of raw
channels denoted here using calligraphic symbols:
2 3 2 3
nDN;1 R
6n 7 6 7
6 DN;2 7 6 G1 7
6 7 ¼ 6 7: (4)
4 nDN;3 5 4 G2 5
EQ-TARGET;temp:intralink-;e004;116;656

nDN;4 B

Although vector notation has been used here to represent a Bayer block, a true raw pixel vector is
obtained only after the color demosaic has been performed, in which case there will be four raw
values associated with each photosite. The Bayer CFA uses twice as many green filters as red and
blue, which means that two values G1 and G2 associated with different positions in each Bayer
block will be obtained in general. This is beneficial in terms of overall signal-to-noise ratio since
photosites belonging to the green mosaics are more efficient in terms of photoconversion.
Furthermore, the Bayer pattern is optimal in terms of reducing aliasing artifacts when three types
of filters are arranged on a square grid.14 Although it is thought that a greater number of green
filters provides enhanced resolution for the luminance signal since the standard 1924 CIE lumi-
nosity function for photopic vision peaks at 555 nm,20 it has been argued that a Bayer CFA with
two times more blue pixels than red and green would in fact be optimal for this purpose.14 When
demosaicing raw data corresponding to a standard Bayer CFA, the final output will show false
mazes or meshes if the ratio between G1 and G2 varies over the image.21 Software raw converters
may average G1 and G2 together to eliminate such artifacts.21
Since there are fundamentally only three camera response functions, R1 ðλÞ, R2 ðλÞ, and R3 ðλÞ,
color characterization for a Bayer CFA regards G1 and G2 as a single channel, G. The raw values
can be expressed as follows:
Z λ Z λ Z λ
2 2 2
R¼k R1 ðλÞẼe;λ dλ; G ¼ k
EQ-TARGET;temp:intralink-;e005;116;376 R2 ðλÞẼe;λ dλ; B ¼ k R3 ðλÞẼe;λ dλ: (5)
λ1 λ1 λ1

The camera response functions are defined by Eq. (2), the integration is over the spectral pass-
band of the camera, E~ e;λ is the average spectral irradiance at the photosite, and k is a constant.
Expressions for E~ e;λ and k are given in the Appendix.
The actual raw values obtained in practice are quantized values modeled by taking the integer
part of Eq. (5). When transforming from the camera raw space, it is useful to normalize the raw
values to the range [0,1] by dividing Eq. (5) by the raw clipping point, which is the highest
available DN.

2.3 Reference White

Using the above normalization, the reference white of a camera raw space is defined by the unit
vector
2 3 2 3
R 1
6 7 6 7
4 G 5 ¼ 4 1 5: (6) EQ-TARGET;temp:intralink-;e006;116;182

B 1

Expressed in terms of CIE XYZ tristimulus values or ðx; yÞ chromaticity coordinates with Y ¼ 1,
the reference white of a camera raw space is the WP of the scene illumination that yields maxi-
mum equal raw values for a neutral subject. (The WP of a SPD is defined by the CIE XYZ
tristimulus values that correspond to a 100% neutral diffuse reflector illuminated by that SPD.)

Optical Engineering 110801-5 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

It follows that the reference white of a camera raw space can in principle be determined
experimentally by finding the illuminant that yields equal raw values for a neutral subject.
Note that if the DCRaw open-source raw converter is used to decode the raw file, it is essential
to disable WB. In terms of CIE colorimetry, the camera raw space reference white is formally
defined by
2 3 2 3
XðWPÞ RðWPÞ ¼ 1
6 7 6 7
4 YðWPÞ 5 ¼ T 4 GðWPÞ ¼ 1 5 ; (7)
EQ-TARGET;temp:intralink-;e007;116;675

ZðWPÞ scene BðWPÞ ¼ 1 scene

where YðWPÞ ¼ 1 and the subscripts denote that the WP is that of the scene illumination. The
3 × 3 characterization matrix T converts from the camera raw space to CIE XYZ and should be
optimized for the required scene illumination. The optimum T is unknown at this stage but can in
principle be determined using the optimization procedure to be outlined in Sec. 2.4.
Although CIE color spaces use normalized units such that their reference whites correspond
to WPs of CIE standard illuminants, camera raw spaces are not naturally normalized in such a
manner. Consequently, the reference white of a camera raw space is not necessarily a neutral
color as it is typically located far away from the Planckian locus and so does not necessarily have
an associated correlated color temperature (CCT).
Note that a WP can be associated with a CCT provided its ðx; yÞ chromaticity coordinates are
sufficiently close to the Planckian locus, but there are many such coordinates that correspond to
the same CCT. To distinguish between them, a Duv value, informally referred to as a color tint,
can be assigned.22 This is determined by converting ðx; yÞ into ðu; vÞ chromaticity coordinates on
the CIE 1960 UCS chromaticity diagram,23,24 where isotherms are normal to the Planckian locus.
In this representation, CCT is a valid concept only for ðu; vÞ coordinates positioned a distance
from the Planckian locus that is within Duv ¼ 0.05 along an isotherm.25
To see that the reference white of a camera raw space is far from the Planckian locus, consider
the Nikon D700 raw values for a neutral diffuse reflector illuminated by CIE illuminants A and
D65, respectively,
2 3 2 3
RðWPÞ ¼ 0.8878 XðWPÞ ¼ 1.0985
EQ-TARGET;temp:intralink-;e008;116;385

6 7 6 7
4 GðWPÞ ¼ 1.0000 5 ¼ T −1 A 4 YðWPÞ ¼ 1.0000 5
BðWPÞ ¼ 0.4017 A ZðWPÞ ¼ 0.3558 A
2 3 2 3
RðWPÞ ¼ 0.4514 XðWPÞ ¼ 0.9504
6 7 6 7
4 GðWPÞ ¼ 1.0000 5 ¼ T −1
D65 4 YðWPÞ ¼ 1.0000 5 ; (8)
BðWPÞ ¼ 0.8381 D65 ZðWPÞ ¼ 1.0888 D65

where T A and T D65 are example characterization matrices optimized for CIE illuminants A and
D65, respectively. As shown in Fig. 4, the WPs of these standard illuminants are very close to the
Planckian locus. Illuminant A has CCT ¼ 2856 K and Duv ¼ 0.0, and illuminant D65 has
CCT ¼ 6504 K and Duv ¼ 0.0032. Evidently, the above Nikon D700 raw values are very differ-
ent from the camera raw space unit vector, and it would be necessary to apply large multipliers
to the red and blue raw pixel values in both cases. These multipliers are known as raw channel
multipliers as they are typically applied to the red and blue raw channels before the color demo-
saic as part of the color conversion strategy used by the internal image-processing engines of
traditional digital cameras.
An estimate of the Nikon D700 reference white can be obtained by approximating Eq. (7)
using a readily available characterization matrix in place of T. Applying T A yields ðx; yÞ ¼
ð0.3849; 0.3058Þ, which corresponds to Duv ¼ −0.0378. This has an associated CCT ¼ 3155 K
as the Duv value is just within the allowed limit, but Fig. 4 shows that the color tint is a strong
magenta. This is true of typical camera raw spaces in general.21 A similar estimate for the
Olympus E-M1 camera yields ðx; yÞ ¼ ð0.3599; 0.2551Þ, which corresponds to Duv ¼ −0.0637.
This does not have an associated CCT, and the color tint is a very strong magenta.
Although the fact that camera raw space reference whites are not neutral in terms of CIE
colorimetry has no bearing on the final reproduced image, it will be shown in Sec. 5 that the

Optical Engineering 110801-6 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

Fig. 4 Estimated reference whites of the Nikon D700 and Olympus E-M1 camera raw spaces in
relation to the WPs of CIE illuminants A and D65. The Planckian locus is represented by the black
curve. Only visible chromaticities contained within the sRGB color space are shown in color.

camera raw space reference white is utilized as a useful intermediary step in the color conversion
strategy used by traditional digital cameras.

2.4 Camera Color Characterization

Recall the linear transformation from the camera raw space to CIE XYZ defined by Eq. (1):
2 3 2 3
X R
6 7 6 7
4 Y 5 ≈ T 4 G 5; EQ-TARGET;temp:intralink-;sec2.4;116;353

Z B

where T is a 3 × 3 characterization matrix:

2 3
T 11 T 12 T 13
6 7
T ¼ 4 T 21EQ-TARGET;temp:intralink-;e009;116;279 T 22 T 23 5: (9)
T 31 T 32 T 33

The color conversion is approximate since the Luther-Ives condition is not satisfied exactly. As
mentioned in the introduction, T can be optimized for the characterization illuminant, i.e., the
scene illumination used to perform the characterization.1,2 The optimum matrix T is dependent
upon the SPD itself, but it largely depends upon the characterization illumination WP provided
the illuminant is representative of a real world SPD.
Characterization matrices optimized for known illuminants can be determined by color-error
minimization procedures based upon photographs taken of a standard color chart.2 Although
various minimization techniques have been developed, including WP-preserving techniques,26
the procedure discussed below is based on the standardized method B of ISO 17321-1.15
Note that ISO 17321-1 uses processed images output by the camera rather than raw data and
consequently requires inversion of the camera opto-electronic conversion function (OECF).27
The OECF defines the nonlinear relationship between irradiance at the sensor plane and the
digital output levels of a viewable output image such as a JPEG file produced by the camera.

Optical Engineering 110801-7 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

To bypass the need to experimentally determine the OECF, a variation of method B from ISO
17321-1 is described below. This method uses the DCRaw open-source raw converter to decode
the raw file so that the raw data can be used directly.28,29
1. Take a photograph of a color chart illuminated by a specified illuminant. Since the raw
values scale linearly, only their relative values are important. However, the f-number N
and exposure duration t should be chosen so as to avoid clipping.
2. Calculate relative XYZ tristimulus values for each patch of the color chart:
Z λ Z λ Z λ
2 2 2
X¼k xðλÞEe;λ RðλÞdλ Y ¼ k
EQ-TARGET;temp:intralink-;e010;116;641 yðλÞEe;λ RðλÞdλ Z ¼ k zðλÞEe;λ RðλÞdλ; (10)
λ1 λ1 λ1

where Ee;λ is the spectral irradiance incident at the color chart measured using a spec-
trometer; xðλÞ, yðλÞ, and zðλÞ are the color-matching functions of the CIE XYZ color
space; and the integration is discretized into a sum with a 10-nm increment and limits
λ1 ¼ 380 nm and λ2 ¼ 780 nm. Unless a tristimulus colorimeter is used, the calculation
requires knowledge of the spectral reflectance of each patch. Spectral reflectance has been
denoted by RðλÞ in the above equations, and this should not be confused with the camera
response functions. The normalization constant k can be chosen so that Y is in the range
[0,1] using the white patch as a white reference.
3. Obtain a linear demosaiced output image directly in the camera raw space without con-
verting to any other color space. Gamma encoding, tone curves, and WB must all be
disabled. Since the present method bypasses the need to determine and invert the OECF,
it is crucial to disable WB; otherwise, raw channel multipliers may be applied to the raw
channels. If using the DCRaw open-source raw converter, an appropriate command is
dcraw -v -r 1 1 1 1-o 0 -4 -T filename:
EQ-TARGET;temp:intralink-;sec2.4;116;441

This yields a 16-bit linear demosaiced output TIFF file in the camera raw space. If working
with raw channels rather than demosaiced raw pixel vectors, an appropriate command is
EQ-TARGET;temp:intralink-;sec2.4;116;388 dcraw -v -D -4 -T filename:

The above DCRaw commands are explained in Table 3.

4. Measure average R, G, and B values over a 64 × 64 block of pixels at the center of each
patch. Each patch can then be associated with an appropriate average raw pixel vector.
5. Build a 3 × n matrix A containing the vectors of the XYZ color space for each patch
1; : : : ; n as columns:
2 3
X1 X2 · · · Xn
6 7
A ¼ 4 Y 1 Y 2 · · · Y n 5: (11) EQ-TARGET;temp:intralink-;e011;116;295

Z1 Z2 · · · Zn

Similarly, build a 3 × n matrix B containing the corresponding raw pixel vectors as

columns:
2 3
R1 R2 · · · Rn
6 7
B ¼ 4 G1 G2 · · · Gn 5: (12) EQ-TARGET;temp:intralink-;e012;116;213

B1 B2 · · · Bn

6. Estimate the 3 × 3 characterization matrix T that transforms B to A:

A ≈ T B:
EQ-TARGET;temp:intralink-;e013;116;139 (13)

A preliminary solution is obtained using linear least-squares minimization:2,15

T ¼ ABT ðBBT Þ−1 ;

EQ-TARGET;temp:intralink-;e014;116;101 (14)

where the T superscript denotes the transpose operator.

Optical Engineering 110801-8 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

7. Use the preliminary T estimate to calculate a new set of estimated CIE XYZ tristimulus
values A 0 according to Eq. (13). Transform A and A 0 to the perceptually uniform CIE LAB
reference color space, and calculate the color difference ΔEi between the estimated tri-
stimulus values and real tristimulus values for each patch i. The set fΔEi g can be used to
calculate the DSC/SMI.8,15 Note that, to satisfy the Luther-Ives condition exactly, it it is
necessary that A 0 ¼ A, in which case a DSC/SMI score of 100 would be obtained.
8. Optimize T by minimizing fΔEi g using the nonlinear optimization technique recom-
mended by ISO 17321-1. The final DSC/SMI defines the final potential color error.
Ideally, include a constraint that preserves the characterization illuminant WP.
9. Scale the final T according to the normalization required for its practical implementation.
This is discussed below.
Provided WB was disabled in step 3, the characterization matrix T can be used with arbitrary
scene illumination. However, optimum results will be obtained for scene illumination with a WP
that closely matches that of the characterization illuminant.
Figure 5 shows how the matrix elements of an optimized characterization matrix vary as a
function of characterization illuminant CCT for the Olympus E-M1 camera.
For the same camera, Fig. 6(a) shows a photo of a color chart in the camera raw space taken
under D65 illumination. When the camera raw space RGB values are interpreted as RGB values
in the sRGB color space for display purposes without any color characterization matrix applied,
a strong green color tint is revealed, which arises from the greater transmission of the green
Bayer filter. Figure 6(b) shows the same photo converted into the sRGB color space by applying
an optimized characterization matrix T, followed by a matrix that converts the colors from the
CIE XYZ color space to sRGB. Evidently, the colors are now displayed correctly.

2.5 Characterization Matrix Normalization

Normalization of a characterization matrix refers to scaling of the entire matrix so that all matrix
elements are scaled identically. A typical normalization applied in practice is to ensure that the
matrix maps between the characterization illuminant WP expressed using the CIE XYZ color
space and the camera raw space such that the raw data just saturates when a 100% neutral diffuse
reflector is photographed under the characterization illuminant. The green raw channel is
typically the first to saturate.
For example, if the characterization illuminant is D65, then T can be normalized such that its
inverse provides the following mapping:

Fig. 5 Variation of the matrix elements of a characterization matrix for the Olympus E-M1 camera
as a function of characterization illuminant CCT.

Optical Engineering 110801-9 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

Fig. 6 (a) Photo of a color chart in the camera raw space taken under D65 illumination.
(b) The same photo converted to the sRGB color space.

2 3 2 3
RðWPÞ XðWPÞ ¼ 0.9504
6 7 6 7
4 GðWPÞ 5
EQ-TARGET;temp:intralink-;e015;116;409 ¼ T −1 4 YðWPÞ ¼ 1.0000 5 ; (15)
BðWPÞ D65 ZðWPÞ ¼ 1.0888 D65

where maxfRðWPÞ; GðWPÞ; BðWPÞg ¼ 1. Since the green raw channel is typically the first to
saturate under most types of illumination, it will typically be the case that GðWPÞ ¼ 1, whereas
RðWPÞ < 1 and BðWPÞ < 1.
For example, the Olympus E-M1 characterization matrices used by Fig. 5 for the 4200 and
6800 K calibration illuminants are defined by
2 3 2 3
0.8680 0.3395 0.2133 1.2105 0.2502 0.1882
6 7 6 7
T 4200 K ¼ 4 0.2883
EQ-TARGET;temp:intralink-;e016;116;296 0.8286 −0.0216 5; T 6800 K ¼ 4 0.4586 0.8772 −0.1328 5:
0.0425 −0.2647 1.7637 0.0936 −0.2788 1.9121
(16)

These matrices are normalized such that the WP of the characterization illuminant maps to raw
values where the green raw channel just reaches saturation:
2 3 2 3
XðWPÞ ¼ 1.0019
EQ-TARGET;temp:intralink-;e017;116;196

RðWPÞ ¼ 0.6337
6 7 6 7
6 GðWPÞ ¼ 1.0000 7 ¼ T −1 6 7
4 5 4200 K 4 YðWPÞ ¼ 1.0000 5 ;
BðWPÞ ¼ 0.5267 ZðWPÞ ¼ 0.6911 4200 K
4200 K
2 3 2 3
RðWPÞ ¼ 0.4793 XðWPÞ ¼ 0.9682
6 7 6 7
6 GðWPÞ ¼ 1.0000 7 ¼ T −1 6 7
4 5 6800 K 4 YðWPÞ ¼ 1.0000 5 : (17)
BðWPÞ ¼ 0.7312 6800 K ZðWPÞ ¼ 1.1642 6800 K

Optical Engineering 110801-10 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

3 White Balance
A remarkable property of the HVS is its ability to naturally adjust to the ambient lighting
conditions. For example, if a 100% neutral diffuse reflector is placed in a photographic scene
illuminated by daylight, the reflector appears to be neutral white. Later in the day when there is
a change in the chromaticity or CCT of the scene illumination, the color of the reflector would
be expected to change accordingly. However, the reflector will continue to appear neutral white.
In other words, the perceived color of objects remains relatively constant under varying types of
scene illumination, which is known as color constancy.3,4
The chromatic adaptation mechanism by which the HVS achieves color constancy is com-
plex and not fully understood, but a simplified explanation is that the HVS aims to discount the
chromaticity of the illuminant.30 Back in 1902, von-Kries postulated that this is achieved by an
independent scaling of each eye cone response function.3,4 The color stimulus that an observer
adapted to the ambient conditions considers to be neutral white (perfectly achromatic with 100%
relative luminance) is defined as the adapted white.31
Since camera response functions do not naturally emulate the HVS by discounting the chro-
maticity of the scene illumination, an output image will appear too warm or too cold if it is
displayed using illumination with a WP that does not match the adapted white for the photo-
graphic scene at the time the photograph was taken. This is known as incorrect WB. The issue
can be solved by implementing the following computational strategy.
1. Inform the camera of the adapted white before taking a photograph. Due to the complex
dependence of the true adapted white on the ambient conditions, this task is replaced by a
simpler one in practice, namely to identify the scene illumination WP. For example, a WB
preset corresponding to the scene illumination can be manually selected, a scene illumi-
nation CCT estimate can be manually entered, or the camera can compute its own estimate
by analyzing the raw data using the automatic WB function. In all cases, the camera esti-
mate for the scene illumination WP is known as the camera neutral32 or adopted white
(AW).31 (This illuminant estimation step should not be confused with WB. Illuminant
estimation refers to the computational approaches used by the automatic WB function
to estimate the scene illumination WP. A very simple illuminant estimation approach
is the “gray world” method,33 which assumes that the average of all of the scene colors
will turn out to be achromatic. Another simple approach is to assume that the brightest
white is likely to correspond to the scene illumination WP.34 However, practical illuminant
estimation algorithms are much more sophisticated.35,36)
2. Choose a standard reference white that will be used when displaying the output image.
If the image will be displayed using a standard output-referred color space such as sRGB,
the chosen reference white will be that of the output-referred color space, which is CIE
illuminant D65 in the case of sRGB.
3. Chromatically adapt the image colors by adapting the scene illumination WP estimate (the
AW) so that it becomes the reference white of the chosen output-referred color space. This
white balancing step is achieved by applying a CAT.
The CAT needs to be applied as part of the overall color conversion from the camera raw space
to the chosen output-referred color space. Different approaches for combining these components
exist. The typical approach used in color science is to convert from the camera raw space to CIE
XYZ, apply the CAT, and then convert to the chosen output-referred color space. In the case
of sRGB,
2 3 2 3
RL R
6 7 6 7
4 GL 5
EQ-TARGET;temp:intralink-;e018;116;169 ¼ M−1
sRGB CATAW→D65 T 4 G 5 ; (18)
BL D65 B scene

where T is a characterization matrix that converts from the camera raw space to CIE XYZ and
is optimized for the scene AW, the matrix CATAW→D65 applied in the CIE XYZ color space is
a CAT that adapts the AW to the D65 reference white of the sRGB color space, and finally

Optical Engineering 110801-11 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

M−1
sRGB is the matrix that converts from CIE XYZ to the linear form of the sRGB color
space:
2 3
3.2410 −1.5374 −0.4986
6 7
M−1sRGB ¼ 4 −0.9692 1.8760 0.0416 5: (19)
EQ-TARGET;temp:intralink-;e019;116;710

0.0556 −0.2040 1.0570

In particular, the AW in the camera raw space is mapped to the reference white of the output-
referred color space defined by the unit vector in the output-referred color space:
2 3 2 3
RL ¼ 1 RðAWÞ
6 7 6 7
4 GL ¼ 1 5 ¼ M−1
sRGB CATAW→D65 T 4 GðAWÞ 5
EQ-TARGET;temp:intralink-;e020;116;624 : (20)
BL ¼ 1 D65 BðAWÞ scene

When the encoded output image is viewed on a calibrated display monitor, a scene object that
the HVS regarded as being white at the time the photograph was taken will now be displayed
using the D65 reference white. Ideally, the ambient viewing conditions should match those
defined as appropriate for viewing the sRGB color space.
If the scene illumination WP estimate is far from the true scene illumination WP, then incor-
rect WB will be evident to the HVS. If the scene illumination CCT estimate is higher than the
true CCT, then the photo will appear too warm. Conversely, if the scene illumination CCT esti-
mate is lower than the true CCT, then the photo will appear too cold.
Figure 7(a) shows a photo of a color chart taken under 2700 K CCT tungsten illumination
using the Olympus E-M1 camera. A characterization matrix T was applied to convert the colors
into CIE XYZ, followed by M−1 sRGB to convert the colors to sRGB. Evidently, the true color of the
scene illumination is revealed since no chromatic adaptation has been performed by the camera.
In other words, the photo appears too warm in relation to the D65 reference white of the sRGB

Fig. 7 (a) Photo of a color chart taken under 2700 K CCT tungsten illumination and converted to
the sRGB color space for display without any chromatic adaptation. (b) White balanced photo
obtained by including a CAT to adapt the scene illumination WP to the D65 reference white of
the sRGB color space.

Optical Engineering 110801-12 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

color space. Figure 7(b) shows the same photo after white balancing by including a CAT that
chromatically adapts the scene illumination WP to the sRGB color space D65 reference white,
which has a 6504 K CCT and Duv ¼ 0.0032 color tint.

3.1 Chromatic Adaptation Transforms

A CAT is a computational technique for adjusting the WP of a given SPD. It achieves this goal by
attempting to mimic the chromatic adaptation mechanism of the HVS. In the context of digital
cameras, the most important CATs are the Bradford CAT and raw channel scaling.
In 1902, von-Kries postulated that the chromatic adaptation mechanism be modeled as an
independent scaling of each eye cone response function,3,4 which is equivalent to scaling the L,
M, and S tristimulus values in the LMS color space. To illustrate the von-Kries CAT, consider
adapting the scene illumination WP estimate (the AW) to the WP of D65 illumination:
2 3 2 3
X X
6 7 6 7
4Y5
EQ-TARGET;temp:intralink-;e021;116;579 ¼ CATAW→D65 4 Y 5 : (21)
Z D65 Z scene

In this case, the von-Kries CAT that must be applied to all raw pixel vectors can be written as
2 3
LðD65Þ
6 LðAWÞ 0 0 7
6 7
6 MðD65Þ 7
−1 6 7
CATAW→D65 ¼ MvK 6 0 0 7MvK : (22)
6 MðAWÞ 7
EQ-TARGET;temp:intralink-;e022;116;504

6 7
4 SðD65Þ 5
0 0
SðAWÞ

The matrix MvK transforms each raw pixel vector into a diagonal matrix in the LMS color space.
Modern forms of MvK include matrices based on the cone fundamentals defined by the CIE
in 200637 and the Hunt–Pointer–Estevez transformation matrix38 defined by
2 3
0.38971 0.68898 −0.07868
6 7
MvK ¼ 4 −0.22981
EQ-TARGET;temp:intralink-;e023;116;368 1.18340 0.04641 5: (23)
0.00000 0.00000 1.00000

After applying MvK , the L, M, and S values are independently scaled according to the von-Kries
hypothesis. In the present example, the scaling factors arise from the ratio between the AW and
D65 WPs. These can be obtained from the following WP vectors:
2 3 2 3
LðAWÞ XðWPÞ
EQ-TARGET;temp:intralink-;e024;116;270

6 7 6 7
4 MðAWÞ 5 ¼ MvK 4 YðWPÞ 5
SðAWÞ ZðWPÞ scene
2 3 2 3
LðD65Þ XðWPÞ ¼ 0.9504
6 7 6 7
4 MðD65Þ 5 ¼ MvK 4 YðWPÞ ¼ 1.0000 5 : (24)
SðD65Þ ZðWPÞ ¼ 1.0888 D65

Finally, the inverse of the transformation matrix MvK is applied to convert each raw pixel
vector back into the CIE XYZ color space.
The Bradford CAT39 can be regarded as an improved version of the von-Kries CAT. A sim-
plified linearized version is recommended by the ICC for use in digital imaging.40 The linear
Bradford CAT can be implemented in an analogous fashion as the von-Kries CAT, the difference
being that the L, M, and S tristimulus values are replaced by ρ, γ, and β, which correspond to a
“sharpened” artificial eye cone space. The transformation matrix is defined by

Optical Engineering 110801-13 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

2 3
0.8951 0.2664 −0.1614
6 7
MBFD ¼ 4 −0.7502
EQ-TARGET;temp:intralink-;e025;116;735 1.7135 0.0367 5: (25)
0.0389 −0.0685 1.0296

Analogous to the independent scaling of the eye cone response functions hypothesized by
von-Kries, a type of CAT can be applied in the camera raw space by directly scaling the raw
channels. Consider a Bayer block for the AW obtained by photographing a 100% neutral diffuse
reflector under the scene illumination. The following operation will adapt the AW to the refer-
ence white of the camera raw space:
2 3 2 3
R R
6 7 6 7
4 G 5 ¼ CATAW→RW 4 G 5 ; (26) EQ-TARGET;temp:intralink-;e026;116;621

B RW B scene

where
2 3
1
6 RðAWÞ 0 0 7
6 7
6 7
6 1 7
CATAW→RW ¼D¼6 0 0 7 : (27)
6 GðAWÞ 7
EQ-TARGET;temp:intralink-;e027;116;549

6 7
4 1 5
0 0
BðAWÞ scene

The diagonal scaling factors, known as raw channel multipliers, can be obtained directly from
the raw data using the AW calculated by the camera. For example, AW ¼ D65 for D65 scene
illumination, in which case
2 3
1
6 RðD65Þ 0 0 7
6 7
6 7
6 1 7
CATD65→RW ¼ DD65 ¼ 6 0 0 7; (28)
6 GðD65Þ 7
EQ-TARGET;temp:intralink-;e028;116;416

6 7
4 1 5
0 0
BðD65Þ

where RðD65Þ, GðD65Þ, and BðD65Þ are extracted from the Bayer block for a 100% neutral
diffuse reflector photographed under D65 scene illumination.
In the context of digital cameras, the type of CAT defined by raw channel multipliers has been
found to work better in practice, particularly for extreme cases.21,32 A reason for this is that the raw
channel multipliers are applied in the camera raw space prior to application of a color conversion
matrix. The camera raw space corresponds to a physical capture device, but CATs such as the linear
Bradford CAT are applied in the CIE XYZ color space after applying a color conversion matrix that
contains error. In particular, color errors that have been minimized in a nonlinear color space such
as CIE LAB will be unevenly amplified, so the color conversion will no longer be optimal.41

4 Smartphone Cameras
Smartphone manufacturers, along with commercial raw conversion software developers, typi-
cally implement the conventional type of computational color conversion strategy used in color
science that was introduced in Sec. 3. Since the camera raw space is transformed into CIE XYZ
as the first step, image processing techniques can be applied in the CIE XYZ color space (or
following a transformation into some other intermediate color space) before the final transfor-
mation to an output-referred RGB color space.
Consider the white-balanced transformation from the camera raw space to an output-referred
RGB color space. Unlike in traditional digital cameras, the color demosaic is typically carried
out first, so the vector notation used for the camera raw space below refers to raw pixel vectors

Optical Engineering 110801-14 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

rather than Bayer blocks. In the case of sRGB, the transformation that must be applied to every
raw pixel vector is defined by
2 3 2 3
RL R
6 7 −1 6 7
4 GL 5 ¼ MsRGB CATAW→D65 T 4 G 5 : (29)
EQ-TARGET;temp:intralink-;e029;116;711

BL D65 B scene

The conversion can be decomposed into three steps.

1. After the camera has estimated the scene illumination WP (the AW), a characterization
matrix T optimized for the AW that converts from the camera raw space to CIE XYZ is
applied:
2 3 2 3
X R
6 7 6 7
4Y 5 ¼ T4 G 5 : (30) EQ-TARGET;temp:intralink-;e030;116;595

Z scene B scene

The optimized matrix T is typically normalized such that the AW in the CIE XYZ space is
obtained when the raw pixel vector corresponding to a neutral diffuse reflector illuminated
by the AW just reaches saturation:
2 3 2 3
XðAWÞ RðAWÞ
6 7 6 7
4 YðAWÞ 5 ¼ T 4 GðAWÞ 5 ; (31) EQ-TARGET;temp:intralink-;e031;116;499

ZðAWÞ scene BðAWÞ scene

where YðWPÞ ¼ 1 and maxfRðAWÞ; GðAWÞ; BðAWÞg ¼ 1. As discussed in Sec. 2.5,

the green component is typically the first to saturate, so RðAWÞ < 1 and BðAWÞ < 1
in general.
2. Since T does not alter the AW, a CAT is applied to achieve WB by adapting the AW to the
reference white of the chosen output-referred color space. This is D65 in the case of sRGB:
2 3 2 3
X X
6 7 6 7
4Y 5 ¼ CATAW→D65 4 Y 5 : (32) EQ-TARGET;temp:intralink-;e032;116;377

Z D65 Z scene

The ICC recommends implementing the CAT using the linear Bradford CAT matrix
defined by Eq. (25).
3. A matrix that converts from CIE XYZ to the linear form of the chosen output-referred
color space is applied. In the case of sRGB,
2 3 2 3
RL X
6 7 −1 6 7
4 GL 5 ¼ MsRGB 4 Y 5 : (33) EQ-TARGET;temp:intralink-;e033;116;266

BL D65 Z D65

Finally, the digital output levels of the output image are determined by applying the nonlinear
gamma encoding curve of the output-referred color space and reducing the bit depth to 8. In
modern digital imaging, encoding gamma curves are designed to minimize visible banding arti-
facts when the bit depth is reduced, and the non-linearity introduced is later reversed by the
display gamma.28
To see that WB is correctly achieved, the above steps can be followed for the specific case of the
raw pixel vector that corresponds to the AW. As required by Eq. (20), it is found that this maps to
the reference white of the output-referred color space defined by the unit vector in that color space:
2 3 2 3
RL ¼ 1 RðAWÞ
6 7 6 7
4 GL ¼ 1 5 ¼ M−1sRGB CATAW→D65 T 4 GðAWÞ 5
EQ-TARGET;temp:intralink-;sec4;116;111 :
BL ¼ 1 D65 BðAWÞ scene

Optical Engineering 110801-15 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

Although the matrix transformation defined by Eq. (29) appears to be straightforward, the
characterization matrix T should in principle be optimized for the AW. However, it is impractical
to determine a characterization matrix optimized for each possible scene illumination WP that
could occur. For example, if CCTs are specified to the nearest Kelvin and color tint is neglected,
then 12,000 matrices would be required to cover scene illumination WPs from 2000 to 14,000 K.
The computationally simplest solution used on some mobile phone cameras is to approxi-
mate the optimized characterization matrix T using a single fixed matrix optimized for a rep-
resentative illuminant. For example, this could be D65 illumination, in which case T optimized
for the AW is approximated as T D65 . The drawback of this very simple approach is that the color
conversion loses some accuracy when the scene illumination WP differs significantly from the
WP of the representative illuminant.
As described below, an advanced solution to the problem is to adopt the type of approach used
by the Adobe DNG converter.32 The idea is to interpolate between two preset characterization
matrices that are optimized for use with either a low-CCT or high-CCT illuminant. For a given
scene illumination, an interpolated matrix optimized for the CCT of the AW can be determined.

4.1 Interpolation Algorithm

If using the advanced approach mentioned above, the optimized characterization matrix T
required by Eq. (29) can be calculated by interpolating between two characterization matrices
T 1 and T 2 based on the scene illumination CCT estimate denoted by CCT(AW), together with the
CCTs of the two characterization illuminants denoted by CCT1 and CCT2 , respectively, with
CCT1 < CCT2 . For example, illuminant 1 could be a low-CCT illuminant such as CIE illuminant
A, whereas illuminant 2 could be a high-CCT illuminant such as D65.
The first step is to appropriately normalize T 1 and T 2 . Although characterization matrices are
typically normalized according to their corresponding characterization illuminant WPs as dem-
onstrated in Sec. 2.5, it is more convenient to normalize T 1 and T 2 according to a common WP
when implementing an interpolation algorithm. Unfortunately, the AW cannot be expressed
using the CIE XYZ color space at this stage since T is yet to be determined. Instead, the common
WP could be chosen to be the reference white of the output-referred color space, which is D65
for sRGB. In this case, T 1 and T 2 should both be scaled according to Eq. (15):
2 3 2 3
RðWPÞ XðWPÞ ¼ 0.9504
EQ-TARGET;temp:intralink-;e034;116;363

6 7 6 7
4 GðWPÞ 5 ¼ T 1 −1 4 YðWPÞ ¼ 1.0000 5 ;
BðWPÞ D65 ZðWPÞ ¼ 1.0888 D65
2 3 2 3
RðWPÞ XðWPÞ ¼ 0.9504
6 7 6 7
4 GðWPÞ 5 ¼ T 2 −1 4 YðWPÞ ¼ 1.0000 5 ; (34)
BðWPÞ D65 ZðWPÞ ¼ 1.0888 D65

where YðWPÞ ¼ 1 and maxfRðWPÞ; GðWPÞ; BðWPÞg ¼ 1.

Unless the smartphone utilizes a color sensor that can directly estimate the scene illumination
WP in terms of ðx; yÞ chromaticity coordinates, the AW is calculated by the camera in terms of
raw values RðAWÞ, GðAWÞ, and BðAWÞ, so the AW cannot be expressed using the CIE XYZ
color space prior to the interpolation. However, the corresponding CCT(AW) requires knowl-
edge of the ðx; yÞ chromaticity coordinates, which means converting to CIE XYZ via a matrix
transformation T that itself depends upon the unknown CCT(AW). This problem can be solved
using a self-consistent iteration procedure.32
1. Make a guess for the AW chromaticity coordinates, ðxðAWÞ; yðAWÞÞ. For example, the
chromaticity coordinates corresponding to one of the characterization illuminants could
be used.
2. Find the CCT value CCT(AW) that corresponds to the chromaticity coordinates
ðxðAWÞ; yðAWÞÞ. A widely used approach is to convert ðxðAWÞ; yðAWÞÞ into the corre-
sponding ðuðAWÞ; vðAWÞÞ chromaticity coordinates on the 1960 UCS chromaticity dia-
gram,23,24 where isotherms are normal to the Planckian locus. This enables CCT(AW) to

Optical Engineering 110801-16 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

be determined using Robertson’s method.42 Alternatively, approximate formulas43–45 or

more recent algorithms46 can be implemented.
3. Perform the interpolation so that

TðAWÞ ¼ f½T 1 ðCCT1 Þ; T 2 ðCCT2 Þ;

EQ-TARGET;temp:intralink-;e035;116;697 (35)

where f is the interpolation function. The interpolation is valid for

EQ-TARGET;temp:intralink-;sec4.1;116;654 CCT1 ≤ CCTðAWÞ ≤ CCT2 :

If CCTðAWÞ < CCT1 , then T should be set equal to T 1 , and if CCTðAWÞ > CCT2 , then T
should be set equal to T 2 .
4. Use T to transform the AW from the camera raw space to the CIE XYZ color space:
2 3 2 3
XðAWÞ RðAWÞ
6 7 6 7
4 YðAWÞ 5 ¼ T 4 GðAWÞ 5 : (36) EQ-TARGET;temp:intralink-;e036;116;586

ZðAWÞ scene BðAWÞ scene

This yields a new guess for ðxðAWÞ; yðAWÞÞ.

5. Repeat the procedure starting from step 2 until ðxðAWÞ; yðAWÞÞ, CCT(AW), and T all
converge to a stable solution.
After the interpolation has been carried out, T inherits the normalization of Eq. (34). However,
the AW can now be expressed using the CIE XYZ color space, so T can be renormalized to
satisfy Eq. (31).
If the smartphone utilizes a color sensor that can directly estimate the scene illumination WP
in terms of ðx; yÞ chromaticity coordinates, then only steps 2 and 3 above are required.

5 Traditional Digital Cameras

Consider again the white-balanced transformation from the camera raw space to an output-
referred RGB color space. In the case of sRGB, the transformation is defined by Eq. (29):
2 3 2 3
RL R
6 7 −1 6 7
4 GL 5 ¼ M sRGB CATAW→D65 T 4 G 5 ;
EQ-TARGET;temp:intralink-;sec5;116;351

BL D65 B scene

where CATAW→D65 adapts the scene illumination WP estimate (the AW) to the sRGB color space
D65 reference white. Traditional camera manufacturers typically re-express the above equation
in the following manner:
2 3 2 3
RL R
6 7 6 7
4 GL 5 ¼ RD4 G 5 : (37) EQ-TARGET;temp:intralink-;e037;116;254

BL D65 B scene

This equation can be interpreted by decomposing the conversion into two steps.
1. The matrix D is a diagonal WB matrix containing raw channel multipliers appropriate for
the AW:
2 3
1
6 RðAWÞ 0 0 7
6 7
6 7
6 1 7
D¼6 0 0 7 : (38)
6 GðAWÞ 7
EQ-TARGET;temp:intralink-;e038;116;150

6 7
4 1 5
0 0
BðAWÞ scene

Optical Engineering 110801-17 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

These are applied to the raw channels before the color demosaic. As shown by Eq. (27),
the raw channel multipliers, in particular, serve to chromatically adapt the AW to the refer-
ence white of the camera raw space:
2 3 2 3
R¼1 RðAWÞ
6 7 6 7
4G¼15 ¼ D4 GðAWÞ 5 : (39) EQ-TARGET;temp:intralink-;e039;116;699

B ¼ 1 reference BðAWÞ scene

2. The matrix R is a color rotation matrix optimized for the scene illumination. After the
color demosaic has been performed, R is applied to convert directly from the camera raw
space to the linear form of the chosen output-referred color space. By comparison of
Eqs. (29) and (37), R is algebraically defined as

R ¼ M−1
EQ-TARGET;temp:intralink-;e040;116;586
−1
sRGB CATAW→D65 T D : (40)

Color rotation matrices have the important property that each of their rows sums to unity:

Rð1;1Þ þ Rð1;2Þ þ Rð1;3Þ ¼ 1;

EQ-TARGET;temp:intralink-;e041;116;542

Rð2;1Þ þ Rð2;2Þ þ Rð2;3Þ ¼ 1;

Rð3;1Þ þ Rð3;2Þ þ Rð3;3Þ ¼ 1: (41)

Consequently, R maps the reference white of the camera raw space directly to the reference
white of the output-referred color space.21 In the case of sRGB,
2 3 2 3
RL ¼ 1 R¼1
6 7 6 7
4 GL ¼ 1 5 ¼ R4 G ¼ 1 5 : (42) EQ-TARGET;temp:intralink-;e042;116;451

BL ¼ 1 D65 B ¼ 1 reference

Combining Eqs. (39) and (42) shows that overall WB is achieved since the raw pixel vector
corresponding to the AW is mapped to the reference white of the output-referred color space:
2 3 2 3
RL ¼ 1 RðAWÞ
6 7 6 7
4 GL ¼ 1 5 ¼ R D 4 GðAWÞ 5 :
EQ-TARGET;temp:intralink-;e043;116;363 (43)
BL ¼ 1 D65 BðAWÞ scene

Like the characterization matrix T, the color rotation matrix R should in principle be opti-
mized for the scene illumination. Rather than use an interpolation-based approach, the reformu-
lation in the form of Eq. (37) enables traditional camera manufacturers to adopt an alternative
and computationally simple approach that can be straightforwardly implemented on fixed-point
number architecture.

5.1 Multiplier and Matrix Decoupling

Although Eq. (37) appears to be a straightforward reformulation of Eq. (29), it has several advan-
tages that arise from the raw channel multipliers contained within the WB matrix D having been
extracted. As shown in Fig. 8, the variation of the elements of a color rotation matrix with respect
to CCT is very small. The stability is greater than that of the elements of a conventional char-
acterization matrix T, as evident from comparison of Figs. 5 and 8.
Consequently, it suffices to determine a small set of n preset color rotation matrices that cover
a range of WPs or CCTs, with each matrix optimized for a particular preset WP or CCT:

Ri ¼ M−1
EQ-TARGET;temp:intralink-;e044;116;119
−1
sRGB CATAW→D65 T i Di ; (44)

where i ¼ 1: : : n. When the AW is calculated by the camera, the color rotation matrix Ri opti-
mized for the closest-matching WP or CCT preset can be selected. However, the WB matrix D

Optical Engineering 110801-18 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

Fig. 8 Variation of the matrix elements of the raw-to-sRGB color rotation matrix R used by the
Olympus E-M1 camera as a function of CCT.

appropriate for the AW is always applied prior to Ri , so the overall color conversion can be
expressed as
2 3 2 3
RL R
6 7 6 7
4 GL 5
EQ-TARGET;temp:intralink-;e045;116;440 ¼ ðM−1
sRGB CATAW→D65 T i D−1
i Þ D4 G 5 : (45)
BL D65 B scene

Since D is decoupled from the rotation matrices, this approach will achieve correct WB without
the need to interpolate the rotation matrices.
It should be noted that the camera raw space correctly represents the scene (albeit via a non-
standard color model) and that the raw channel multipliers contained within D are not applied to
“correct” anything concerning the representation of the true scene white by the camera raw
space, as often assumed. The multipliers are applied to chromatically adapt the AW to the refer-
ence white of the camera raw space as part of the overall CAT required to achieve WB by emu-
lating the chromatic adaptation mechanism of the HVS. As shown in Fig. 4, the reference white
of a camera raw space is typically a magenta color when expressed using CIE colorimetry, but it
serves as a useful intermediary stage in the required color transformation as it facilitates the
extraction of a channel scaling component that can be decoupled from the matrix operation.
Other advantages of the reformulation include the following.
• The raw channel multipliers contained within D can be applied to the raw channels before
the color mosaic is performed. This results in a demosaic of better quality.21
• The method can be efficiently implemented on fixed-point architecture.47
• If desired, part of the raw channel scaling can be carried out in the analog domain using
analog amplification. This is beneficial for image quality if the analog-to-digital converter
(ADC) does not have a sufficiently high bit depth. Note that this type of analog ampli-
fication will affect the input to output-referred unit conversion factors gi defined by
Eq. (80) in the Appendix.
• The raw channel multipliers contained within D that appear in Eq. (37) are stored in the
proprietary raw file metadata and are applied by the internal JPEG image-processing
engine of the camera. Since the raw channel multipliers do not affect the raw data, they
can be utilized by external raw conversion software provided by the camera manufacturer
and can be easily adjusted by the user.

Optical Engineering 110801-19 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

• Scene illumination presets that include a color tint can be straightforwardly implemented
by storing the appropriate preset color rotation matrices and raw channel multipliers, as
illustrated in Sec. 5.2.

5.2 Example: Olympus E-M1

Although the color matrices used by the camera manufacturers are generally unknown, certain
manufacturers such as Sony and Olympus do reveal information about the color rotation matrices
used by their cameras that can be extracted from the raw metadata.
Table 1 lists the data illustrated in Fig. 8 for the preset color rotation matrices used by the
Olympus E-M1 digital camera, along with the scene illumination CCT ranges over which each
matrix is applied. Figure 9 shows how the raw channel multipliers for the same camera vary as a
function of CCT. The data was extracted from raw metadata using the freeware “ExifTool”
application.48 The color conversion strategy of the camera can be summarized as follows.

1. The camera determines the scene illumination WP estimate (the AW) using either an auto-
WB algorithm, a selected scene illumination preset, or a custom CCT provided by the user.

Table 1 Raw-to-sRGB color rotation matrices corresponding to ranges of in-camera custom

CCTs for the Olympus E-M1 camera with 12-100/4 lens and v4.1 firmware. The middle column
lists the matrices extracted from the raw metadata, which are 8-bit fixed-point numbers. By dividing
by 256, the right column lists to four decimal places the same matrices such that each row sums to
unity rather than 256.

CCT range (K) Rotation matrix (fixed point) Rotation matrix

2 3 2 3
320 −36 −28 1.2500 −0.1406 −0.1094
6 7 6 7
2000 → 3000 4 −68 308 16 5 4 −0.2656 1.2031 0.0625 5
14 −248 490 0.0547 −0.9688 1.9141
2 3 2 3
332 −52 −24 1.2969 −0.2031 −0.0938
6 7 6 7
3100 → 3400 4 −58 320 −6 5 4 −0.2266 1.2500 −0.0234 5
12 −192 436 0.0469 −0.7500 1.7031
2 3 2 3
340 −60 −24 1.3281 −0.2344 −0.0938
6 7 6 7
3500 → 3700 4 −56 324 −12 5 4 −0.2188 1.2656 −0.0469 5
12 −172 416 0.0469 −0.6719 1.6250
2 3 2 3
346 −68 −22 1.3516 −0.2656 −0.0859
6 7 6 7
3800 → 4000 4 −52 332 −24 5 4 −0.2031 1.2969 −0.0938 5
10 −160 406 0.0391 −0.6250 1.5859
2 3 2 3
346 −68 −22 1.3516 −0.2656 −0.0859
6 7 6 7
4200 → 4400 4 −48 332 −28 5 4 −0.1875 1.2969 −0.1094 5
12 −160 404 0.0469 −0.6250 1.5781
2 3 2 3
354 −76 −22 1.3828 −0.2969 −0.0859
6 7 6 7
4600 → 5000 4 −44 336 −36 5 4 −0.1719 1.3125 −0.1406 5
10 −148 394 0.0391 −0.5781 1.5391
2 3 2 3
366 −88 −22 1.4297 −0.3438 −0.0859
6 7 6 7
5200 → 5600 4 −42 340 −42 5 4 −0.1641 1.3281 −0.1641 5
10 −136 382 0.0391 −0.5313 1.4922
2 3 2 3
374 −96 −22 1.4609 −0.3750 −0.0859
6 7 6 7
5800 → 6600 4 −42 348 −50 5 4 −0.1641 1.3594 −0.1953 5
8 −124 372 0.0313 −0.4844 1.4531
2 3 2 3
388 −108 −24 1.5156 −0.4219 −0.0938
6 7 6 7
6800 → 14000 4 −38 360 −66 5 4 −0.1484 1.4063 −0.2578 5
8 −112 360 0.0313 −0.4375 1.4063

Optical Engineering 110801-20 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

Fig. 9 Raw channel multipliers used by the Olympus E-M1 camera as a function of CCT. The
camera uses the same multipliers for both of the green channels.

Table 2 Raw-to-sRGB color rotation matrices and associated raw channel multipliers corre-
sponding to in-camera scene modes for the Olympus E-M1 camera with 12-100/4 lens and
v4.1 firmware. All values are 8-bit fixed-point numbers that can be divided by 256. Since the scene
mode presets include a color tint away from the Planckian locus, the multipliers and matrices do
not necessarily have the same values as the custom CCT presets with the same CCT listed in
Table 1.

Scene mode CCT (K) Multipliers Rotation matrix (fixed point)

2 3
366 −88 −22
6 7
Fine weather 5300 474 256 414 4 −42 340 −42 5
10 −136 382
2 3
388 −108 −24
6 7
Fine weather with shade 7500 552 256 326 4 −38 360 −66 5
8 −112 360
2 3
374 −96 −22
6 7
Cloudy 6000 510 256 380 4 −42 348 −50 5
8 −124 372
2 3
320 −36 −28
6 7
Tungsten (incandescent) 3000 276 256 728 4 −68 308 16 5
14 −248 490
2 3
430 −168 −6
6 7
Cool white fluorescent 4000 470 256 580 4 −50 300 6 5
12 −132 376
2 3
366 −88 −22
6 7
Underwater 450 256 444 4 −42 340 −42 5
10 −136 382
2 3
366 −88 −22
6 7
Flash 5500 562 256 366 4 −42 340 −42 5
10 −136 382

The AW is used to calculate the appropriate raw channel multipliers via Eq. (38) so that the
diagonal WB matrix D can be applied to the raw channels. In particular, D serves to adapt
the AW to the reference white of the camera raw space.
2. After the color demosaic is performed, the camera chooses the preset color rotation matrix
Ri optimized for illumination with a CCT that provides the closest match to the CCT
associated with the AW, or the closest matching scene illumination preset.

Optical Engineering 110801-21 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

3. The camera applies Ri to convert to the output-referred color space selected in-camera by the
user, such as sRGB. In particular, the camera raw space reference white is mapped to the
reference white of the selected output-referred color space, which is D65 in the case of sRGB.
The Olympus E-M1 camera also includes several scene illumination presets. The color rotation
matrices and associated raw channel multipliers for these scene presets are listed in Table 2. For a
given CCT, notice that the scene preset matrices and multipliers are not necessarily the same as
those listed in Table 1. This is because the scene preset renderings include a color tint away from
the Planckian locus, so the chromaticity coordinates are not necessarily the same as those listed
in Table 1 for a given CCT. For the same reason, notice that the “fine weather,” “underwater,” and
“flash” scene mode presets actually use the same color rotation matrix but use very different raw
channel multipliers.
For any given camera model, all preset color rotation matrices are dependent on factors such
as the output-referred color space selected by the user in the camera settings (such as sRGB or
Adobe® RGB), the lens model used to take the photograph, and the firmware version. Due to
sensor calibration differences between different examples of the same camera model, there can
also be a dependence on the individual camera used to take the photograph.
For example, Fig. 10(a) shows a photo of a color chart in the camera raw space taken under
D65 illumination. Like Fig. 6(a), the green color tint arises from the fact that the camera raw
space RGB values are being interpreted as RGB values in the sRGB color space for display
purposes without any color characterization matrix applied to convert the colors. Figure 10(b)
shows the same photo after applying the diagonal WB matrix D to chromatically adapt the AW to
the camera raw space reference white. The raw channel multipliers remove the green tint, but the
photo remains in the camera raw space. Remarkably, the colors appear realistic, although desa-
turated. To illustrate that the camera raw space reference white is actually a magenta color when
expressed using CIE colorimetry, Fig. 10(c) converts (b) to the sRGB color space without any
further chromatic adaptation by applying a conventional characterization matrix T followed by
M−1sRGB . In contrast, Fig. 10(d) was obtained by applying the appropriate raw channel multipliers
followed by the sRGB color rotation matrix R in place of T and M−1 sRGB . The color rotation matrix
includes a CAT that adapts the camera raw space reference white to the sRGB color space D65

Fig. 10 (a) Photo of a color chart in the camera raw space taken under D65 illumination. (b) After
application of the appropriate raw channel multipliers. These remove the green tint, but the photo
remains in the camera raw space. (c) After application of the appropriate raw channel multipliers
and converting to sRGB without any further chromatic adaptation. The white patch reveals the true
color of the camera raw space reference white. (d) After application of the appropriate raw channel
multipliers and an sRGB color rotation matrix R.

Optical Engineering 110801-22 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

reference white. In this particular case, D ¼ DD65 , so the color rotation matrix R defined by
Eq. (40) becomes

R ≡ RD65 ¼ M−1
EQ-TARGET;temp:intralink-;e046;116;711
−1
sRGB T D65 DD65 : (46)

Substituting into Eq. (37) yields

2 3 2 3
RL R
6 7 −1 −1 6 7
4 GL 5 ¼ MsRGB T D65 DD65 DD65 4 G 5
EQ-TARGET;temp:intralink-;e047;116;667 : (47)
BL D65 B scene

Consequently, the rotation matrix reverses the effect of the WB matrix since the scene and dis-
play illumination is the same.

6 DCRaw Open-Source Raw Converter

The widely used DCRaw open-source raw converter (pronounced “dee-see-raw”) written by D.
Coffin can process a wide variety of raw image file formats. It is particularly useful for scientific
analysis as it can decode raw files without demosaicing, it can apply linear tone curves, and it can
directly output to the camera raw space and the CIE XYZ color space. Some relevant commands
are listed in Table 3. However, DCRaw by default outputs directly to the sRGB color space with a
D65 illumination WP by utilizing a variation of the traditional digital camera strategy described
in the previous section.28

Table 3 A selection of relevant DCraw commands available in version 9.28. Note that the RGB
output colorspace options use color rotation matrices and so should only be used with the correct
raw channel multipliers due to the inbuilt CAT.

-v Print verbose messages

-w Use camera WB, if possible

-A <x y w h> Average a gray box for WB

-r <r g b g> Set custom WB

+M/-M Use/do not use an embedded color matrix

-H [0-9] Highlight mode (0 = clip, 1 = unclip, 2 = blend, 3+ = rebuild)

-o [0-6] Output colorspace (raw, sRGB, Adobe, Wide, ProPhoto, XYZ, ACES)

-d Document mode (no color, no interpolation)

-D Document mode without scaling (totally raw)

-W Do not automatically brighten the image

-b <num> Adjust brightness (default = 1.0)

-g <p ts> Set custom gamma curve (default = 2.222 4.5)

-q [0-3] Set the interpolation quality

-h Half-size color image (twice as fast as “-q 0”)

-f Interpolate RGGB as four colors

-6 Write 16-bit instead of 8-bit

-4 Linear 16-bit, same as “-6 -W -g 1 1”

-T Write TIFF instead of PPM

Optical Engineering 110801-23 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

Recall that the color rotation matrix optimized for use with the scene illumination is defined
by Eq. (40):
R ¼ M−1
EQ-TARGET;temp:intralink-;sec6;116;711
−1
sRGB CATAW→D65 T D :

Although digital cameras typically use a small set of preset rotation matrices optimized for a selec-
tion of preset illuminants, DCRaw instead takes a very computationally simple approach that uses
only a single rotation matrix optimized for D65 scene illumination, R ≈ RD65 . This is achieved
using a characterization matrix T D65 optimized for D65 illumination, which means that the D−1
matrix contained within R is replaced by D−1 D65 and the CATAW→D65 matrix is not required:

RD65 ¼ M−1
EQ-TARGET;temp:intralink-;e048;116;622
−1
sRGB T D65 DD65 : (48)

The diagonal WB matrix DD65 contains raw channel multipliers appropriate for D65 illumination:
2 3 2 3
1 1
6 RðWPÞ 0 0 7 6 0 0 7
6 7 6 RðD65Þ 7
6 7 6 7
6 0 1 7 6 1
0 7
DD65 ¼ 6 0 7 ¼6 0 7: (49)
6 GðWPÞ 7 6 GðD65Þ 7
EQ-TARGET;temp:intralink-;e049;116;580

6 7 6 7
4 1 5 4 0 0
1 5
0 0 BðD65Þ
BðWPÞ D65

The overall transformation from the camera raw space to the linear form of sRGB is defined by
2 3 2 3
RL R
6 7 6 7
4 GL 5 ≈ RD65 D4 G 5 ; (50) EQ-TARGET;temp:intralink-;e050;116;471

BL D65 B scene

which can be more explicitly written as

2 3
RðD65Þ
2 3 6 0 0 72 3
RL 6 RðAWÞ 7 R
6 GðD65Þ 7
6 7 6 76 7
4 GL 5 ≈ M−1 T D65 6 0 0 74 G 5 : (51)
sRGB
6 GðAWÞ 7
EQ-TARGET;temp:intralink-;e051;116;399

BL 6 7 B
D65 4 BðD65Þ 5 scene
0 0
BðAWÞ

Consequently, all chromatic adaptation is performed using raw channel multipliers. Notice that the
WB matrix D appropriate for the scene illumination estimate is always applied to the raw data in
Eq. (50), so WB is always correctly achieved in principle.
Although the color transformation matrix T D65 is optimized for D65 scene illumination,
applying the color rotation matrix RD65 to transform from the camera raw space to sRGB is
valid for any scene illumination CCT since color rotation matrices vary very slowly as a function
of CCT, as evident from Fig. 8. However, RD65 is the optimum choice for D65 scene illumi-
nation, so a drawback of this simplified approach is that the overall color transformation loses
some accuracy when the scene illumination differs significantly from D65.

6.1 Example: Olympus E-M1

DCRaw uses color rotation matrices obtained via Eq. (48), so a T D65 characterization matrix is
required for a given camera model. For this purpose, DCRaw uses the Adobe “ColorMatrix2”
matrices from the Adobe® DNG converter.32
Due to highlight recovery logic requirements, the Adobe matrices map in the opposite direc-
tion to the conventional characterization matrices defined in Sec. 2.4, and therefore
 −1
1
T D65 ¼ ColorMatrix2 ; (52)
c
EQ-TARGET;temp:intralink-;e052;116;97

Optical Engineering 110801-24 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

where c is a normalization constant. For the Olympus E-M1 digital camera, the DCRaw source
code stores the ColorMatrix2 entries in the following manner:

7687; −1984; −606; −4327; 11928; 2721; −1381; 2339; 6452:

EQ-TARGET;temp:intralink-;sec6.1;116;711

Dividing by 10,000 and rearranging in matrix form yields

2 3
0.7687 −0.1984 −0.0606
6 7
ColorMatrix2 ¼ 4 −0.4327 1.1928 EQ-TARGET;temp:intralink-;e053;116;668 0.2721 5: (53)
−0.1381 0.2339 0.6452

Recall from Sec. 2.5 that characterization matrices are typically normalized so that the WP of the
characterization illuminant maps to raw values such that the maximum value (typically the green
channel) just reaches saturation when a 100% neutral diffuse reflector is photographed under the
characterization illuminant. Although the ColorMatrix2 matrices are optimized for CIE illuminant
D65, they are by default normalized according to the WP of CIE illuminant D50 rather than D65:
2 3 2 3
RðWPÞ XðWPÞ ¼ 0.9642
6 7 6 7
4 GðWPÞ 5 ¼ ColorMatrix24 YðWPÞ ¼ 1.0000 5 ; (54) EQ-TARGET;temp:intralink-;e054;116;546

BðWPÞ D50 ZðWPÞ ¼ 0.8249 D50

where maxfRðWPÞ; GðWPÞ; BðWPÞg ¼ 1. Accordingly, they need to be rescaled for use with
DCRaw:
2 3 2 3
RðWPÞ XðWPÞ ¼ 0.9504
6 7 1 6 7
4 GðWPÞ 5 ¼ ColorMatrix24 YðWPÞ ¼ 1.0000 5 ; (55)
c
EQ-TARGET;temp:intralink-;e055;116;460

BðWPÞ D65 ZðWPÞ ¼ 1.0888 D65

where maxfRðWPÞ; GðWPÞ; BðWPÞg ¼ 1. In the present example, it is found that c ¼ 1.0778, so
2 3
0.7133 −0.1841 −0.0562
6 7
T −1
D65 ¼ 4 −0.4015 1.1068 0.2525 5: (56) EQ-TARGET;temp:intralink-;e056;116;386

−0.1281 0.2170 0.5987

By considering the unit vector in the sRGB color space, the above matrix can be used to obtain the
raw tristimulus values for the D65 illumination WP:
2 3 2 3
RðWPÞ ¼ 0.4325 RL ¼ 1
6 7 6 7
4 GðWPÞ ¼ 1.0000 5 ¼ T −1
D65 M sRGB 4 GL ¼ 1 5 ; (57) EQ-TARGET;temp:intralink-;e057;116;301

BðWPÞ ¼ 0.7471 D65 BL ¼ 1 D65

where MsRGB converts from the linear form of sRGB to CIE XYZ. Now Eq. (49) can be used to
extract the raw channel multipliers for scene illumination with a D65 WP:
2 3
2.3117 0 0
6 7
DD65 ¼ 4 0 1 0 5: (58) EQ-TARGET;temp:intralink-;e058;116;217

0 0 1.3385

Finally, the color rotation matrix can be calculated from Eq. (48):
2 3
1.7901 −0.6689 −0.1212
6 7
RD65 ¼ 4 −0.2167 1.7521 −0.5354 5: EQ-TARGET;temp:intralink-;e059;116;145 (59)
0.0543 −0.5582 1.5039

Each row sums to unity as required. The form of the matrix is similar to the in-camera Olympus
matrices listed in Table 1. For comparison purposes, the appropriate listed matrix is the one valid

Optical Engineering 110801-25 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

for scene illuminant CCTs ranging from 5800 to 6600 K. Some numerical differences are expected
since D65 illumination has a Duv ¼ 0.0032 color tint. Other numerical differences are likely due to
differing characterization practices between Olympus and Adobe. Additionally, Adobe uses HSV
(hue, saturation, and value) tables to emulate the final color rendering of the in-camera JPEG
processing engine.

6.2 DCRaw and MATLAB

As shown in Table 3, DCRaw includes many commands that are useful for scientific research.
However, it is important to note that the RGB output color space options use color rotation
matrices rather than the concatenation of the raw to CIE XYZ and CIE XYZ to RGB matrices.
Since color rotation matrices include an inbuilt CAT, these options will only achieve the expected
result in combination with the correct raw channel multipliers. For example, setting each raw
channel multiplier to unity will not prevent some partial chromatic adaptation from being per-
formed if the sRGB output is selected since the DCRaw color rotation matrix incorporates the
D−1D65 matrix, which is a type of CATRW→D65 .
A robust way to use DCRaw for scientific research is via the “dcraw -v -D -4 -T filename”
command, which provides linear 16-bit TIFF output in the raw color space without white balancing,
demosaicing, or color conversion. Subsequent processing can be performed after importing the TIFF
file into MATLAB® using the conventional “imread” command. Reference 49 provides a processing
tutorial. The color chart photos in the present article were produced using this methodology.
For example, after importing the file into MATLAB via the above commands, a viewable
output image in the sRGB color space without any white balancing can be obtained by applying
the appropriate characterization matrix T after the color demosaic, followed by direct application
of the standard CIE XYZ to sRGB matrix, M −1 sRGB .

7 Adobe DNG
The Adobe® DNG is an open-source raw file format developed by Adobe.32,50 The freeware
DNG Converter can be used to convert any raw file into the DNG format.
Although the DNG converter does not aim to produce a viewable output image, it does perform
a color conversion from the camera raw space into the profile connection space (PCS) based on the
CIE XYZ color space with a D50 illumination WP.40 (This is not the actual reference white of CIE
XYZ, which is CIE illuminant E.) Consequently, the color processing model used by the DNG
converter must provide appropriate characterization matrices along with a strategy for achieving
correct WB in relation to the PCS. When processing DNG files, raw converters can straightfor-
wardly map from the PCS to any chosen output-referred color space and associated reference white.
The DNG specification provides two different color processing models referred to here as
method 1 and method 2. Method 1 adopts a similar strategy as smartphones and commercial raw
converters, the difference being that the data remain in the PCS. By using raw channel multi-
pliers, method 2 adopts a similar strategy as traditional digital cameras. However, the multipliers
are applied in conjunction with a so-called forward matrix instead of a rotation matrix since the
mapping is to the PCS rather than to an output-referred RGB color space.

7.1 Method 1: Color Matrices

The transformation from the camera raw space to the PCS is defined as follows:
2 3 2 3
X R
6 7 −1 6 7
Y
4 5 ¼ CAT AW→D50 C 4G5 :
EQ-TARGET;temp:intralink-;e060;116;160 (60)
Z D50 B scene

Here C is an Adobe color matrix optimized for the scene AW. Due to highlight recovery logic
requirements, Adobe color matrices map in the direction from the CIE XYZ color space to the
camera raw space:

Optical Engineering 110801-26 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

2 3 2 3
R X
6 7 6 7
4G5
EQ-TARGET;temp:intralink-;e061;116;735 ¼ C4 Y 5 : (61)
B scene Z scene

This is the opposite direction to a conventional characterization matrix T, so

C ∝ T −1 :
EQ-TARGET;temp:intralink-;e062;116;670 (62)

After the C inverse maps from the camera raw space to CIE XYZ, the linear Bradford CAT is
applied to adapt the AW to the WP of the PCS.
Analogous to the issue described in Sec. 4 for smartphones, the implementation of Eq. (60) is
complicated by the fact that C should be optimized for the scene AW. The optimized C matrix is
determined by interpolating between two color matrices labeled ColorMatrix1 and
ColorMatrix2, where ColorMatrix1 should be obtained from a characterization performed using
a low-CCT illuminant such as CIE illuminant A and ColorMatrix2 should be obtained from a
characterization performed using a high-CCT illuminant such as CIE illuminant D65.32
The optimized matrix C is calculated by interpolating between ColorMatrix1 and
ColorMatrix2 based on the scene illumination CCT estimate denoted by CCT(AW), together
with the CCTs associated with each of the two characterization illuminants denoted by
CCT1 and CCT2 , respectively, with CCT1 < CCT2 .

7.2 Color Matrix Normalization

Recall from Sec. 2.5 that characterization matrices are typically normalized so that the charac-
terization illuminant WP in the CIE XYZ color space just saturates the raw data in the camera
raw space and that the green raw channel is typically the first to saturate. However, in the present
context, the Adobe ColorMatrix1 and ColorMatrix2 matrices require a common normalization
that is convenient for performing the interpolation. Analogous to Sec. 4.1, the AW is not known
in terms of the CIE XYZ color space prior to the interpolation. Instead, ColorMatrix1 and
ColorMatrix2 are by default normalized so that the WP of the PCS just saturates the raw data:
2 3 2 3
RðWPÞ XðWPÞ ¼ 0.9642
EQ-TARGET;temp:intralink-;e063;116;374

6 7 6 7
4 GðWPÞ 5 ¼ ColorMatrix14 YðWPÞ ¼ 1.0000 5 ;
BðWPÞ D50 ZðWPÞ ¼ 0.8249 D50
2 3 2 3
RðWPÞ XðWPÞ ¼ 0.9642
6 7 6 7
4 GðWPÞ 5 ¼ ColorMatrix24 YðWPÞ ¼ 1.0000 5 ; (63)
BðWPÞ D50 ZðWPÞ ¼ 0.8249 D50

where maxfRðWPÞ; GðWPÞ; BðWPÞg ¼ 1. For example, the default ColorMatrix1 and
ColorMatrix2 for the Olympus E-M1 camera are, respectively, normalized as follows:
2 3 2 32 3
RðWPÞ ¼ 0.5471 −0.5742 XðWPÞ ¼ 0.9642
EQ-TARGET;temp:intralink-;e064;116;239

1.1528 0.0118
6 7 6 76 7
4 GðWPÞ ¼ 1.0000 5 ¼ 4 −0.2453 1.0205 0.2619 54 YðWPÞ ¼ 1.0000 5 ;
BðWPÞ ¼ 0.6560 D50 −0.0751 0.189 0.6539 ZðWPÞ ¼ 0.8249 D50
2 3 2 32 3
RðWPÞ ¼ 0.4928 0.7687 −0.1984 −0.0606 XðWPÞ ¼ 0.9642
6 7 6 76 7
4 GðWPÞ ¼ 1.0000 5 ¼ 4 −0.4327 1.1928 0.2721 54 YðWPÞ ¼ 1.0000 5 : (64)
BðWPÞ ¼ 0.6330 D50 −0.1381 0.2339 0.6452 ZðWPÞ ¼ 0.8249 D50

The interpolated C initially inherits this normalization. However, after C has been deter-
mined, the CIE XYZ values for the AW will be known. Consequently, the Adobe DNG
SDK source code later re-normalizes Eq. (60) so that the AW in the camera raw space maps to
the WP of the PCS when the raw data just saturates:

Optical Engineering 110801-27 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

2 3 2 3
XðWPÞ ¼ 0.9641 RðAWÞ
6 7 6 7
4 YðWPÞ ¼ 1.0000 5
EQ-TARGET;temp:intralink-;e065;116;735 ¼ CATAW→D50 C−1 4 GðAWÞ 5 ; (65)
ZðWPÞ ¼ 0.8249 D50 BðAWÞ scene

where maxfRðWPÞ; GðWPÞ; BðWPÞg ¼ 1. This is equivalent to re-normalizing C as follows:

2 3 2 3
RðAWÞ XðAWÞ
6 7 6 7
4 GðAWÞ 5 ¼ C4 YðAWÞ 5 ; (66) EQ-TARGET;temp:intralink-;e066;116;670

BðAWÞ scene ZðAWÞ scene

where YðAWÞ ¼ 1 and maxfRðWPÞ; GðWPÞ; BðWPÞg ¼ 1.

7.3 Linear Interpolation Based on Inverse CCT

The method 1 interpolation algorithm is the same as that described in Sec. 4.1, except that
ColorMatrix1, ColorMatrix2, and C replace T 1 , T 2 , and T, respectively. Furthermore, the
Adobe DNG specification requires the interpolation method to be linear interpolation based upon
inverse CCT.32
Again, the interpolation itself is complicated by the fact that the AW is typically calculated by
the camera in terms of raw values RðAWÞ, GðAWÞ, and BðAWÞ, but the corresponding
CCT(AW) requires knowledge of the ðx; yÞ chromaticity coordinates. This means converting to
CIE XYZ via a matrix transformation C that itself depends upon the unknown CCT(AW), which
can be solved using a self-consistent iteration procedure.

1. Make a guess for the AW chromaticity coordinates, ðxðAWÞ; yðAWÞÞ. For example, the
chromaticity coordinates corresponding to one of the characterization illuminants could
be used.
2. Find the CCT value CCT(AW) that corresponds to the chromaticity coordinates
ðxðAWÞ; yðAWÞÞ using one of the methods listed in step 2 of Sec. 4.1.
3. Perform a linear interpolation:

C ¼ α ColorMatrix1 þ ð1 − αÞ ColorMatrix2;
EQ-TARGET;temp:intralink-;e067;116;367 (67)

where α is the CCT-dependent weighting that depends upon inverse-CCT:

ðCCTðAWÞÞ−1 − ðCCT2 Þ−1

α¼ : (68)
ðCCT1 Þ−1 − ðCCT2 Þ−1
EQ-TARGET;temp:intralink-;e068;116;324

These weights (denoted by g and 1 − g in the Adobe DNG SDK source code) are
illustrated in Fig. 11 for a pair of example CCT1 and CCT2 values. The interpolation
is valid for

Fig. 11 Linear interpolation weighting factors α and 1 − α-based on inverse CCT with CCT1 ¼
2855 K and CCT2 ¼ 6504 K.

Optical Engineering 110801-28 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

CCTð1Þ ≤ CCTðAWÞ ≤ CCTð2Þ:

EQ-TARGET;temp:intralink-;sec7.3;116;735

If CCTðAWÞ < CCT1 , then C should be set equal to ColorMatrix1, and if

CCTðAWÞ > CCT2 , then C should be set equal to ColorMatrix2.
4. Use C to transform the AW from the camera raw space to CIE XYZ:
2 3 2 3
XðAWÞ RðAWÞ
6 7 6 7
4 YðAWÞ 5 ¼ C−1 4 GðAWÞ 5 : (69)
EQ-TARGET;temp:intralink-;e069;116;675

ZðAWÞ scene BðAWÞ scene

This yields a new guess for ðxðAWÞ; yðAWÞÞ.

5. Repeat the procedure starting from step 2 until ðxðAWÞ; yðAWÞÞ, CCT(AW), and C all
converge to a stable solution.
6. Normalize the color conversion according to Eq. (65).
Figure 12 illustrates the results of inverse-CCT based linear interpolation using the Adobe color
matrices defined by Eq. (64) for the Olympus E-M1 camera. Note that the ColorMatrix2 is the
same as that defined by Eq. (53), which was extracted from the DCRaw source code.
Since C maps in the direction from the CIE XYZ color space to the camera raw space, the
inverse of the interpolated C can be compared to a conventional characterization matrix T at a
given illuminant CCT. Figure 13 shows the inverse of the interpolated C plotted as a function of
CCT, and this figure can be compared with Fig. 5, which shows conventional characterization
matrices for the same camera optimized for a selection of CCTs. Although the two plots use
different normalizations since the characterization matrices are normalized according to their
characterization illuminant WP rather than the WP of the PCS, the variation with respect to
CCT is similar. However, it is evident that the interpolated C loses accuracy for CCTs below
CCT1 .

7.4 Method 2: Forward Matrices

Consider the transformation from the camera raw space to the PCS defined by Eq. (60):

Fig. 12 Optimized color matrix C plotted as a function of CCT and obtained via inverse-CCT-
based linear interpolation of the Adobe ColorMatrix1 (illuminant A, CCT2 ¼ 2855 K) and
ColorMatrix2 (illuminant D65, CCT2 ¼ 6504 K) color conversion matrices for the Olympus E-
M1 camera.

Optical Engineering 110801-29 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

Fig. 13 Inverse of the interpolated color matrix C plotted in Fig. 12.

2 3 2 3
X R
6 7 6 7
EQ-TARGET;temp:intralink-;sec7.4;116;489

4Y 5 ¼ CATAW→D50 C−1 4 G 5 ;
Z D50 B scene

where C is the Adobe color matrix optimized for the scene AW. Method 2 reformulates the above
transformation in the following manner:
2 3 2 3
X R
6 7 6 7
4Y 5 ¼ FD4 G 5 : (70) EQ-TARGET;temp:intralink-;e070;116;411

Z D50 B scene

The color conversion can be decomposed into two steps.

1. Analogous to the color conversion strategy of traditional digital cameras described in

Sec. 5, the diagonal matrix D defined by Eq. (38) contains raw channel multipliers appro-
priate for the AW, i.e., the estimated scene illumination WP estimate:
2 1 3
RðAWÞ 0 0
6 7
D¼6 4 0
1
GðAWÞ 0 7 5 : EQ-TARGET;temp:intralink-;sec7.4;116;293

1
0 0 BðAWÞ scene

In particular, the raw channel multipliers serve to chromatically adapt the AW to the refer-
ence white of the camera raw space:
2 3 2 3
R¼1 RðAWÞ
6 7 6 7
4 G ¼ 1 5 ¼ D4 GðAWÞ 5 : (71) EQ-TARGET;temp:intralink-;e071;116;200

B¼1 BðAWÞ scene

Note that the Adobe DNG specification also accommodates raw channel multipliers
applied in the analog domain.32 However, the latest digital cameras utilize ADCs with
a relatively high bit depth of order 12 or 14 and consequently elect to apply raw channel
multipliers in the digital domain.
2. The forward matrix F is a type of characterization matrix that maps from the camera raw
space to the PCS and is optimized for the scene illumination. Since the PCS is based on the

Optical Engineering 110801-30 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

Fig. 14 Optimized forward matrix F plotted as a function of CCT and obtained via inverse-CCT-
based linear interpolation of the Adobe ForwardMatrix1 (illuminant A, CCT2 ¼ 2855 K) and
Forward Matrix2 (illuminant D65, CCT2 ¼ 6504 K) matrices for the Olympus E-M1 camera.
Evidently, the elements of the optimized forward matrix F vary very slowly and in a stable manner
as a function of CCT, analogous to the color rotation matrix elements illustrated in Fig. 8.

CIE XYZ color space with a D50 illumination WP, the forward matrix F includes a built-
in CAT as it must also adapt the reference white of the camera raw space to the WP of D50
illumination:
2 3 2 3
XðWPÞ ¼ 0.9642 R¼1
6 7 6 7
4 YðWPÞ ¼ 1.0000 5 ¼ F4 G ¼ 1 5: (72)
EQ-TARGET;temp:intralink-;e072;116;400

ZðWPÞ ¼ 0.8249 D50 B¼1

Since the forward matrix F should be optimized for the scene AW, it is in practice determined by
interpolating between two forward matrices analogous to the interpolation approach used by
method 1. The Adobe DNG specification provides tags for two forward matrices labeled
ForwardMatrix1 and ForwardMatrix2, which should again be obtained from characterizations
performed using a low-CCT illuminant and a high-CCT illuminant, respectively. The same inter-
polation method described in the previous section should be used, with ForwardMatrix1,
ForwardMatrix2, and F replacing ColorMatrix1, ColorMatrix2, and C, respectively,

F ¼ α ForwardMatrix1 þ ð1 − αÞ ForwardMatrix2:
EQ-TARGET;temp:intralink-;e073;116;252 (73)

Figure 14 shows the optimized forward matrix interpolated from ForwardMatrix1 and
ForwardMatrix2 and expressed as a function of CCT for the Olympus E-M1 camera.

7.5 Forward Matrix Specification

By comparing Eqs. (60) and (70), F is algebraically related to the color matrix C as follows:

EQ-TARGET;temp:intralink-;e074;116;148 F ¼ CATAW→D50 C−1 D−1 : (74)

Since F is interpolated from ForwardMatrix1 and ForwardMatrix2 in practice, these are

defined as

Optical Engineering 110801-31 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

ForwardMatrix1 ¼ CATAW→D50 ColorMatrix1−1 D−1

EQ-TARGET;temp:intralink-;e075;116;735

ForwardMatrix2 ¼ CATAW→D50 ColorMatrix2−1 D−1 : (75)

According to Eq. (72), the optimized forward matrix F is by definition normalized such that the
unit vector in the camera raw space maps to the D50 WP of the PCS.32 This means that
ForwardMatrix1 and ForwardMatrix2 must also be normalized in this manner. For example,
the default ForwardMatrix1 and ForwardMatrix2 for the Olympus E-M1 camera are, respec-
tively, normalized as follows:
2 3 2 32 3
XðWPÞ ¼ 0.9643 R¼1
EQ-TARGET;temp:intralink-;e076;116;634

0.4734 0.3618 0.1291

6 7 6 76 7
4 YðWPÞ ¼ 0.9999 5 ¼ 4 0.2765 0.6827 0.0407 54 G ¼ 1 5;
ZðWPÞ ¼ 0.8251 D50 0.2116 0.0006 0.6129 B¼1
2 3 2 32 3
XðWPÞ ¼ 0.9643 0.4633 0.3244 0.1766 R¼1
6 7 6 76 7
4 YðWPÞ ¼ 1.0000 5 ¼ 4 0.2779 0.6661 0.0560 54 G ¼ 1 5: (76)
ZðWPÞ ¼ 0.8252 D50 0.1722 0.0033 0.6497 B¼1

The official D50 WP of the PCS is actually X ¼ 0.9642, Y ¼ 1.0000, and Z ¼ 0.8249,40 which
is a 16-bit fractional approximation of the true D50 WP defined by X ¼ 0.9642, Y ¼ 1.0000,
and Z ¼ 0.8251.

8 Conclusions
The opening section of this paper showed how the DCRaw open-source raw converter can be
used to directly characterize a camera without needing to determine and invert the OECF and
illustrated how characterization matrices are normalized in practice. As a consequence of camera
metameric error, the camera raw space for a typical camera was shown to be warped away from
the triangular shape accessible to additive linear combinations of three fixed primaries on the xy
chromaticity diagram, and the available gamut was shown to be dependent on the characteri-
zation illuminant. It was also shown that the reference white of a typical camera raw space has a
strong magenta color tint.
Subsequently, this paper investigated and compared the type of color conversion strategies
used by smartphone cameras and commercial raw converters, the image-processing engines of
traditional digital cameras, DCRaw, and the Adobe DNG converter.
Smartphones and raw conversion software applications typically adopt the type of color con-
version strategy familiar in color science. This involves the application of a characterization
matrix T to transform from the camera raw space to the CIE XYZ color space, a CAT to chro-
matically adapt the estimated WP of the scene illumination to the reference white of an output-
referred color space (such as D65 for sRGB), and finally a transformation from CIE XYZ to the
linear form of the chosen output-referred color space. Since the optimized characterization
matrix is CCT-dependent unless the Luther-Ives condition is satisfied, an optimized matrix can
be determined by interpolating between two preset characterization matrices, one optimized for a
low-CCT illuminant and the other optimized for a high-CCT illuminant. Simpler solutions
include using a fixed characterization matrix optimized for representative scene illumination.
For traditional digital cameras, this paper showed how the overall color conversion is typ-
ically reformulated in terms of raw channel multipliers D along with a set of color rotation
matrices R. The raw channel multipliers act as a type of CAT by chromatically adapting the
scene illumination WP estimate to the reference white of the camera raw space. Since the rows
of a color rotation matrix each sum to unity, the rotation matrix subsequently transforms from the
camera raw space directly to the chosen output-referred RGB color space and at the same time
chromatically adapts the camera raw space reference white to that of the output-referred color
space. It was shown that the variation of the elements of a color rotation matrix with respect to
CCT is very small, so only a small selection of preset rotation matrices are needed, each opti-
mized for a specified preset illuminant. This enables raw channel multipliers appropriate for the
scene illumination WP estimate to be applied in combination with the preset rotation matrix

Optical Engineering 110801-32 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

associated with the closest-matching WP. The primary advantages of the reformulation are that
interpolation is not required and the method can be efficiently implemented on fixed-point archi-
tecture. Furthermore, image quality can be improved by applying the raw channel multipliers
prior to the color demosaic.
It was shown that DCRaw uses a similar model as traditional digital cameras, except that only
a single color rotation matrix is used for each camera, specifically a matrix optimized for D65
illumination, RD65 . Although the overall color conversion loses some accuracy when the scene
illumination differs significantly from D65, an advantage of decoupling the raw channel multi-
pliers from the characterization information represented by the color rotation matrix is that WB
can be correctly achieved for any type of scene illumination provided raw channel multipliers
appropriate for the scene illumination are applied. It was shown that the rotation matrices used by
DCRaw can be derived from the inverses of the “ColorMatrix2” color characterization matrices
used by the Adobe DNG converter.
The Adobe DNG converter maps the camera raw space and scene illumination WP estimate
to an intermediate stage in the overall color conversion, namely the PCS based on the CIE XYZ
color space with a D50 WP. Method 1 defines the approach that is also used in commercial raw
converters and advanced smartphones. A color matrix C optimized for the scene illumination is
obtained by interpolating between the “ColorMatrix1” low-CCT and “ColorMatrix2” high-CCT
preset matrices. Due to highlight recovery logic requirements, these color matrices map in the
opposite direction to conventional characterization matrices. Furthermore, the ColorMatrix1 and
ColorMatrix2 matrices are initially normalized according to the WP of the PCS rather than their
corresponding characterization illuminants. Since the Adobe color matrices are freely available,
their appropriately normalized inverses can serve as useful high-quality characterization matrices
when camera characterization equipment is unavailable.
Method 2 offered by the Adobe DNG converter uses raw channel multipliers in a similar
manner as traditional digital cameras. However, these are applied in combination with a so-called
forward matrix rather than a rotation matrix since the Adobe DNG converter does not directly
map to an output-referred RGB color space, so the forward matrix rows do not each sum to unity.
Although the optimized forward matrix is determined by interpolating the “ForwardMatrix1”
and “ForwardMatrix2” preset matrices, the variation of the optimized forward matrix with
respect to CCT is very small, analogous to a rotation matrix.

9 Appendix: Raw Data Model

Consider the raw values expressed as an integration over the spectral passband of the camera
according to Eq. (5):
Z λ Z λ Z λ
2 2 2
R¼k R1 ðλÞẼe;λ dλ; G ¼ k
EQ-TARGET;temp:intralink-;sec9;116;297R2 ðλÞẼe;λ dλ; B ¼ k R3 ðλÞẼe;λ dλ:
λ1 λ1 λ1

Although E~ e;λ can be regarded as the average spectral irradiance at a photosite, it is more pre-
cisely described as the spectral irradiance convolved with the camera system point-spread func-
tion (PSF) hðx; y; λÞ and sampled at positional coordinates ðx; yÞ on the sensor plane:
 
x y
Ẽe;λ ðx; yÞ ¼ ½Ee;λ;ideal ðx; yÞ  hðx; y; λÞcomb ; ; (77)
px py
EQ-TARGET;temp:intralink-;e077;116;214

where px and py are the pixel pitches in the horizontal and vertical directions. A noise model can
also be included.28,51 The quantity denoted by Ee;λ;ideal ðx; yÞ is the ideal spectral irradiance at the
sensor plane that would theoretically be obtained in the absence of the system PSF:
    
π x y 1 x y
Eλ;ideal ðx; yÞ ¼ Le;λ ; T cos φ
4
; ; (78)
m m N 2w m m
EQ-TARGET;temp:intralink-;e078;116;131

where Le;λ is the corresponding scene spectral radiance, m is the system magnification, N w is the
working f-number of the lens, T is the lens transmittance factor, and φ is the object-space angle

Optical Engineering 110801-33 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

between the optical axis and the indicated scene coordinates. If the vignetting profile of the lens
is known, the cosine fourth term can be replaced by the relative illumination factor, which is an
image-space function describing the real vignetting profile.52
The constant k that appears in Eq. (5) places an upper bound on the magnitude of the raw
values. It can be shown28 that k is given by
Ap t
k¼ ; (79)
gi e
EQ-TARGET;temp:intralink-;e079;116;675

where t is the exposure duration and gi is the conversion factor between electron counts and raw
values for mosaic i, expressed using e− ∕DN units.53,54 The conversion factor is inversely propor-
tional to the ISO gain GISO , which is the analog gain setting of the programmable gain amplifier
situated upstream from the ADC:
U ne;i;FWC
gi ¼ ; U¼ : (80)
GISO;i nDN;i;clip
EQ-TARGET;temp:intralink-;e080;116;584

Here U is the unity gain, which is the gain setting at which gi ¼ 1. Full-well capacity is denoted
by ne;i;FWC, and nDN;i;clip is the raw clipping point, which is the maximum available raw level.
This value is not necessarily as high as the maximum raw level provided by the ADC given its
bit-depth M, which is 2M − 1 DN, particularly if the camera includes a bias offset that is sub-
tracted before the raw data is written.28,53
The least analog amplification is defined by GISO ¼ 1, which corresponds to the base ISO
gain.28,51 The numerical values of the corresponding camera ISO settings S are defined using the
JPEG output rather than the raw data.55,56 These user values also take into account digital gain
applied via the JPEG tone curve. When comparing raw output from cameras based on different
sensor formats, equivalent rather than the same exposure settings should be used when
possible.57
As noted in Sec. 2.2, the actual raw values obtained in practice are quantized values modeled
by taking the integer part of Eq. (5), and it is useful to subsequently normalize them to the range
[0,1] by dividing Eq. (5) by the raw clipping point.

References
1. E. Machishima, “Color CCD imager having a system for preventing reproducibility deg-
radation of chromatic colors,” U.S. Patent 5,253,047 (1993).
2. K. E. Spaulding, R. M. Vogel, and J. R. Szczepanski, “Method and apparatus for color-
correcting multi-channel signals of a digital camera,” U.S. Patent 5,805,213 (1998).
3. J. Von Kries, “Chromatic adaptation,” in Sources of Color Science, D. L. MacAdam, Ed.,
pp. 145–148, MIT Press, Cambridge (1970). [Originally published in Festschrift der
Albrecht-Ludwigs-Universitat (1902)].
4. H. E. Ives, “The relation between the color of the illuminant and the color of the illuminated
object,” Trans. Illum. Eng. Soc. 7, 62–72 (1912). [Reprinted in Color Res. Appl. 20, pp. 70–
75 (1995)].
5. J. Nakamura, “Basics of image sensors,” Chapter 3 in Image Sensors and Signal Process-
ing for Digital Still Cameras, J. Nakamura, Ed., pp. 53–93, CRC Press, Taylor & Francis
Group, Boca Raton, Florida (2006).
6. J. Jiang et al., “What is the space of spectral sensitivity functions for digital color cameras?”
in IEEE Workshop Appl. Comput. Vision, pp. 168–179 (2013).
7. R. Luther, “Aus dem Gebiet der Farbreizmetrik (On color stimulus metrics),” Z. Tech. Phys.
12, 540–558 (1927).
8. P.-C. Hung, “Sensitivity metamerism index for digital still camera,” Proc. SPIE 4922, 1–14
(2002).
9. P.-C. Hung, “Color theory and its application to digital still cameras,” Chapter 7 in Image
Sensors and Signal Processing for Digital Still Cameras, J. Nakamura, Ed., pp. 205–222,
CRC Press, Taylor & Francis Group, Boca Raton, Florida (2006).

Optical Engineering 110801-34 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

10. P. M. Hubel et al., “Matrix calculations for digital photography,” in Proc., IS&T Fifth Color
Imaging Conf., pp. 105–111 (1997).
11. J. Holm, I. Tastl, and S. Hordley, “Evaluation of DSC (digital still camera) scene analysis
error metrics—Part 1,” in Proc., IS&T/SID Eighth Color Imaging Conf., pp. 279–287 (2000).
12. R. M. Vogel, “Digital imaging device optimized for color performance,” U.S. Patent
5,668,596 (1997).
13. G. D. Finlayson and Y. Zhu, “Finding a colour filter to make a camera colorimetric by
optimisation,” Lect. Notes Comput. Sci. 11418, 53–62 (2019).
14. D. Alleysson, S. Susstrunk, and J. Hérault, “Linear demosaicing inspired by the human
visual system,” IEEE Trans. Image Process. 14(4), 439–449 (2005).
15. International Organization for Standardization, “Graphic technology and photography—
colour target and procedures for the colour characterisation of digital still cameras
(DCSs),” ISO 17321-1:2012 (2012).
16. International Electrotechnical Commission, “Multimedia systems and equipment—colour
measurement and management—Part 2-1: colour management—default RGB colour
space—sRGB,” IEC 61966-2-1:1999 (1999).
17. Adobe Systems Incorporated, “Adobe® RGB (1998) color image encoding,” Version 2005-
05 (2005).
18. International Organization for Standardization, “Photography and graphic technology—
extended colour encodings for digital image storage, manipulation and interchange—
Part 2: reference output medium metric RGB colour image encoding (ROMM RGB),”
ISO 22028-2:2013 (2013).
19. J. Holm, “Capture color analysis gamuts,” in Proc. 14th Color and Imaging Conf., IS&T,
pp. 108–113 (2006).
20. B. E. Bayer, “Color imaging array,” U.S. Patent 3,971,065 (1976).
21. D. Coffin, Private Communication (2015).
22. Y. Ohno, “Practical use and calculation of CCT and DUV,” LEUKOS 10(1), 47–55 (2014).
23. D. L. MacAdam, “Projective transformations of I. C. I. color specifications,” J. Opt. Soc.
Am. 27, 294 (1937).
24. CIE (Commission Internationale de l’Eclairage), Proc. 14th Session, Brussels, Vol. A, p. 36
(1959).
25. Commission Internationale de l’Eclairage, “Colorimetry,” CIE 15:2004, 3rd ed., Vienna,
(2004).
26. G. D. Finlayson and M. S. Drew, “White-point preserving color correction,” in Proc., IS&T
Fifth Color Imaging Conf., pp. 258–261 (1997).
27. International Organization for Standardization, “Photography—electronic still picture cam-
eras—methods for measuring opto-electronic conversion functions (OECFs),” ISO
14524:2009 (2009).
28. D. A. Rowlands, Physics of Digital Photography, IOP Publishing Ltd., Bristol (2017).
29. D. Varghese, R. Wanat, and R. K. Mantiuk, “Colorimetric calibration of high dynamic range
images with a ColorChecker chart,” in 2nd Int. Conf. and SME Workshop HDR Imaging,
A. Chalmers and K. Bouatouch, Eds., The Eurographics Association (2014).
30. J. McCann, “Do humans discount the illuminant?” Proc. SPIE 5666, 9–16 (2005).
31. International Organization for Standardization, “Photography—electronic still picture im-
aging—vocabulary,” ISO 12231:2012 (2012).
32. Adobe Systems Incorporated, “Adobe digital negative (DNG) specification,” Version 1.4
(2012).
33. G. Buchsbaum, “A spatial processor model for object color perception,” J. Franklin Inst.
310, 1–26 (1980).
34. E. H. Land and J. McCann, “Lightness and Retinex theory,” J. Opt. Soc. Am. 61(1), 1–11
(1971).
35. S. Hordley, “Scene illuminant estimation: past, present and future,” Color Res. Appl. 31, 303
(2006).
36. E. Y. Lam and G. S. K. Fung, “Automatic white balancing in digital photography,” Chapter
10 in Single-Sensor Imaging: Methods and Applications for Digital Cameras, R. Lukac,
Ed., pp. 267–294, CRC Press, Boca Raton, Florida (2009).

Optical Engineering 110801-35 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Rowlands: Color conversion matrices in digital cameras: a tutorial

37. Commission Internationale de l’Eclairage, “Fundamental chromaticity diagram with

physiological axes—Part 1,” CIE 170-1:2006, Vienna (2006).
38. M. D. Fairchild, Color Appearance Models, 3rd ed., Wiley, New York (2013).
39. K. M. Lam, “Metamerism and colour constancy,” PhD Thesis, University of Bradford
(1985).
40. International Color Consortium, “Image technology colour management—architecture, pro-
file format, and data structure,” Specification ICC.1:2010 (profile version 4.3.0.0) (2010).
41. J. Holm et al., “Color processing for digital photography,” in Color Engineering: Achieving
Device Independent Color, P. Green and L. MacDonald, Eds., pp. 179–220, John Wiley &
Sons, Ltd., Chichester (2002).
42. A. R. Robertson, “Computation of correlated color temperature and distribution tempera-
ture,” J. Opt. Soc. Am. 58, 1528 (1968).
43. Q. Xingzhong, “Formulas for computing correlated color temperature,” Color Res. Appl.
12(5), 285 (1987).
44. C. S. McCamy, “Correlated color temperature as an explicit function of chromaticity coor-
dinates,” Color Res. Appl. 17, 142 (1992).
45. J. Hernández-Andrés, R. L. Lee, and J. Romero, “Calculating correlated color temperatures
across the entire gamut of daylight and skylight chromaticities,” Appl. Opt. 38, 5703 (1999).
46. C. Li et al., “Accurate method for computing correlated color temperature,” Opt. Express
24(13), 14066 (2016).
47. K. A. Parulski, R. H. Hibbard, and J. D’Luna, “Programmable Digital Circuit for
Performing a Matrix Multiplication,” U.S. Patent No. 5,001,663 (1991).
48. “ExifTool”, application by P. Harvey, https://exiftool.org.
49. R. Sumner, “Processing RAW Images in MATLAB,” unpublished, Department of Electrical
Engineering, UC Santa Cruz (2014).
50. Adobe Systems Incorporated, “Introducing the digital negative specification: information
for manufacturers” (2004).
51. D. A. Rowlands, Field Guide to Photographic Science, SPIE Field Guides, Vol. FG46, SPIE
Press, Bellingham (2020).
52. P. Maeda, P. Catrysse, and B. Wandell, “Integrating lens design with digital camera sim-
ulation,” Proc. SPIE 5678, 48 (2005).
53. E. Martinec, “Noise, dynamic range and bit depth in digital SLRs,” unpublished (2008).
54. T. Mizoguchi, “Evaluation of image sensors,” Chapter 6 in Image Sensors and Signal
Processing for Digital Still Cameras, J. Nakamura, Ed., pp. 179–203, CRC Press,
Taylor & Francis Group, Boca Raton, Florida (2006).
55. Camera & Imaging Products Association, “Sensitivity of digital cameras,” CIPA DC-004
(2004).
56. International Organization for Standardization, “Photography—digital still cameras—deter-
mination of exposure index, ISO speed ratings, standard output sensitivity, and recom-
mended exposure index,” ISO 12232:2006 (2006).
57. D. A. Rowlands, “Equivalence theory for cross-format photographic image quality compar-
isons,” Opt. Eng. 57 (11), 110801 (2018).

D. Andrew Rowlands received his BSc degree in mathematics and physics and his PhD in
physics from the University of Warwick, UK, in 2000 and 2004, respectively. He has held
research positions at the University of Bristol, UK, Lawrence Livermore National Laboratory,
USA, Tongji University, China, and the University of Cambridge, UK, and he has authored three
books on the science of digital photography.

Optical Engineering 110801-36 November 2020 • Vol. 59(11)

Downloaded From: https://www.spiedigitallibrary.org/journals/Optical-Engineering on 24 Nov 2021

Terms of Use: https://www.spiedigitallibrary.org/terms-of-use