Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

COLOR-BASED FACE DETECTION IN THE “15 SECONDS OF FAME”

ART INSTALLATION

Franc Solina, Peter Peer, Borut Batagelj, Samo Juvan, Jure Kovač

[email protected]
University of Ljubljana, Faculty of Computer and Information Science
Tržaška 25, SI-1000 Ljubljana, Slovenia

ABSTRACT performing some kind of color surgery on them. By sep-

arating the face from the background or selecting distinct
“15 seconds of fame1 ” is an interactive art installation which
facial features (i.e. mouths, eyes, hair) he obtained image
elevates the face of a randomly selected observer for 15 sec-
areas which he highlighted or covered with paint. Warhol
onds into a “work of art”. The installation was inspired by Andy
portrayed in this fashion celebrities from arts and politics
Warhol’s statement that “In the future everybody will be famous
(i.e. Mao-Tse Toung, Marilyn Monroe, Jackie Kennedy,
for 15 minutes” as well as by the pop-art style of his works. The
etc.). Some of these images are true icons of the 20th cen-
critical part of the installation is detection of faces which en-
tury [7].
ables the cropping of faces from the wide-angle image of people
standing in front of the installation. In the paper we describe
color-based face detection in detail and many problems that had
to be solved to make it reliable in different illumination condi-
tions and scenes.

1. INTRODUCTION

Technology influenced artistic production throughout his-

tory. In the later part of the 20th century the fast advent
of computer and information technology in particular left
a strong impression in the art world [5, 22].
The Computer Vision Laboratory at the University of Lju-
bljana collaborates with the Video and New Media De-
partment at the Academy of Fine Arts in Ljubljana by
Figure 1: LCD computer monitor dressed up like a pre-
supporting artistic creativity using Internet, teleoperation
cious painting. The round opening above the picture is for
and web cameras since 1995 [2, 4, 19]. The installation
the digital camera lens.
described in this paper was envisioned by Franc Solina al-
ready in 1996 and finally fully implemented in 2002 with The installation “15 seconds of fame” (Fig. 1) intends to
the help of his graduate students. The “15 seconds of make instant celebrities out of common people by putting
fame” installation was publicly exhibited at the 8th Inter- their portraits on the museum wall. Instead of providing
national festival of computer arts, 28 May-1 June 2002 in 15 minutes of fame as prophesied by Warhol we decided
Maribor, at interINFOS’02, 21-25 October 2002 in Lju- to shorten this time to 15 seconds to make the installation
bljana and in the Finžgar gallery, 14-30 November 2002 more dynamic. This period also limits the time necessary
in Ljubljana [1]. for computer processing of each picture. Since the indi-
The installation “15 seconds of fame” was inspired by vidual portraits, which are presented by the installation,
Andy Warhol’s celebrated statement that “In the future ev- are selected by chance out of many faces of people who
erybody will be famous for 15 minutes”[21] and his pho- are standing in front of the installation in that moment,
tography derived paintings of famous people. Warhol took the installation tries to convey that fame tends to be not
faces and persons from mass media, banal in their newspa- only short-lived, but also random. However, people can
per everydayness, and transformed them into paintings by prolong their fame by ordering their portrait over the In-
1 This work was supported by the Ministry of Education, Science and ternet, printing it on paper, framing it and putting it on the
Sports, Republic of Slovenia (Program Computer Vision 1539-506) wall.

38
Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

a) b)

c) d)

Figure 3: Stages in the process of finding faces in an image: a) downsize the resolution 2048×1536 of the original
image to 160×120 pixels, b) eliminate all pixels that can not represent a face, c) segment all the regions containing face-
like pixels using region growing, d) eliminate regions which can not represent a face using heuristic rules. Some color
transformations of the fourth face from the left are shown in Fig. 4.

computer to detect faces in it (Fig. 3b, c, and d show steps

in the face detection). The software then randomly selects
one of the faces and crops it from the larger picture (Fig. 4
top left). This processing performs the same function as a
photographer who would take from that viewpoint a por-
trait of one of the visitors using a telephoto lens.

Figure 2: A group of people in front of the installation.

1.1. How the installation works

The visible part of the “15 seconds of fame” installa-

tion consists of a LCD computer monitor which is framed
like a precious picture and hangs on the wall. The digi-
tal camera is hidden behind the frame above the computer
monitor so that only a round opening for the lens is visi-
ble (Fig. 1). Pictures of gallery visitors which are standing
in front of the installation (Fig. 2) are taken by the digi-
tal camera using a wide-angle lens setting (Fig. 3a). The
camera is connected with the hidden computer via USB
connection, so that the camera can be remotely controlled Figure 4: Some color transformations in pop-art manner
by the computer. Each digital photo is analyzed by the of the face selected in Fig. 3.

39
Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

The selected portrait is then transformed using randomly

selected color filters to automatically generate a Warhol
inspired pop-art portrait (Fig. 4). The resulting portrait
is then displayed for 15 seconds on the picture/monitor
together with a unique ID number. In the mean time, the
processing of the next portrait is taking place, so that after
fifteen seconds another pop-art portrait can be displayed.
In this fashion every fifteen seconds a new picture is taken,
a new portrait selected and processed, so that it can be
displayed in the next 15 second period.
If several people are standing in front of the installation
the software tries to select each time a different person.
Even if just a single person is present in front of the instal-
lation, the randomly selected graphic effects assure that Figure 5: Basic steps of our face detection method [17]:
the displayed portraits are never the same. If the system 1) input image, 2) all non-skin colors are eliminated, 3)
does not detect any face, the last detected face is being dis- image filtered with median filter, 4) segmented white re-
played, but with a different graphic effect in each 15 sec- gions, 5) eliminated insignificant white regions, 6) traced
onds period. Examples of such transformations of a single edges, 7) circles within candidate face regions indicating
portrait can be seen in Fig. 4. Since portraits displayed possible eye positions, 8) output image with confirmed
by the installation are saved in a database, they can be faces.
ordered by sending e-mail to [email protected]
lj.si with the corresponding portrait ID number in the
subject field. The basic idea of the method is as follows: find in the
The outline of the rest of the paper is as follows: in Sec- image all skin-like regions, which contain possible can-
tion 2 we explain the process of detecting human faces didates for an eye, then on the basis of geometric face
in images, Section 3 is on eliminating the influence of characteristics try to join two eye candidates into an eye
non-standard illumination in images, graphical transfor- pair and, finally, confirm or refuse the face candidate us-
mations to achieve pop-art effects are described in Section ing skin color information. The main steps of the method
4, Section 5 describes how the resulting portraits are dis- described in [17] is illustrated in Fig. 5. The skin color of
played, Section 6 how e-mail ordering of portraits works, fair complexion under uniform daylight illumination is de-
and finally, Section 7 concludes the article. termined with the following rules [15, 17] which describe
the skin cluster in the RGB color space

2. FINDING FACES IN COLOR IMAGES R > 95 & G > 40 & B > 20 &
max{R, G, B} − min{R, G, B} > 15 &
Automatic face detection is like most other automatic ob-
|R − G| > 15 & R > G & R > B.
ject-detection methods difficult, especially if sample vari-
ations are significant. Large sample variations in face de- We tested also other color models [15] but the best results
tection arise due to a large variety of individual face ap- were obtained with this one.
pearances and due to differences in illumination. The goal of the method was to reach maximum classifica-
There are a few distinct approaches to face detection (for tion accuracy over the learning and testing sets of images
a detailed survey see [13]). We decided to use in the in- under the following constraints: near real-time operation
stallation a color-based approach of face detection that we on a standard personal computer, plain background, uni-
developed earlier [17]. In the next Subsection 2.1 we de- form ambient natural or studio illumination, faces of fair-
scribe our original method while in Subsection 2.2 we re- complexion, which must be entirely present in the image,
veal the simplifications of this method that we had to make and faces turned away from the frontal view for at most
for the installation. 30 degrees.
During the processing the method requires some thresh-
2.1. Our original face detection method olds, which are set quite tolerantly, but they become effec-
tive in a sequence. All thresholds were defined experimen-
We developed a face detection method which consists of tally. The method was developed using different training
two distinct parts: making of face hypotheses and verifica- sets of pictures [3] and tested over an independent test-
tion of these face hypotheses [17]. This method combines ing set of two public image-databases [16, 18] with good
two approaches: color-based and feature-based approach. results [17].

40
Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

Since the original face detection algorithm is computa- ditions. Once we realized that we will not be able to
tionally demanding, we decided to develop a simpler ver- completely control the illumination in different exhibition
sion for integration in the “15 seconds of fame” instal- spaces, we decided to improve our face detection results
lation. by eliminating the influence of non-standard illumination.

2.2. The simplified face detection method 3. ELIMINATING THE INFLUENCE OF

In order to confirm a given face candidate in our instal- NON-STANDARD ILLUMINATION
lation, we modified the original face detection method as
follows. When the color picture is downloaded from the The purpose of studying methods for eliminating the in-
digital camera (we normally use resolution 2048×1536 fluence of non-standard illumination in our project is to
pixels) to the computer, the system reads the picture and improve the results of our face detection algorithm. Non-
first decreases the resolution in a pyramid manner down standard illumination are by definition those that are more
to 160×120 pixels. The system searches for face candi- or less different from daylight illumination (CIE standard
dates in the lower resolution to speed up the process, but D65) [23]. We find such illumination almost anywhere
the selected face is later cropped from the original resolu- in enclosed spaces with artificial illumination, where the
tion for final processing. The system then eliminates from installation could potentially be exhibited. There are two
the picture all pixels that do not correspond to skin color. main groups of methods for addressing this problem: color
After this color segmentation the system applies a region compensation methods and color constancy methods.
growth algorithm, which segments all face-like pixels into
candidate face regions. 3.1. Color compensation methods
Each candidate face region must pass some simple tests to
qualify as a true face region. Since the confirmation of hy- Methods in this group have low time complexity (order of
pothesis in the simplified method relies entirely on simple O(n2 )) and they do not need a preliminary learning step.
heuristic tests and not on structural information (i.e. eye This means that they are easy and straightforward to im-
pairs), we had to make them more stringent. Initially, for plement. Their effectiveness is relatively high on sets of
example, we had problems with bare arms which the sys- images with some input constraints. Illumination should
tem recognized as faces. The face candidate region must be relatively close to standard illumination. The input im-
be large enough (based on the assumption about the min- age is transformed in the way that the colors in the image
imal size of the face in the original picture), have rough are leveled in respect to some statistical quantity.
face proportions (width/height ratio of the rectangular box Grey World (GW) [11] algorithm presents simple and fast
containing the candidate face region), and have a high method for color compensation on images which are de-
enough percentage of skin pixels within this box. The re- fined in RGB color space [10]. It is based on the pre-
sults are much better, although still not perfect. However, sumption that the average surface color on the image is
this is not too annoying—now and then it still happens that achromatic. This means that the average color, which is
someone’s arm or palm becomes famous for 15 seconds. reflected from the surfaces, corresponds to the color of
The side benefit of the simplified method is that faces seen the illumination. To execute the algorithm we have to
from the profile can also be detected. calculate the averages for each channel R, G and B for
The final step of finding a face for the portrait is selecting the whole image. Averages are then transformed with the
one of the faces (randomly, but with help of some heuris- help of a linear scale factor to values that correspond to
tics, like giving bigger priority to bigger regions and re- the mean gray value of one standard illuminant.
gions that are higher in the picture), mapping the face co- Modified Grey World [9] method is very similar to basic
ordinates from the lower resolution picture to the original GW algorithm with the difference that the scale factors are
resolution picture and cropping the face out of the original calculated in a slightly different way. The average of each
picture. Fig. 3 illustrates the described process. The se- channel is calculated by counting each color only once,
lected portrait is cropped from the original digital image regardless of how many times the color appears in the im-
as a square area and resized to the resolution of 400×400 age. By doing so, we eliminate the influence of colors rep-
pixels. resented by a large number of pixels in the image on the
The databases that we used for developing and testing average color. The method is effective on images, which
of our original face detection method contained pictures do not contain a lot of different colors.
taken in daylight or under white-balanced studio illumi- White-Patch Retinex [11] method is like the Modified GW
nation. When the performance of the face detection algo- method just a special version of the basic GW method.
rithm during the first public exhibition was not fully satis- The difference lies again in the method of calculating the
factory, it was mainly due to difficult illumination con- scaling factors. Instead of the average color we use the

41
Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

maximal value of each channel in the whole image. The 3.3. Comparison of methods
Retinex method is above all suitable for dark images.
3.3.1. Color compensation methods

3.2. Color constancy methods In order to determine the influence of these algorithms
on our face detection results, some experiments were per-
Methods belonging to this group differ from the color com- formed on the set of images gathered in our lab and at
pensation methods above all in the need to integrate a pre- the first public showing of the installation. The testing
liminary learning step. They require the knowledge about set is composed of 160 images taken under four different
illumination properties and properties of the capturing de- types of illumination conditions. One subset of images (40
vices, e.g. digital cameras. The input image is then trans- images) was taken under standard daylight, in the second
formed in such a way that it reflects the state, which is in- subset (40 images) objects were illuminated by incandes-
dependent of the illumination. Thus a stable presentation cent lamps assembled into a chandelier, the third subset
of colors under different illuminations is achieved. Gen- (40 images) was taken under camera flash light conditions,
erally speaking the methods consist of two distinct steps: and the last subset of images (40 images) was taken under
scene illumination detection and standard illumination re- neon light illumination conditions. After that, one of the
construction. In the first step, the algorithm determines color compensation methods was applied and finally, face
with the help of preliminary knowledge which illumina- detection algorithm was applied to original and prepro-
tion out of the set of known illuminations is present in the cessed images.
image. In the second step, it applies the necessary trans-
Results gathered in Tab. 1 show perceivable improvement
formations to reconstruct the standard (or other wanted)
in face detection on images taken under different than stan-
illumination.
dard illumination conditions when one of the compensa-
In the Color by Correlation method [9] the preliminary tion algorithms was previously applied (see Figs. 7 and
knowledge about the illumination properties is represented 8). Grey World algorithm performed especially well since
with a set of colors, which can appear under specific illu- for the flashlight, incandescent and neon light conditions
mination, i.e. colors that are visible under specific illumi- a considerable increase in TP/Det percentage can be no-
nant. ticed.

Figure 6: The Macbeth color checker [8].

After the illumination detection based on correlation tech-

nique takes place, we need to reconstruct the image scene
under some standard illumination conditions. To perform
such reconstruction, certain transformations should be ap- Figure 7: GW performance: the upper part of the figure
plied. To calculate the transformation parameters, we need shows how face detector failed to detect faces due to con-
the information about the spectral power distribution. We fusion caused by surrounding colors. This was improved
can gain this information with the help of the Macbeth by GW preprocessing as seen in the lower part of the pic-
color checker (Fig. 6) [8]. We need two images of the ture. The image was chosen from the incandescent subset
Macbeth color checker captured under different illumina- of images.
tions. The first one should be captured under the same
illumination as the input image, which we want to recon- All these results are dependent on our skin detection tech-
struct. The second image should be captured under stan- nique used for face detection, which works in the 3D color
dard (or other wanted) illumination, which we want to use space (RGB). Some research suggests that skin detection
in the reconstruction process. in 2D color space (Y U V ) might be better since it is less

42
Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

Method None GW MGW RET None GW MGW RET

Illuminant standard incandescent
All Faces 109 95
Detected 75 70 65 76 45 57 42 43
TP 68 65 60 68 28 45 31 29
FP 7 5 5 8 17 12 11 14
FN 40 44 48 40 67 50 64 66
TP/Det 90,66 92,85 92,31 89,47 62,22 78,95 73,81 67,44
FN/All 36,70 40,37 44,04 36,70 70,53 52,63 67,37 69,47
Illuminant flashlight neon
All Faces 112 78
Detected 55 47 43 39 63 64 29 60
TP 38 39 36 32 50 54 26 49
FP 17 8 7 7 13 10 3 11
FN 74 73 76 79 28 24 52 29
TP/Det 69,09 82,98 83,72 82,05 77,77 84,37 89,66 81,66
FN/All 66,07 65,18 67,86 70,53 35,90 30,77 66,66 37,18

Table 1: Color compensation results show the number of all detections (Detected), the number of detected faces as true
positives (TP), number of false detections as false positives (FP) and number of faces missed as false negatives (FN) on
four subsets of images which represent different illumination conditions (standard, incandescent, flashlight and neon),
previously preprocessed by Grey World (GW), Modified GW (MGW), White Patch Retinex (RET) or no preprocessing
at all (None). Row All Faces shows the number of faces in particular subset of images. TP/Det shows the percentage
of true positives out of all detections and FN/All shows the percentage of false negatives out of all faces in the subset.
For the installation the first percentage is extremely important, while the second one is merely informative, since we have
consciously eliminated faces that were too small for further processing but were included in the number of all faces! Note
that TP + FN ≤ All, since the region characterized as TP can contain one or more faces but counts as only one TP.

The results also show that the performance of these tech-

niques depends very much on the type of the illumination.
Therefore a considerable amount of precaution should be
taken in decisions about the usage of these techniques. On
the other hand all of these algorithms are very effective
from the time complexity point of view and as such they
enable the possibility of performing a simple initialization
test when the scene is first placed under certain illumina-
tion. In this way we can determine which algorithm would
produce the best results under certain type of illumination.

3.3.2. Color constancy methods

Tests for the correlation method were performed in order

to determine the best merit (function) for illumination de-
Figure 8: Retinex performance: In the upper part of the tection. The testing set contained 156 images with none,
figure we see two false detections, while on the lower part one or multiple faces, taken under 5 various illumination
this problem was overcome by the Retinex algorithm. The conditions. The white subset represents the standard il-
image was taken from the flashlight subset. lumination conditions, while red, blue, green and yellow
subsets were generated by illuminating the scene with the
corresponding color light.
brightness dependent than detection in the 3D color space First, illumination detection on images was performed with
(RGB) [10]. Our experiments on the data collected with a correlation technique, then the images were reconstructed
this installation, however, do not prove the advantage of under approximation of standard illumination and finally
the 2D color space [15]. face detection algorithm was applied on images (Figs. 9

43
Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

Figure 9: Correlation performance on image from the yel-

low subset: detection results on image without prepro-
cessing (top) and after illumination reconstruction (bot-
tom).

and 10). Results are summarized in Tab. 2. Results of sub-

set white are shown as a comparable reference to other re- Figure 10: Correlation performance on images from the
sults. Columns with no method previously applied (None) green (left) and blue (right) subset. On the original image
can contain zero detections. This can occur if face detec- (top) no faces could be detected. After corrected illumi-
tion algorithm finds no skin color in an image, which can nation (middle) three faces could be detected on the left
often be the case in extreme illumination conditions. image.
Results in Tab. 2 show the positive effect of color correc-
tion on images with non-standard illumination conditions.
If we do not apply preprocessing, the face detector finds
some faces only on images from the yellow subset. After chandeliers or similar types of illumination that emit a pre-
the preprocessing step almost all faces were recovered un- vailing yellow light into the room and as a consequence a
der this illumination, while under all other illuminations prevailing yellow color in captured images. This causes
the number of recovered faces was not that high, but the the shift of a large part of image color space into the color
difference with the results gained without preprocessing is subspace, which represents skin color. An illustrative ex-
enormous. In the blue subset we see a very large number ample of this property can be observed on the white walls
of false detections caused by the mixture of incandescent of a room. Normally they are white, but under incandes-
and blue light. This mixture was necessary for enhance- cent lamp illumination they are more bright yellow than
ment of other than blue color channels (R and G) since white. And since walls can occupy large parts of an im-
blue filter was very strong. If only blue channel is present, age, it can happen that most of the image pixels are rec-
we have as much information as in achromatic pictures. ognized as skin-like pixels (see Figs. 7 and 9). This type
Incandescent light caused an interesting effect of mak- of illumination can have a serious negative influence on
ing all shadows in reconstructed pictures slightly yellow the number of false face detections (false positives). In
and as a consequence many false faces were found. Nev- case of incandescent lamp illumination we should choose
ertheless, with the exception of blue illumination these among color compensation methods described in Section
results are quite comparable with the results in Section 3.1. Based on the results of these algorithms and con-
3.3.1, where face detection was tested under close to stan- straints discussed in Section 3.3.1, we decided to use GW
dard illumination conditions. algorithm as it performs best when minor lighting devi-
ations from standard illumination are present. Although,
some form of automatic selection is taken into future con-
3.4. Method selection sideration.
The nature of illumination in some galleries can represent A totally different story can be observed in discotheques,
a real problem since it normally differs from daylight il- where illumination emits color light (e.g. blue, green,
lumination. In older rooms in particular we usually have red etc.) into the room. This shifts all scene colors to-

44
Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

Illuminant white yellow green blue red

Method None None C None C None C None C
All Faces 38 35 23 42 34
Detected 43 13 40 0 18 19 80 10 25
TP 35 7 34 0 13 0 29 0 23
FP 8 6 6 0 5 19 51 10 2
FN 2 28 1 23 10 42 13 34 11
TP/Det 81,39 53,85 85,00 0 72,22 0 36,25 0 92,00
FN/All 5,26 80,00 2,86 100,00 43,48 100,00 30,95 100,00 32,35

Table 2: Correlation results show the number of all detections (Detected), number of correct face detections as true
positives (TP), number of detections that turned out not to be faces as false positives (FP) and the number of faces
missed by detection algorithm as false negatives (FN) for different subsets of images (white, yellow, green, blue and red),
previously processed by correlation algorithm (C) and with no preprocessing at all (None). Row All Faces shows the
number of faces in a particular subset of images. TP/Det shows the percentage of true positives out of all detections and
FN/All shows the percentage of false negatives out of all faces in the subset. The TP/Det is for the installation extremely
important, while FN/All is merely informative for the performance of our face detector. Small faces are deliberately
eliminated from further processing already by the face detection algorithm. Note that TP + FN ≤ All, since the region
characterized as TP can contain one or more faces but counts as only one TP.

wards the color of the illumination. Consequently, a lot face/background, eyes, mouth, hair etc. These tasks are
of skin-like pixels are recognized as non-skin-like pixels still too complex to be routinely solved in a few seconds
and the number of correctly detected faces (true positives) on a large variety of input images. We decided therefore
is decreased, since we can not reliably find skin-like pix- to try to achieve somewhat similar effects with much sim-
els. When deviations from standard illumination are much pler means. Our system does not search for any features
more noticeable, we must choose a correlation technique in the face but just processes the input image with selected
with a proper illumination reconstruction. When illumi- filters.
nation conditions are constant over a large period of time, We selected filters that achieve effects similar to segmen-
no illumination detection is necessary. By manually se- tation. They drastically reduce the number of colors by
lecting the illumination we eliminate all the risks linked joining similar looking pixels into uniform looking re-
with false illumination detection and assure the best illu- gions. The system randomly selects one out of 34 prede-
mination reconstruction. termined filter combinations and applies it to the portrait.
Eliminating the influence of non-standard illumination be- 17 filters consist of different combinations of three well
fore face detection ensures much better results. The whole known filters: posterize, color balance and hue-saturation
system is much more flexible and the installation can be balance. The other half of filters are actually based on the
exhibited almost anywhere. first 17 filters, but with an important additional property,
which we call random coloring. Random coloring works
in the following way: the system selects a pixel from the
4. POP-ART COLOR TRANSFORMATIONS
already filtered image and then finds all other pixels in
As mentioned in the introduction, Andy Warhol took pho- the image of the same color. Then the system randomly
tographs of people and performed some kind of color sur- selects a new color from the RGB color space for these
gery on them. In this process Warhol sometimes seg- pixels. In this way, we achieve millions of different col-
mented the face from the background, delineated the con- oring effects and our pop-art portraits almost never look
tours, highlighted some facial features (the mouth or the completely the same. Eight different randomly obtained
eyes), started the process with the negative instead of the filtering effects on the same input portrait can be seen in
positive photography, overlayed the photo with geomet- Fig. 4.
ric color screens etc. [7]. His techniques of transform-
ing a photography into a painting could be described with 5. DISPLAY OF PORTRAITS
a set of formal construction rules, like the ones used in
shape grammars [12, 14]. The application of such rules The installation displays on the framed monitor the se-
for generation of new pop-art portraits which our instal- lected portraits in 15 second intervals. For the display of
lation tries to perform would require automatic segmenta- the final result the system also selects randomly among
tion of input images into its constituent perceptual parts: five possible configurations: in 75% of cases it shows

45
Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

just a single processed portrait, in 25% of cases it sub-

divides the square display area into four smaller squares
each showing a smaller version of the portrait. Each of
the four smaller portraits can be processed with a different
filter and the right-hand portrait column can be mirrored
along the vertical axis. This way of stacking together mul-
tiple images also resembles Andy Warhol’s way of dis-
playing and amassing of images.
Since gallery visitors often stay in front of this installation
for a longer period of time, we integrated a rule that pre-
vents the selection of the same face, or a face at the nearly
same location, in two subsequent 15 second intervals.
In the lower left corner of the LCD display is a counter
counting the seconds from 15 to 0, reminding the currently
“famous” visitor that his fame is fading away.

6. E-MAIL ORDERING OF PORTRAITS

Figure 12: Children having fun in front of the installation.

sender of the request. In addition, a temporary web page

valid for three days is generated, showing the requested
portrait (Fig. 11). On this page one can randomly change
the pop-art effects and save the new versions. If during
an exhibition the installation is not connected to the Inter-
net online, the portraits are sent later. From all requested
portraits a special web gallery of “famous” people is auto-
matically built for each public exhibition on the project’s
web page [1].

7. CONCLUSIONS

The use of computer vision methods in art projects is stim-

ulating and somewhat specific. The concept of the instal-
lation “15 seconds of fame” requires from the vision
Figure 11: Temporary web page showing the e-mail or- system to find at least one face in the image which is then
dered portrait. in turn transformed into a pop-art portrait. Therefore a
high percentage of true positive face detections is very im-
When the installation was exhibited publicly for the first portant, so that the installation does not display too often
time almost everybody who was featured by the installa- other body parts or objects. False negative face detections
tion wanted to get a copy of his or her pop-art portrait. are on the other hand not a problem if at least one face out
Since we were already storing all images in a database for of many is found in each input image, since only one face
later experiments in face detection, we decided to make is selected randomly anyway.
the finished portraits available to the public. Making the installation robust and easy to install and ad-
Therefore, we display now along each portrait a unique minister was also a challenge. In this paper we described
ID number in the lower right corner. A visitor who would in detail how the problem of non-standard illumination
like to get his portrait must note this ID number and send was solved so that the installation can be exhibited under
it in the subject field of an e-mail message addressed to a large variety of illuminations. Since the installation will
[email protected] up to one month after hopefully be exhibited at several art festivals in the near
the end of the exhibition. The system periodically checks future, it was also important to make the whole system
the e-mail messages, finds the requested portraits in the easy to administer remotely over the Internet, so that the
database and e-mails them as an attachment back to the presence of the authors is not required during the whole

46
Proceedings of Mirage 2003, INRIA Rocquencourt, France, March, 10-11 2003

exhibition period. [8] G. D. Finlayson. Color Constancy in Diagonal Cromathic-

As a side product of the installation a huge database for ity Space, Proceedings of the 5th International Conference
experimentation with face detection has been assembled. on Computer Vision (ICCV’95), pp. 218–223, 1995.
This database which shows mainly groups of people (i.e. [9] G. D. Finlayson, S. D. Hordley, P. M. Hubel. Color by Cor-
Figs. 3a, 7 and 8) is available on the Internet along with relation: A Simple, Unifying Framework for Color Con-
our older CVL face database2 [3]. stancy, IEEE Transactions on Pattern Analysis and Ma-
The installation “15 seconds of fame” was well received chine Intelligence, 23(11):1209–1221, 2001.
by the public (Fig. 12). Even people without any prior in- [10] A. Ford, A. Roberts. Colour Space Conversions, 1998,
formation on how the installation works quickly realized http://www.inforamp.net/˜poynton/
that the installation displays portraits of people who are PDFs/coloureq.pdf
present at the moment. But getting a share of that “fame” [11] B. Funt, K. Bernard, L. Martin. Is Machine Colour Con-
proved to be more elusive. People who would step right in stancy Good Enough, Proceedings of the 5th European
front of the installation, trying somehow to force the sys- Conference on Computer Vision (ECCV’98), pp. 445–459,
tem to select them for the next 15 second period of fame, 1998.
would be more often disappointed seeing that somebody [12] J. Gros, F. Solina. Describing artworks using shape gram-
else way back or on the side was selected instead. The mars. Electrotechnical Review, 59(5):314–320, 1992.
only strategy that worked was to stay in front of the instal- [13] E. Hjelmas, B. K. Low. Face detection: A survey. Com-
lation long enough. puter Vision and Image Understanding, 83:236–274, 2001.
There are also several possible improvements on the con-
[14] J. L. Kirsch, R. A. Kirsch. A rule system for analysis in the
ceptual side of this art installation. We could try, for ex- visual arts, Leonardo, 21(4):445–452, 1988.
ample, to determine the facial expression of each selected
[15] J. Kovač, P. Peer, F. Solina. 2D versus 3D color space face
face to classify it as happy, angry or sad [6] and choose
detection. Technical report, Computer Vision Laboratory,
for the pop-art portrait a range of colors that matches that
Faculty of Computer and Information Science, University
mood. A completely new version of the installation enti- of Ljubljana, 2003.
tled “secret thoughts”, which is under development, will http://eprints.fri.uni-lj.si
paste the texture of the selected face on an animated 3D
[16] M2VTS Face Database:
face model and make the face speak out controversial mes-
http://www.tele.ucl.ac.be/M2VTS/
sages.
[17] P. Peer, F. Solina. An Automatic Human Face Detection
Method, Proceedings of Computer Vision Winter Work-
8. REFERENCES shop, Ed. N. Brändle, pp. 122–130, Rastenfeld, Austria,
1999.
[1] 15 seconds of fame: http://black.fri.uni-
lj.si/15sec [18] PICS Image Database:
http://pics.psych.stir.ac.uk/
[2] Art Net Lab: http://black.fri.uni-lj.si
[19] F. Solina, Internet based art installations. Informatica,
[3] CVL face database: http://www.lrv.fri.uni- 24(4):459–466, 2000.
lj.si/database.html
[20] J. C. Terrillon, M. N. Shirazi, H. Fukamachi, S. Akamatsu.
[4] S. Dragan, F. Solina. New information technologies in fine Comparative Performance of Different Skin Chrominance
arts. In F. Solina and S. Dragan, editors, Proc. Conf. New Models and Chrominance Spaces for the Automatic De-
Information Technologies in Fine Arts, Fourth Int. Multi- tection of Human Faces in Color Images, Proceedings of
Conf. Information Society – IS2001, pp. 441–444, Ljubl- 4th IEEE International Conference on Automatic Face and
jana, 2001. Gesture Recognition, pp. 54-61, 2000.
[5] T. Druckrey, Ed. Ars Electronica: Facing the Future. MIT [21] Andy Warhol (quoted in The Washington Post, 15 Nov
Press, Cambridge MA, 2002. 1979) in James B. Simpson, Simpson’s Contemporary
[6] I. Essa, A. Pentland. Coding, analysis, interpretation and Quotations, 1988.
recognition of facial expressions. IEEE Trans. Pattern [22] S. Wilson, Information Arts. Intersections of Art, Science,
Analysis and Machine Intelligence, 19(7):757–763, 1997. and Technology. MIT Press, Cambridge MA, 2001.
[7] F. Feldman (Editor), J. Schellman, C. Defendi, J. Schell- [23] Joint ISO/CIE Standard ISO 10526:1999/CIE S005/E-
man (Contributor), A. Warhol. Andy Warhol Prints: A Cat- 1998 CIE Standard Illuminants for Colorimetry, 1999,
alogue Raisonne 1962–1987. Distributed Art Publishers, http://www.cie.co.at/cie/
3rd Rev edition, October 1997.
2 The older CVL face database consists of: one face per image, 7 im-
ages of the same person in different position and with different face ex-
pressions, 114 persons, all captured under studio illumination.

47