Morphing Deep Learning Paper 2
Morphing Deep Learning Paper 2
Copyright. When the final version is published, the copy of record will be available at the IET Digital Library
Abstract: Face morphing represents nowadays a big security threat in the context of electronic identity documents as well as
an interesting challenge for researchers in the field of face recognition. Despite of the good performance obtained by state-
of-the-art approaches on digital images, no satisfactory solutions have been identified so far to deal with cross-database
testing and printed-scanned images (typically used in many countries for document issuing). In this work, novel approaches
are proposed to train Deep Neural Networks for morphing detection: in particular generation of simulated printed-scanned
images together with other data augmentation strategies and pre-training on large face recognition datasets, allowed to
reach state-of-the-art accuracy on challenging datasets from heterogeneous image sources.
1
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
morphing process. In [10] the authors propose a technique for authors of [33] exploit features from 3D shape and the diffuse
morphing detection based on the analysis of micro-texture reflectance component estimated from the image. Finally, a
variations using Binarized Statistical Image Features (BSIF): landmark-based morphing detection approach is proposed in
an SVM classifier is trained to discriminate bona [34] to compare bona fide and suspected morphed images.
fide/morphed faces. The authors of [11] argue that the Overall, an analysis of the literature allows to identify
morphed images are characterized by a different texture with two major challenges for morphing detection techniques: i)
respect to the unaltered ones and that a progressive JPG robustness to the P&S process; ii) ability to generalize across
compression can further highlight this aspect; the image different databases [35]. The present work mainly focuses on
content is finally represented by different corner features these two aspects. In particular, this paper provides the
exploited for classification. In [12] [13] [14] morphing following contributions:
detection is based on Benford features extracted from Adoption of a simple P&S simulation model [36] for data
quantized DCT coefficients, in [15] key-points features (such augmentation, enabling the possibility of producing
as SURF, ORB, FAST, etc.) are used, while in [16] [17] [18] training images without the cost/effort of the real P&S
[19] texture features such as LBP or Binarized Statistical process. The state-of-the-art about simulating the
Image Features (BSIF) are analyzed. An interesting outcome printing/scanning process in the context of face
of [16] is that low-level features are not robust when used in recognition is very limited. To the best of our knowledge
cross-database testing or in the presence of simple image the most relevant paper [37] has been proposed very
manipulations (e.g., rescaling). The authors of [20], [21] and recently. The approach exploits generative networks to
[22] exploit the principle of image source identification for simulate the real process, providing interesting results
morphing detection, observing that a morphing is a computer- from the visual point of view. This techniques requires a
generated image and its sensor-pattern noise is different from training stage based on real P&S images; on the contrary,
that of a real image. Other works make use of topological an advantage of the model used in our work is that no
analysis of facial landmarks to detect alterations introduced real P&S images are needed for training and a variety of
by morphing [23] [24]; the idea is interesting in principle, but devices or acquisition conditions can easily be simulated
overall the results obtained are unsatisfactory for real just varying the main algorithm parameters. The
application. Most of the referred approaches, when tested on experimental results will show that such simulation
digital images only, provide good classification performance, produces a significant performance improvement on
but the use of different databases and different evaluation morphing detection from P&S images.
metrics make a comparison quite difficult. Extensive experiments using four different well-known
Deep learning techniques based on Convolutional DNN architectures on several test datasets and public
Neural Network (CNN) have been proposed for face benchmarks.
morphing detection [17] [25] [26] [27]. The authors of [17] Thorough performance evaluation on several public
evaluate some networks, pre-trained for face recognition, as benchmarks and comparison with state-of-the-art
feature extractor for digital images, without performing any techniques.
fine-tuning on the specific morphing detection task, while in Experimental results confirming the feasibility of the
[25] two pre-trained networks, AlexNet [7] and VGG19 [8], print/scan simulation model proposed here to deal with
are used for feature extraction after a fine-tuning step. The real P&S images.
authors perform tests on both digital and P&S images and the
experimental results clearly confirm that the second type of 3. Automatic image generation
images represent the main challenge for morphing detection.
In order to exploit the great potential of CNNs for
In [26] some CNNs are used for morphing detection from
classification, a very large set of images is typically needed
digital images; the accuracy of pre-trained networks is
and usually data augmentation techniques are applied [38] to
compared to that of networks learned from scratch, finally
increase the number of samples available for training;
leading to the conclusion that pre-trained networks are more
geometric and photometric transformations are the most
robust for this task. The authors of [27] analyze the accuracy
frequently adopted modifications. In the context of morphing
of pre-trained networks against semantic (partial morphing on
detection, the network training requires both real and
some specific face regions) and black box attacks (partial
morphed image samples, possibly in the two formats (digital
occlusions), and highlight, for the two kind of images, the
and P&S). To avoid the effort/cost of collecting a large
most relevant regions analyzed by the networks for
dataset we proposed novel techniques for automatically
classification. Finally the authors of [28] combine features of
generating high quality morphed face images (Section 3.1)
different nature, hand-crafted and extracted by CNNs,
and simulating the P&S process (Section 3.2).
demonstrating that a substantial improvement in detection
performance can be achieved by their integration. 3.1. Face morphing
A limited number of approaches perform morphing
detection by image-pair comparison. The first approach has Morphed images can be obtained quite easily using
been introduced in [29] [30] where the inverse process of one of the many existing tools and plugins (e.g., Sqirlz Morph
morphing (called demorphing) is adopted to revert the effects [39]). However, the systematic generation of morphed images
produced by morphing. The demorphing technique proved to with specific characteristics can be better realized by ad hoc
be effective both on digital and P&S images. The same techniques. Here we adopt the approach described in [29]
detection scheme has been considered in [31] where different which includes an automatic image retouching phase to
features are evaluated both for single-image and differential minimize visible artifacts. Given two images 𝐼0 and 𝐼1 , the
morphing detection. Deep features extracted from different process generates a set of frames 𝕄 = {𝐼𝛼 , 𝛼 ∈ ℝ, 0 < 𝛼 <
networks are used in [32] for image comparison, while the 1} representing the transformation of the first image (𝐼0 ) into
2
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
the second one (𝐼1 ) (see Figure 1). In general, each frame is a 𝑃0 and 𝑃1 are the two sets of landmarks in 𝐼0 and 𝐼1 ,
weighted linear combination of 𝐼0 and 𝐼1 , obtained by respectively;
geometric warping of the two images based on corresponding 𝑃𝛼 is the set of landmarks aligned according to the
landmarks and pixel-by-pixel blending. Formally: frame weight factor 𝛼;
𝑤𝐵→𝐴 (𝐩) is a warping function.
𝐼𝛼 (𝐩) = (1 − 𝛼) ⋅ 𝐼0 (𝑤𝑃𝛼→𝑃0 (𝐩)) + 𝛼 ⋅ 𝐼1 (𝑤𝑃𝛼→𝑃1 (𝐩)), (1)
𝐼0 𝐼1
Figure 1. Example of morphed frames obtained by the morphing procedure, gradually moving from 𝐼0 to 𝐼1 (first row). In the
second row the result of the automatic retouching process used to remove visible artifacts is shown.
3
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
color and saturation can be obtained though 𝛽𝐾 and 𝛽𝑋 visible in the example, the digital image is much richer of fine
parameters (see Figure 4). Finally the parameters of the details (high frequencies) which are noticeably attenuated
Gaussian smoothing filter (𝑘 and 𝜎) produce the most evident after P&S. The spectrum of the simulated P&S image (Figure
modification introduced by the P&S process, i.e. the blurring 6.(f)) is quite similar to that of the real one (Figure 6.(e)). We
effect represented in Figure 5. can quantify the similarity between the image spectra
adopting commonly used metrics such as the spectral angle
[40] (a measure of distance between two spectra) or the
correlation value. If we compare the digital image and the real
P&S of Figure 6, the spectral angle is quite high (0.69) with
a correlation value of 0.77. The similarity between the real
P&S and the simulated one is much higher, as confirmed by
the smaller spectral angle (0.38) and a higher correlation
value (0.93).
The parameters used for image generation (see Table
1) have been chosen in order to produce images visually
𝜔 = 10 𝜔 = 15 𝜔 = 25
similar to the real P&S ones (MorphDBP&S database
Figure 2. Variation of ω parameter in the P&S simulation described in Section 4.2), but no specific optimizations have
model applied to Figure 6.(a): this parameter mainly affects been carried out (see Figure 6).
image contrast and brightness.
Table 1. Parameter values used in for P&S simulation.
Parameter 𝜔 𝛽𝑋 𝛽𝐾 𝛾 𝑘1 , 𝑘2 𝜎1 , 𝜎2
Value 15.5 20 20 0.5 3 1.2
𝛽𝐾 = 20 𝛽𝑋 = 50 𝛽𝐾 = 20 𝛽𝑋 = 20 𝛽𝐾 = 50 𝛽𝑋 = 20
Figure 4. Variation of 𝜷𝑲 and 𝜷𝑿 in the P&S simulation
Figure 6. For the digital image (a) the result of the real (b)
model applied to Figure 6.(a): these parameters control the
and simulated (c) P&S processes is provided. The
image color and saturation.
corresponding image spectrum is also given for the digital
image (d), the real (e) and the simulated P&S (f).
4. Databases
4.1. Training sets
For network training we used the Progressive
Morphing Database (PMDB) described in [29]. It contains
6000 morphed images automatically generated starting from
𝑘=5 𝑘 = 15 𝑘 = 25 280 different subjects selected from the AR [41], FRGC [42]
and Color Feret [43] [44] databases using different morphing
Figure 5. Variation of 𝒌 in the P&S simulation model applied
factors (𝛼 ∈ {0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45} in Eq.
to Figure 6.(a): this parameter controls the amount of image
(1)).
blurring.
Since PMDB contains a different number of bona fide
In Figure 6 a real P&S image is compared with a and morphed images, a new balanced database (called
simulated P&S image of the same digital image. The image 𝐷𝑖𝑔𝑖𝑡𝑎𝑙) has been derived as follows:
spectrum is also reported to appreciate the low-level signal 1. two images of each subject are chosen resulting in
modifications produced by the P&S process. As clearly 560 bona fide images;
4
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
2. 560 morphed images are randomly selected from the 1. the eye centers and the nose tip are detected using
PMDB morphed images. Neurotechnology VeriLook SDK 10.0 [49];
The P&S process has been simulated by applying the 2. the image is resized to obtain an eye center distance
procedure described in Section 3.2 on all 𝐷𝑖𝑔𝑖𝑡𝑎𝑙 images; we of 150 pixels;
̃.
will refer to this dataset as 𝑃&𝑆 3. a sub-image of size 350×400 pixels is cropped
centered on the nose tip.
4.2. Test sets
The models trained on the datasets introduced in
Section 4.1 are then tested on the following databases:
MorphDBD [29]: it consists of 130 bona fide images
(not morphed) and 100 morphed images (50 males
and 50 females) produced with a significant manual
intervention in order to minimize visible artifacts
(see Figure 7).
MorphDBP&S [29]: P&S version of MorphDBD. The
images have been printed on high quality
photographic paper by a professional photographer
and then scanned (see Figure 8).
NIST FRVT-MORPH benchmark [45] including
several image subsets of variable quality. The morphed Figure 9. Normalized images from Figure 7 (first row) and
images in this dataset have been generated with a Figure 8 (second row).
plurality of morphing algorithms, thus representing a
very hard challenge. 4.4. Data augmentation
SOTAMD benchmark [46], including high quality Both training databases (𝐷𝑖𝑔𝑖𝑡𝑎𝑙 and 𝑃&𝑆̃ ) contain
morphed images (both digital and printed/scanned). 1120 images, not many for an effective network training. To
increase the number of samples, data augmentation is applied
The MorphDBD, MorphDBP&S and SOTAMD datasets obtaining different augmented databases (see Table 2).
are publicly available for testing in the FVC-onGoing In particular, the following transformations are
platform [47] [48], a web-based automated evaluation system applied:
for biometric recognition algorithms. - horizontal mirroring;
- rotation centered on the nose tip ({-5°,0°,+5°});
(a) (b) (c) - horizontal and vertical translation ({-1,0,+1});
- multi-crop, i.e. extracting from each image (size
350×400) five sub-images corresponding to the four
corners and the central region [7]. The crop size is
fixed according to the image input size of the
specific network (see Section 5). In the tests where
multi-crop is not enabled, only the central region is
used.
Figure 7. Images from MorphDBD database: digital version of 5. Deep Neural Networks for morphing detection
bona fide images of two subjects (a) and (c) and the resulting In this work we considered different well-known pre-
morphed image (b). trained deep neural networks (see Table 3). The first two
networks, already used for morphing detection in previous
works [25] [26], have been trained on natural images (i.e.
ImageNet [50]) and therefore the learned filters are not
specific for face representation. The last two networks are
state-of-the-art models trained on very large face datasets: we
can expect that the filters in the low and intermediate levels
of these networks are capable of extracting very powerful
face feature that can be exploited for morphing detection.
The last layer of all the considered architectures has
Figure 8. Images from MorphDBP&S database: P&S version of been changed to deal with a two class problem (morphed vs
the images reported in Figure 7. bona fide): as a consequence, the corresponding weights need
to be learned from scratch.
𝐷𝑖𝑔𝑖𝑡𝑎𝑙
560 560 1120
̃
𝑃&𝑆 √
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 √ √ √
30240 30240 60480
̃
𝑃&𝑆 √ √ √ √
𝐴𝑢
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 √ √ √
16800 16800 33600
̃
𝑃&𝑆 √ √ √ √
𝑀𝑐
AlexNet [7] [51] AlexNet - BVLC version ImageNet [50], specific 227 × 227
Natural 1.2M
VGG19 [8] VGG – 19 weight layers ILSVRC subsets [52] 224 × 224
VGG-Face16 [53] VGG – 16 weight layers [8] VGG-Face dataset [53] 2.6M
Face 224 × 224
VGG-Face2 [54] ResNet-50 [55] VGG-Face 2 dataset [54] More than 3M
6
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
trained on large face datasets with AlexNet and VGG19 problem specific feature detectors are needed to solve such a
struggling to reach decent performance. Since P&S removes complex problem.
most of the digital artifacts we argue that more powerful and
Table 4. Performance indicators of the evaluated networks on the testing databases using different training sets. The best result on
each test database is highlighted in bold.
BPCER (%) at
Test Net Training Accuracy (%) EER (%)
APCER=10% APCER=5% APCER=1%
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 98.3 1.8 0.8 0.8 3.8
AlexNet
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 96.1 1.3 0.8 1.5 1.5
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 92.2 3.9 0.8 3.8 10.8
VGG19
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 94.3 4.3 0.8 3.1 5.4
MorphDBD
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 93.9 3.9 0.8 1.5 10.0
VGG-Face16
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 97.4 0.9 0.0 0.0 0.0
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 95.2 1.8 0.0 1.5 3.1
VGG-Face2
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 93.0 0.9 0.0 0.8 0.8
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 43.5 28.7 50.8 53.8 66.2
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 43.5 32.7 64.6 74.6 83.1
AlexNet
̃
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + 𝑃&𝑆𝐴𝑢 67.4 20.9 43.1 52.3 70.0
̃
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 + 𝑃&𝑆𝑀𝑐 83.5 13.9 25.4 41.5 77.7
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 47.0 32.7 57.7 71.5 89.2
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 44.3 30.4 52.3 66.9 84.6
VGG19
̃
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + 𝑃&𝑆𝐴𝑢 60.4 18.2 36.9 45.4 70.0
̃
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 + 𝑃&𝑆𝑀𝑐 56.5 24.8 49.2 54.6 55.4
MorphDBP&S
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 60.4 12.7 13.8 20.8 69.2
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 56.5 11.3 12.3 22.3 63.1
VGG-Face16
̃
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + 𝑃&𝑆𝐴𝑢 89.6 7.3 7.7 15.4 39.2
̃
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 + 𝑃&𝑆𝑀𝑐 93.5 6.1 2.3 6.9 43.8
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 51.7 16.5 20.0 23.8 40.0
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 45.7 15.7 18.5 33.1 80.0
VGG-Face2
̃
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + 𝑃&𝑆𝐴𝑢 74.3 8.2 6.2 9.2 25.4
̃
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝑀𝑐 + 𝑃&𝑆𝑀𝑐 86.5 6.1 4.6 7.7 17.7
MorphDB_P&S
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 AlexNet ̃
D+SPS_MorphDB_P&S
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + 𝑃&𝑆𝐴𝑢
100% 100%
80% 80%
60% 60%
40% 40%
20% 20%
0% 0%
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
1.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
1.00
MorphDB_P&S
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 VGG-Face16 ̃
D+SPS_MorphDB_P&S
𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + 𝑃&𝑆𝐴𝑢
100% 100%
80% 80%
60% 60%
40% 40%
20% 20%
0% 0%
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
1.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
1.00
Figure 10. Bona fide and morphed score distribution on MorphDB P&S for AlexNet and VGG-Face16 networks obtained using
̃
the 𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 training set (1st column) and the 𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + 𝑃&𝑆 nd
𝐴𝑢 training set (2 column).
7
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
To better analyze the effects of extending the digital reached by the tested algorithms (see Table 5). The proposed
training set with simulated P&S images the bona fide and approach achieves an EER value better than the other
morphed score distributions of AlexNet and VGG-Face16 approaches in the P&S testing set and this is a positive
networks trained with the 𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 and the 𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + indicator since the SOTAMD P&S images have been
̃
𝑃&𝑆 𝐴𝑢 training sets are reported in Figure 10. The graphs produced by multiple processing pipelines reproducing the
clearly show that the networks trained on digital images only real workflow used by different countries for passport issuing.
(𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 ) return a score close to 0 for both bona fide and However, in general, the BPCER values measured in this
morphed images. This means that the modifications benchmark are very bad, confirming that that morphing
introduced by P&S remove the textural details that makes detection from single images is still an open problem.
bona fide and morphed images distinguishable. When Table 5. Results on the SOTAMD benchmark.
training is extended with simulated P&S images
( 𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + 𝑃&𝑆̃ 𝐴𝑢 ), the network pre-trained on face
BPCER (%) at
Test Algorithm EER
images (VGG-Face16) is able to learn P&S specific features APCER = 10% APCER=5%
making it able to discriminate bona fide from morphed Proposed
37.10% 100.00% 100.00%
approach
images. Therefore, the bona fide scores become higher, while
P&S [22] 48.04% 85.86% 97.35%
the morphed scores are generally kept quite low, as clearly
[25] 54.37% 94.89% 98.27%
visible by the score distributions for VGG-Face16. On the [31] 43.34% 100.00% 100.00%
contrary AlexNet does not benefit of this further training step Proposed
whose introduction determines an increment of all the scores 38.99% 100.00% 100.00%
approach
(bona fide and morphed). Digital [22] 44.81% 100.00% 100.00%
6.3. Results on NIST FRVT MORPH [25] 31.80% 65.00% 79.33%
[31] 41.38% 100.00% 100.00%
Two of the most promising solutions identified in our
internal tests (AlexNet trained on 𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 for the digital 7. Conclusions
images and VGG-Face16 trained on 𝐷𝑖𝑔𝑖𝑡𝑎𝑙𝐴𝑢 + 𝑃&𝑆 ̃ 𝐴𝑢 for In this work, different network architectures have been
the P&S images) have been submitted for evaluation at NIST used for single image face morphing detection in both digital
FRVT MORPH which provides a huge and thorough and P&S scenarios. In particular, we focused on P&S images,
comparative evaluation of face morphing detection which still represents a big challenge today. Our initial
algorithms. Please refer to the report [45] and the evaluation experiments on the MorphDBD dataset proved that good
website [58] for the full set of results; in this paper, for lack performance can be achieved on digital images (BPCER=0%
of space,we report in Figure 11 a subset of the NIST DET at APCER=10%), confirming the effectiveness of different
plots comparing single-image based detection algorithms on networks already discussed in [17] [25] [26]. Unfortunately
several image subsets (5 digital and 1 printed and scanned). such low error rates cannot be extended to P&S images
Overall the results show that morphing detection from single (BPCER about 12% at APCER=10% on MorphDBP&S) if
images is a very hard task, in particular when heterogeneous only digital images are used for training. To overcome this
datasets are considered. The proposed approach compares problem, an automatic generation procedure has been
favourably with most of the evaluated approaches, and proposed to simulate the typical P&S image degradation.
presents overall comparable performance with the When combined with automatic morphing generation it
ntnussl_002 algorithm. In the Lincoln subset (Figure 11.e) the allows to produce a vast amount of training data for network
proposed approach is outperformed by other techniques, even training/tuning without the costs/efforts needed to manually
if the best reference value (APCER@BPCER = 0.01%) is print and scan face images. The use of simulated P&S images
reached by the proposed algorithm. In the Print and Scan allowed to significantly improve morphing detection
dataset the proposed approach ranks second among the tested performance, achieving a BPCER=2.3% at APCER=10% on
algorithms and this is very encouraging if we consider that no the MorphDBP&S dataset. As to the different network
real printed/scanned images have been used to train our architectures analysed, the limited size of our training
system; this confirms the efficacy of the simulation procedure databases does not allow to train large models from scratch,
proposed here. so all the CNN used in this work were pre-trained. The
The result in Figure 11.d are worth of attention; in this experiments highlighted that CNN pre-trained on natural
case the morphed images were generated using the morphing images (ImageNet) can perform well on digital images, while
algorithm described in section 3.1 so the level of performance CNN specifically pre-trained on face images (VGG Face
achieved is of course very good, significantly better than all datasets) perform better on P&S images. We argue that to
the other results. This behaviour confirms the importance of detect textural differences between bona fide and morphed
training the system with representative data and suggests that (digital) images, the filters learned from natural images are
a higher robustness can be achieved extending the training quite good, while in presence of P&S images more
data to a variety of morphing algorithms. This would sophisticated and face-specific filters are necessary to detect
probably also allow to improve the results on the subsets of the fine artifacts that survive the printing and scanning
Figure 11.e and Figure 11.f. process.
6.4. Results on SOTAMD benchmark
The same solutions tested at NIST have also been
tested on the SOTAMD benchmark, which revealed to be a
very hard challenge, as confirmed by the very modest results
8
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
(a) (b)
(c) (d)
(e) (f)
Figure 11. DET plots reporting BPCER as a function of APCER for different testing subsets. The horizontal dotted dark green
line represents BPCER=0.01. The proposed algorithm corresponds to the violet curve (unibo_000).
9
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
The tests on the NIST and SOTAMD benchmarks Another important point to investigate in our future
confirm for several data subsets the superiority of the works is to gain some insight about the factors influencing the
proposed approach over other existing solutions, but network decision. Some preliminary works [27] analysed the
generally the results obtained are quite modest. The importance of different face regions for morphing detection
complexity of those two benchmarks is high and single-image on digital images. Further studies are necessary to better
based morphing detection approaches struggle to reach understand and explain these phenomena especially on P&S
decent performance. Morphing detection from single images images: we believe that existing visualization techniques (see
has therefore to be considered a still open challenge and the [59]) can be profitably used to this purpose.
unsatisfactory results suggests the importance of a very Finally, recently Generative Adversarial Networks
robust training, which can only be realized increasing the (GAN) [60] have been successfully used for various image
variability and representativeness of training data. For this generation applications (e.g., [61] [37]); their adoption for
reason a direct extension of our work will be to further morphed image generation and P&S simulation will be
increase the training set, using for instance different investigated in our future researches.
morphing algorithms or different sets of parameters for the
P&S process.
Conference on Biometrics Theory, Applications and
8. References Systems (BTAS), Niagara Falls, NY, 2016.
[11] T. Neubert, “Face Morphing Detection: An Approach
[1] M. Ferrara, A. Franco and D. Maltoni, “On the Based on Image Degradation Analysis,” in
Effects of Image Alterations on Face Recognition International Workshop on Digital Watermarking
Accuracy,” in Face Recognition Across the (IWDW), Magdeburg, Germany, 2017.
Electromagnetic Spectrum, Springer International [12] A. Makrushin, C. Kraetzer, T. Neubert and J.
Publishing, 2016, pp. 195-222. Dittmann, “Generalized Benford's Law for Blind
[2] M. Ferrara, A. Franco and D. Maltoni, “The Magic Detection of Morphed Face Images,” in 6th ACM
Passport,” in IEEE International Joint Conference on Workshop on Information Hiding and Multimedia
Biometrics (IJCB), Clearwater, Florida, USA, 2014. Security, Innsbruck, Austria, 2018.
[3] M. Gomez-Barrero, C. Rathgeb, U. Scherhag and C. [13] A. Makrushin, T. Neubert and J. Dittmann,
Busch, “Is your biometric system robust to morphing “Automatic Generation and Detection of Visually
attacks?,” in 5th International Workshop on Faultless Facial Morphs,” in 12th International Joint
Biometrics and Forensics (IWBF), Coventry, UK, Conference on Computer Vision, Imaging and
2017. Computer Graphics Theory and Applications, Porto,
[4] M. Gomez-Barrero, C. Rathgeb, U. Scherhag and C. Portugal, 2017.
Busch, “Predicting the vulnerability of biometric [14] M. Hildebrandt, T. Neubert, A. Makrushin and J.
systems to attacks based on morphed biometric Dittmann, “Benchmarking face morphing forgery
information,” IET Biometrics, vol. 7, no. 4, pp. 333- detection: Application of stirtrace for impact
341, June 2018. simulation of different processing steps,” in 5th
[5] U. Scherhag, R. Raghavendra, K. B. Raja, M. International Workshop on Biometrics and Forensics
Gomez-Barrero, C. Rathgeb and C. Busch, “On the (IWBF), Coventry, UK, 2017.
vulnerability of face recognition systems towards [15] C. Kraetzer, A. Makrushin, T. Neubert, M.
morphed face attacks,” in 5th International Hildebrandt and J. Dittmann, “Modeling Attacks on
Workshop on Biometrics and Forensics (IWBF), Photo-ID Documents and Applying Media Forensics
Coventry, UK, 2017. for the Detection of Facial Morphing,” in 5th ACM
[6] University of Massachusetts, “Labeled Faces in the Workshop on Information Hiding and Multimedia
Wild,” September 2020. [Online]. Available: Security, Philadelphia, Pennsylvania, USA, 2017.
http://vis-www.cs.umass.edu/lfw/. [16] L. Spreeuwers, M. Schils and R. Veldhuis, “Towards
[7] A. Krizhevsky, I. Sutskever and G. E. Hinton, Robust Evaluation of Face Morphing Detection,” in
“ImageNet classification with deep convolutional 26th European Signal Processing Conference
neural networks,” in 25th International Conference (EUSIPCO), Rome, Italy, 2018.
on Neural Information Processing Systems (NIPS), [17] L. Wandzik, G. Kaeding and R. V. Garcia,
Lake Tahoe, Nevada, 2012. “Morphing Detection Using a General- Purpose Face
[8] K. Simonyan and A. Zisserman, “Very Deep Recognition System,” in 26th European Signal
Convolutional Networks for Large-Scale Image Processing Conference (EUSIPCO), Rome, Italy,
Recognition,” 2015. 2018.
[9] A. Makrushin and A. Wolf, “An Overview of Recent [18] R. Raghavendra, K. B. Raja, S. Venkatesh and C.
Advances in Assessing and Mitigating the Face Busch, “Face morphing versus face averaging:
Morphing Attack,” in 26th European Signal Vulnerability and detection,” in IEEE International
Processing Conference (EUSIPCO), Rome, Italy, Joint Conference on Biometrics (IJCB 2017),
2018. Denver, 2017.
[10] R. Raghavendra, K. B. Raja and C. Busch, “Detecting [19] R. Raghavendra, S. Venkatesh, K. B. Raja and C.
morphed face images,” in IEEE 8th International Busch, “Detecting Face Morphing Attacks with
10
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
Collaborative Representation of Steerable Features,” on Document Analysis Systems (DAS), Vienna,
in International Conference on Computer Vision and Austria, 2018.
Image Processing, 2018. [32] U. Scherhag, C. Rathgeb, J. Merkle and C. Busch,
[20] L. B. Zhang, F. Peng and M. Long, “Face Morphing “Deep Face Representations for Differential
Detection Using Fourier Spectrum of Sensor Pattern Morphing Attack Detection,” IEEE Transactions on
Noise,” in IEEE International Conference on Information Forensics and Security, vol. 15, pp.
Multimedia and Expo (ICME), San Diego, CA, USA, 3625-3639, 2020.
2018. [33] J. M. Singh, R. Ramachandra, K. B. Raja and C.
[21] L. Debiasi, U. Scherhag, C. Rathgeb, A. Uhl and C. Busch, “Robust Morph-Detection at Automated
Busch, “PRNU-based detection of morphed face Border Control Gate using Deep Decomposed 3D
images,” in International Workshop on Biometrics Shape & Diffuse Reflectance,” in International
and Forensics (IWBF), Sassari, Italy, 2018. Conference on Signal-Image Technology & Internet-
[22] U. Scherhag, L. Debiasi, C. Rathgeb, C. Busch and Based, 2019.
A. Uhl, “Detection of face morphing attacks based on [34] N. Damer, V. Boller, Y. Wainakh, F. Boutros, P.
prnu analysis,” IEEE Transactions on Biometrics, Terhörst, A. Braun and A. Kuijper, “Detecting Face
Behavior, and Identity Science, vol. 1, no. 4, pp. 302- Morphing Attacks by Analyzing the Directed
317, 2019. Distances of Facial Landmarks Shifts,” in German
[23] U. Scherhag, D. Budhrani, M. Gomez-Barrero and C. Conference on Pattern Recognition, 2019.
Busch, “Detecting Morphed Face Images Using [35] U. Scherhag, C. Rathgeb and C. Busch,
Facial Landmarks,” in International Conference on “Performance variation of morphed face image
Image and Signal Processing (ICISP), Cherbourg, detection algorithms across different datasets,” in
France, 2018. International Workshop on Biometrics and Forensics
[24] S. Jassim and A. Asaad, “Automatic Detection of (IWBF), Sassari, Italy, 2018.
Image Morphing by Topology-based Analysis,” in [36] C. Lin and S. Chang, “Distortion Modeling and
26th European Signal Processing Conference Invariant Extraction for Digital Image Print-and-
(EUSIPCO), Rome, Italy, 2018. Scan Process,” in International Symposium on
[25] R. Raghavendra, K. B. Raja, S. Venkatesh and C. Multimedia Information Processing (ISMIP), Taipei,
Busch, “Transferable Deep-CNN Features for Taiwan, 1999.
Detecting Digital and Print-Scanned Morphed Face [37] A. Mitkovski, J. Merkle, C. Rathgeb, B. Tams, K.
Images,” in IEEE Conference on Computer Vision Bernardo, N. E. Haryanto and C. Busch, “Simulation
and Pattern Recognition Workshops (CVPRW), of Print-Scan Transformations for Face Images,” in
Honolulu, HI, USA, 2017. International Conference of the Biometrics Special
[26] C. Seibold, W. Samek, A. Hilsmann and P. Eisert, Interest Group (BIOSIG 2020), Darmstadt, 2020.
“Detection of Face Morphing Attacks by Deep [38] L. Taylor and G. Nitschke, “Improving Deep
Learning,” in International Workshop on Digital Learning using Generic Data Augmentation,” 2017.
Watermarking (IWDW), Magdeburg, Germany, [39] Xiberpix, September 2020. [Online]. Available:
2017. http://www.xiberpix.net/SqirlzMorph.html.
[27] C. Seibold, W. Samek, A. Hilsmann and P. Eisert, [40] S. A. Robila, “Using spectral distances for speedup
“Accurate and Robust Neural Networks for Security in hyperspectral image processing,” International
Related Applications Exampled by Face Morphing Journal of Remote Sensing, vol. 26, no. 24, pp. 5629-
Attacks,” arXiv:1806.04265, 2018. 5650, December 2005.
[28] U. Scherhag, C. Rathgeb and C. Busch, “Morph [41] A. M. Martinez and R. Benavente, “The AR face
Deterction from Single Face Image: a Multi- database,” 1998.
Algorithm Fusion Approach,” in 2nd International
[42] P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer,
Conference on Biometric Engineering and
J. Chang, K. Hoffman, J. Marques, J. Min and W.
Applications (ICBEA), Amsterdam, Netherlands,
Worek, “Overview of the face recognition grand
2018.
challenge,” in proceedings IEEE Computer Vision
[29] M. Ferrara, A. Franco and D. Maltoni, “Face and Pattern Recognition, 2005.
Demorphing,” IEEE Transactions on Information
[43] P. J. Phillips, H. Wechsler, J. Huang and P. J. Rauss,
Forensics and Security, vol. 13, no. 4, pp. 1008-1017,
“The FERET database and evaluation procedure for
April 2018.
face-recognition algorithms,” Image and Vision
[30] M. Ferrara, A. Franco and D. Maltoni, “Face Computing, vol. 16, no. 5, pp. 295-306, April 1998.
Demorphing in the Presence of Facial Appearance
[44] P. J. Phillips, H. Moon, S. A. Rizvi and P. J. Rauss,
Variations,” in European Signal Processing
“The FERET evaluation methodology for face-
Conference (EUSIPCO), Rome, Italy, 2018.
recognition algorithms,” IEEE Transactions on
[31] U. Scherhag, C. Rathgeb and C. Busch, “Towards Pattern Analysis and Machine Intelligence, vol. 22,
detection of morphed face images in electronic travel no. 10, pp. 1090-1104, October 2000.
documents,” in 13th IAPR International Workshop
[45] M. Ngan, P. Grother, K. Hanaoka and J. Kuo, “Face
Recognition Vendor Test (FRVT) - Part 4: MORPH
11
This paper is a preprint of a paper accepted by IET Biometrics and is subject to Institution of Engineering and Technology Copyright.
When the final version is published, the copy of record will be available at the IET Digital Library
- Performance of Automated Face Morph Detection,” [59] C. Olah, A. Mordvintsev and L. Schubert, “Feature
July 24, 2020. Visualization,” Distill, 2017.
[46] K. Raja, M. Ferrara, A. Franco, L. Spreeuwers, I. [60] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu,
Batskos, F. De Wit, M. Gomez-Barrero, U. Scherhag, D. Warde-Farley, S. Ozair, A. Courville and Y.
D. Fischer, S. Venkatesh, J. M. Singh, G. Li, L. Bengio, “Generative Adversarial Nets,” in
Bergeron, S. Isadskiy, R. Ramachandra, C. Rathgeb, Conference on Neural Information Processing
D. Frings, U. Seidel, F. Knopjes, R. Veldhuis, D. Systems (NIPS), Montreal, Quebec, Canada, 2014.
Maltoni and C. Busch, “Morphing Attack Detection [61] F. Peng, L. B. Zhang and M. Long, “FD-GAN: Face-
- Database, Evaluation Platform and Benchmarking,” demorphing generative adversarial network for
accepted on IEEE Transactions on Information restoring accomplice's facial image,” 2018.
Forensics and Security (TIFS), 2020.
[47] B. Dorizzi, R. Cappelli, M. Ferrara, D. Maio, D.
Maltoni, N. Houmani, S. G. Salicetti and A. Mayoue,
“Fingerprint and On-line signature Verification
Competitions at ICB 2009,” in Proceedings 3rd
IAPR/IEEE International Conference on Biometrics
(ICB09), Alghero, 2009.
[48] BioLab, September 2020. [Online]. Available:
http://biolab.csr.unibo.it/fvcongoing.
[49] Neurotechnology Inc., “Neurotechnology Web Site,”
September 2020. [Online]. Available:
http://www.neurotechnology.com/.
[50] J. Deng, W. Dong, R. Socher, L. J. Li, K. Li and L.
Fei-Fei, “ImageNet: A large-scale hierarchical image
database,” in IEEE Conference on Computer Vision
and Pattern Recognition, Miami, FL, USA, 2009.
[51] “BVLC AlexNet Caffe Model,” September 2020.
[Online]. Available:
https://github.com/BVLC/caffe/tree/master/models/
bvlc_alexnet.
[52] O. Russakovsky, J. Deng, H. Su, J. Krause, S.
Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla,
M. Bernstein, A. C. Berg and L. Fei-Fei, “ImageNet
Large Scale Visual Recognition Challenge,”
International Journal of Computer Vision, vol. 115,
no. 3, pp. 211-252, December 2015.
[53] O. M. Parkhi, A. Vedaldi and A. Zisserman, “Deep
Face Recognition,” in British Machine Vision
Conference (BMVC), 2015.
[54] Q. Cao, L. Shen, W. Xie, O. M. Parkhi and A.
Zisserman, “VGGFace2: A Dataset for Recognising
Faces across Pose and Age,” in 13th IEEE
International Conference on Automatic Face &
Gesture Recognition (FG 2018), Xi'an, China, 2018.
[55] K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual
Learning for Image Recognition,” in IEEE
Conference on Computer Vision and Pattern
Recognition (CVPR), Las Vegas, NV, USA, 2016.
[56] International Organization for Standardization (ISO),
“Information Technology – Biometric presentation
attack detection – Part 3: Testing and reporting,”
Geneva, Switzerland, 2017.
[57] M. Ferrara, A. Franco and D. Maltoni, “Face
morphing detection in the presence of
printing/scanning and heterogeneous image sources,”
2019.
[58] NIST, September 2020. [Online]. Available:
https://pages.nist.gov/frvt/html/frvt_morph.html.
12