IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 1

Learning a Fixed-Length Fingerprint Representation

Joshua J. Engelsma, Kai Cao, and Anil K. Jain, Life Fellow, IEEE

Abstract—We present DeepPrint, a deep network, which learns to extract fixed-length fingerprint representations of only 200 bytes.
DeepPrint incorporates fingerprint domain knowledge, including alignment and minutiae detection, into the deep network architecture
to maximize the discriminative power of its representation. The compact, DeepPrint representation has several advantages over the
prevailing variable length minutiae representation which (i) requires computationally expensive graph matching techniques, (ii) is
difficult to secure using strong encryption schemes (e.g. homomorphic encryption), and (iii) has low discriminative power in poor quality
fingerprints where minutiae extraction is unreliable. We benchmark DeepPrint against two top performing COTS SDKs (Verifinger and
Innovatrics) from the NIST and FVC evaluations. Coupled with a re-ranking scheme, the DeepPrint rank-1 search accuracy on the
NIST SD4 dataset against a gallery of 1.1 million fingerprints is comparable to the top COTS matcher, but it is significantly faster
(DeepPrint: 98.80% in 0.3 seconds vs. COTS A: 98.85% in 27 seconds). To the best of our knowledge, the DeepPrint representation
arXiv:1909.09901v2 [cs.CV] 18 Dec 2019

is the most compact and discriminative fixed-length fingerprint representation reported in the academic literature.

Index Terms—Fingerprint Matching, Minutiae Representation, Fixed-Length Representation, Representation Learning, Deep
Networks, Large-scale Search, Domain Knowledge in Deep Networks

1 I NTRODUCTION
VER 100 years ago, the pioneering giant of modern day fin-
O gerprint recognition, Sir Francis Galton, astutely commented
on fingerprints in his 1892 book titled “Finger Prints”:
“They have the unique merit of retaining all their
peculiarities unchanged throughout life, and afford in
consequence an incomparably surer criterion of identity
than any other bodily feature.” [1]
Galton went on to describe fingerprint minutiae, the small details
woven throughout the papillary ridges on each of our fingers,
which Galton believed provided uniqueness and permanence prop-
erties for accurately identifying individuals. Over the 100 years
since Galton’s ground breaking scientific observations, fingerprint (a) Level-1 features (b) Level-2 features
recognition systems have become ubiquitous and can be found in a
Fig. 1. The most popular fingerprint representation consists of (a) global
plethora of different domains [2] such as forensics [3], healthcare, level-1 features (ridge flow, core, and delta) and (b) local level-2 features,
mobile device security [4], mobile payments [4], border cross- called minutiae points, together with their descriptors (e.g., texture in
ing [5], and national ID [6]. To date, virtually all of these systems local minutiae neighborhoods). The fingerprint image illustrated here is
a rolled impression from the NIST SD4 database [7]. The number of
continue to rely upon the location and orientation of minutiae minutiae in NIST4 rolled fingerprint images range all the way from 12 to
within fingerprint images for recognition (Fig. 1). 196.
Although automated fingerprint recognition systems based
on minutiae representations (i.e. handcrafted features) have seen and (ii) matching in the encrypted domain, a necessity for
tremendous success over the years, they have several limitations. user privacy protection, is computationally expensive, and
results in loss of accuracy [9].
• Minutiae-based representations are of variable length, • In the context of global population registration, fingerprint
since the number of extracted minutiae (Table 1) varies recognition can be viewed as a 75 billion class problem
amongst different fingerprint images even of the same (≈ 7.5 billion living persons, assuming nearly all with
finger (Fig. 2 (a)). Variations in the number of minutiae 10 fingers) with large intra-class variability and large
originate from a user’s interaction with the fingerprint inter-class similarity (Fig. 2). This necessitates extremely
reader (placement position and applied pressure) and con- discriminative yet compact representations that are com-
dition of the finger (dry, wet, cuts, bruises, etc.). This plementary and at least as discriminative as the traditional
variation in the number of minutiae causes two main minutiae-based representation. For example, India’s civil
problems: (i) pairwise fingerprint comparison is compu- registration system, Aadhaar, now has a database of ≈ 1.3
tationally demanding and varies with number of minutiae billion residents who are enrolled based on their 10 finger-
prints, 2 irises, and face image [6].
• J. J. Engelsma and A. K. Jain are with the Department of Computer Science • Reliable minutiae extraction in low quality fingerprints
and Engineering, Michigan State University, East Lansing, MI, 48824 (due to noise, distortion, finger condition) is problematic,
E-mail: [email protected], [email protected] causing false rejects in the recognition system (Fig. 2 (a)).
• K. Cao is a Senior Biometrics Researcher at Goodix, San Diego, CA
E-mail: [email protected] See also NIST fingerprint evaluation FpVTE 2012 [10].
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2

Fig. 3. Fixed-length, 192-dimensional fingerprint representations ex-

tracted by DeepPrint (shown as 16 × 12 feature maps) from the same
four fingerprints shown in Figure 2. Unlike COTS A, we correctly classify
Fig. 2. Failures of the COTS A minutiae-based matcher (minutiae anno-
the pair in (a) as a genuine pair, and the pair in (b) as an imposter pair.
tated with COTS A). The genuine pair (two impressions from the same
The score threshold of DeepPrint @ FAR = 0.1% is 0.76
finger) in (a) was falsely rejected at 0.1% FAR (score of 9) due to heavy
non-linear distortion and moist fingers. The imposter pair (impressions
from two different fingers) in (b) was falsely accepted at 0.1% FAR (score
of 38) due to the similar minutiae distribution in these two fingerprint face recognition systems which have almost entirely abandoned
images (the score threshold for COTS A @ FAR = 0.1% is 34). In traditional handcrafted features in favor of deep features extracted
contrast, DeepPrint is able to correctly match the genuine pair in (a) by deep networks with remarkable success [11], [12], [13]. How-
and reject the imposter pair in (b). These slap fingerprint impressions
ever, unlike deep network based face recognition systems, we do
come from public domain FVC 2004 DB1 A database [8]. The number
of minutiae in FVC 2004 DB1 A images range from 11 to 87. not completely abandon handcrafted features. Instead, we aim to
integrate handcrafted fingerprint features (minutiae 1 ) into the deep
TABLE 1 network architecture to exploit the benefits of both deep networks
Comparison of variable length minutiae representation with fixed-length and traditional, domain knowledge inspired features.
DeepPrint representation
While prevailing minutiae-matchers require expensive graph
matching algorithms for fingerprint comparison, the 200 byte
(Min, Max) (Min, Max) representations extracted by DeepPrint can be compared using
Matcher
# of Minutiae1 Template Size (kB) simple distance metrics such as the cosine similarity, requiring
COTS A (12, 196) (1.5, 23.7) only d multiplications and d − 1 additions, where d is the dimen-
COTS B (12, 225) (0.6, 5.3) sionality of the representation (for DeepPrint, d = 192)2 . Another
significant advantage of this fixed-length representation is that it
Proposed N.A. 2
0.2† can be matched in the encrypted domain using fully homomorphic
1
Statistics from NIST SD4 and FVC 2004 DB1. encryption [14], [15], [16], [17]. Finally, since DeepPrint is able
2
Template is not explicitly comprised of minutiae. to encode features that go beyond fingerprint minutiae, it is able to
†
Template size is fixed at 200 bytes, irrespective of match poor quality fingerprints when reliable minutiae extraction
the number of minutiae (192 bytes for the features is not possible (Figs. 2 and 3).
and 8 bytes for 2 decompression scalars).
To arrive at a compact and discriminative representation of
only 200 bytes, the DeepPrint architecture is embedded with
To overcome the limitations of minutiae-based matchers, we
present a reformulation of the fingerprint recognition problem. In
1. Note that we do not require explicitly storing minutiae in our final
particular, rather than extracting varying length minutiae-sets for template. Rather, we aim to guide DeepPrint to extract features related to
matching (i.e. handcrafted features), we design a deep network minutiae during training of the network.
embedded with fingerprint domain knowledge, called DeepPrint, 2. The DeepPrint representation is originally 768 bytes (192 features and 4
to learn a fixed-length representation of 200 bytes which discrim- bytes per float value). We compress the 768 bytes to 200 by scaling the floats
to integer values between [0,255] and saving the two compression parameters
inates between fingerprint images from different fingers (Fig. 4). with the features. This loss in precision (which saves significant disk storage
Our work follows the trajectory of state-of-the-art automated space) very minimally effects matching accuracy.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 3

Fig. 4. Flow diagram of DeepPrint: (i) a query fingerprint is aligned via a Localization Network which has been trained end-to-end with the Base-
Network and Feature Extraction Networks (no reference points are needed for alignment); (ii) the aligned fingerprint proceeds to the Base-Network
which is followed by two branches; (iii) the first branch extracts a 96-dimensional texture-based representation; (iv) the second branch extracts a
96-dimensional minutiae-based representation, guided by a side-task of minutiae detection (via a minutiae map which does not have to be extracted
during testing); (v) the texture-based representation and minutiae-based representation are concatenated into a 192-dimensional representation of
768 bytes (192 features and 4 bytes per float). The 768 byte template is compressed into a 200 byte fixed-length representation by truncating floating
point value features into integer value features, and saving the scaling and shifting values (8 bytes) used to truncate from floating point values to
integers. The 200 byte DeepPrint representations can be used both for authentication and large-scale fingerprint search. The minutiae-map can be
used to further improve system accuracy and interpretability by re-ranking candidates retrieved by the fixed-length representation.

fingerprint domain knowledge via an automatic alignment module The primary benefit of the 200 byte representation extracted
and a multi-task learning objective which requires minutiae- by DeepPrint comes into play when performing mega-scale search
detection (in the form of a minutiae-map) as a side task to against millions or even billions of identities (e.g., India’s Aad-
representation learning. More specifically, DeepPrint automati- haar [6] and the FBI’s Next Generation Identification (NGI)
cally aligns an input fingerprint and subsequently extracts both a databases [3]). To highlight the significance of this benefit, we
texture representation and a minutiae-based representation (both benchmark the search performance of DeepPrint against the latest
with 96 features). The 192-dimensional concatenation of these version SDKs (as of July, 2019) of two top performers in the NIST
two representations, followed by compression from floating point FpVTE 2012 (Innovatrics4 v7.2.1.40 and Verifinger5 v10.06 ) on
features to integer value features comprises a 200 byte fixed-length the NIST SD4 [7] and NIST SD14 [21] databases augmented with
representation (192 bytes for the feature vector and 4 bytes for a gallery of nearly 1.1 million rolled fingerprints. Our empirical
storing the 2 compression parameters). As a final step, we utilize results demonstrate that DeepPrint is competitive with these two
Product Quantization [18] to further compress the DeepPrint state-of-the-art COTS matchers in accuracy while requiring only a
representations stored in the gallery, significantly reducing the fraction of the search time. Furthermore, a given DeepPrint fixed-
computational requirements and time for large-scale fingerprint length representation can also be matched in the encrypted domain
search. via homomorphic encryption with minor loss to recognition accu-
Detecting minutiae (in the form of a minutiae-map) as a side- racy as shown in [14] for face recognition.
task to representation learning has several key benefits: More concisely, the primary contributions of this work are:
• We guide our representation to incorporate domain in- • A customized deep network (Fig. 4), called DeepPrint,
spired features pertaining to minutiae by sharing pa- which utilizes fingerprint domain knowledge (alignment
rameters between the minutiae-map output task and the and minutiae detection) to learn and extract a discrimina-
representation learning task in the multi-task learning tive fixed-length fingerprint representation.
framework. • Demonstrating in a manner similar to [29] that Product
• Since minutiae representations are the most popular for Quantization can be used to compress DeepPrint fin-
fingerprint recognition, we posit that our method for gerprint representations, enabling even faster mega-scale
guiding the DeepPrint feature extraction via its minutiae- search (51 ms search time against a gallery of 1.1 million
map side-task falls in line with the goal of “Explainable fingerprints vs. 27,000 ms for a COTS with comparable
AI” [19]. accuracy).
• Given a probe fingerprint, we first use its DeepPrint • Demonstrating with a two-stage search scheme similar
representation to find the top k candidates and then re- to [29] that candidates retrieved by DeepPrint represen-
rank the top k candidates using the minutiae-map provided tations can be re-ranked using a minutiae-matcher in
by DeepPrint 3 . This optional re-ranking add-on further conjunction with the DeepPrint minutiae-map. This further
improves both accuracy and interpretability.
4. https://www.innovatrics.com/
3. The 128 × 128 × 6 DeepPrint minutiae-map can be easily converted into 5. https://www.neurotechnology.com/
a minutiae-set with n minutia: {(x1 , y1 , θ1 ), ..., (xn , yn , θn )} and passed to 6. We note that Verifinger v10.0 performs significantly better than earlier
any minutia-matcher (e.g., COTS A, COTS B, or [20]). versions of the SDK often used in the literature.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 4

TABLE 2
Published Studies on Fixed-Length Fingerprint Representations

HR @ PR = 1.0%1 HR @ PR = 1.0% Template Size Gallery

Algorithm Description
(NIST SD4)2 (NIST SD14)3 (bytes) Size4
Fingercode: Global representation
Jain et al. [22], [23] N.A. N.A. 640 N.A.
extracted using Gabor Filters
MCC: Local descriptors via
Cappelli et al. [24] 3D cylindrical structures 93.2% 91.0% 1,913 2,700
comprised of the minutiae-set representation
Inception v3: Global deep
Cao and Jain [25] representation extracted via 98.65% 98.93% 8,192 250,000
Alignment and Inception v3
PDC: Deep representations extracted at
Song and Feng [26] different resolutions and aggregated 93.3% N.A. N.A. 2,000
into global representation
MDC: Deep representations extracted
Song et al. [27] from minutiae and aggregated into 99.2% 99.6% 1,200 2,700
global representation
Finger Patches: Local deep
representations aggregated into
Li et al. [28] 99.83% 99.89% 1,024 2,700
global representation via global
average pooling
DeepPrint: Global deep representation
Proposed extracted via multi-task CNN 99.75% 99.93% 200† 1,100,000
with built-in fingerprint alignment
1
In some baselines we estimated the data points from a Figure (specific data points were not reported in the paper).
2
Only 2,000 fingerprints are included in the gallery to enable comparison with previous works. (HR = Hit Rate, PR = Penetration Rate)
3
Only last 2,700 pairs (2,700 probes; 2,700 gallery) are used to enable comparison with previous works.
4
Largest gallery size used in the paper.
† The DeepPrint representation can be further compressed to only 64 bytes using product quantization with minor loss in accuracy.

improves system interpretability and accuracy and demon- cylindrical structures computed with minutiae points [24]. While
strates that the DeepPrint features are complementary to both of these representations demonstrated success at the time
the traditional minutiae representation. they were proposed, their accuracy is now significantly inferior to
• Benchmarking DeepPrint against two state-of-the-art state-of-the-art COTS matchers
COTS matchers (Innovatrics and Verifinger) on NIST
Following the seminal contributions of [22], [23] and [24],
SD4 and NIST SD14 against a gallery of 1.1 million
the past 10 years of research on fixed-length fingerprint repre-
fingerprints. Empirical results demonstrate that DeepPrint
sentations [31], [32], [33], [34], [35], [36], [37], [38], [39] has
is comparable to COTS matchers in accuracy at a signifi-
not produced a representation competitive in terms of fingerprint
cantly faster search speed.
recognition accuracy with the traditional minutiae-based represen-
• Benchmarking the authentication performance of Deep-
tation. However, recent studies [25], [26], [27], [28] have utilized
Print on the NIST SD4 and NIST SD14 rolled-fingerprints
deep networks to extract highly discriminative fixed-length finger-
databases and the FVC 2004 DB1 A slap fingerprint
print representations. More specifically, (i) Cao and Jain [25] used
database [8]. Again, DeepPrint shows comparable perfor-
global alignment and Inception v3 to learn fixed-length fingerprint
mance against the two COTS matchers, demonstrating the
representations. (ii) Song and Feng [26] used deep networks to
generalization ability of DeepPrint to both rolled and slap
extract representations at various resolutions which were then
fingerprint databases.
aggregated into a global fixed-length representation. (iii) Song et
• Demonstrating that homomorphic encryption can be used
al. [27] further learned fixed-length minutiae descriptors which
to match DeepPrint templates in the encrypted domain, in
were aggregated into a global fixed-length representation via an
real time (1.26 ms), with minimal loss to matching accu-
aggregation network. Finally, (v) Li et al. [28] extracted local
racy as shown for fixed-length face representations [14].
descriptors from predefined “fingerprint classes” which were then
• An interpretability visualization which demonstrates our
aggregated into a global fixed-length representation through global
ability to guide DeepPrint to look at minutiae-related
average pooling.
features.
While these efforts show tremendous promise, each method
has some limitations. In particular, (i) the algorithms proposed
2 P RIOR W ORK in [25] and [26] both required computationally demanding global
Several early works [22], [23], [24] presented fixed-length fin- alignment as a preprocessing step, and the accuracy is inferior to
gerprint representations using traditional image processing tech- state-of-the-art COTS matchers. (ii) The representations extracted
niques. In [22], [23], Jain et al. extracted a global fixed-length in [27] require the arduous process of minutiae-detection, patch
representation of 640 bytes, called Fingercode, using a set of extraction, patch-level inference, and an aggregation network
Gabor Filters. Cappelli et al. introduced a fixed-length minutiae to build a single global feature representation. (iii) While the
descriptor, called Minutiae Cylinder Code (MCC), using 3D algorithm in [28] obtains high performance on rolled fingerprints
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 5

Fig. 5. Fingerprint impressions from one subject in the DeepPrint training dataset [30]. Impressions were captured longitudinally, resulting in the
variability across impressions (contrast and intensity from environmental conditions; distortion and alignment from user placement). Importantly,
training with longitudinal data enables learning compact representations which are invariant to the typical noise observed across fingerprint
impressions over time, a necessity in any fingerprint recognition system.

(with small gallery size), the accuracy was not reported for slap Algorithm 1 Extract DeepPrint Representation
fingerprints. Since [28] aggregates local descriptors by averaging 1: L(If ): Shallow localization network, outputs x, y, θ
them together, it is unlikely that the approach would work well 2: A: Affine matrix composed with parameters x, y, θ
when areas of the fingerprint are occluded or missing (often times 3: G(If , A): Bilinear grid sampler, outputs aligned fingerprint
the case in slap fingerprint databases like FVC 2004 DB1 A), 4: S(It ): Inception v4 stem
and (v) all of the algorithms, suffer from lack of interpretability 5: E(x): Shared minutiae parameters
compared to traditional minutiae representations. 6: M (x): Minutia representation branch
In addition, existing studies targeting deep, fixed-length finger- 7: D(x): Minutiae map estimation
print representations all lack an extensive, large-scale evaluation 8: T (x): Texture representation branch
of the deep features. Indeed, one of the primary motivations for 9:
fixed-length fingerprint representations is to perform orders of 10: Input: Unaligned 448 × 448 fingerprint image If
magnitude faster large scale search. However, with the exception 11: A ← (x, y, θ) ← L(If )
of Cao and Jain [25], who evaluate against a database of 250K 12: It ← G(If , A)
fingerprints, the next largest gallery size used in any of the 13: Fmap ← S(It )
aforementioned studies is only 2,700. 14: Mmap ← E(Fmap )
As an addendum, deep networks have also been used to 15: R1 ← M (Mmap )
improve specific sub-modules of fingerprint recognition systems 16: H ← D(Mmap )
such as segmentation [40], [41], [42], [43], orientation field 17: R2 ← T (Fmap )
estimation [44], [45], [46], minutiae extraction [47], [48], [49], 18: R ← R1 ⊕ R2
and minutiae descriptor extraction [50]. However, these works all 19: Output: Fingerprint representation R ∈ R192 and minutiae-
still operate within the conventional paradigm of extracting an map H . (H can be optionally utilized for (i) visualization and
unordered, variable length set of minutiae for fingerprint matching. (ii) fusion of DeepPrint scores obtained via R with minutiae-
matching scores.)
3 D EEP P RINT
In the following section, we (i) provide a high-level overview and fingerprint images (≈ 12 fingerprint impressions / finger). The last
intuition of DeepPrint, (ii) present how we incorporate automatic fully connected layer is taken as the representation for fingerprint
alignment into DeepPrint, and (iii) demonstrate how the accuracy comparison during authentication and search.
and interpretability of DeepPrint is improved through the injection The input to DeepPrint is a 448 × 448 7 grayscale fingerprint
of fingerprint domain knowledge. image, If , which is first passed through the alignment module
(Fig. 4). The alignment module consists of a localization network,
3.1 Overview L, and a grid sampler, G [51]. After applying the localization
network and grid sampler to If , an aligned fingerprint It is passed
A high level overview of DeepPrint is provided in Figure 4
to the base-network, S .
with pseudocode in Algorithm 1. DeepPrint is trained with a
The base-network is the stem of the Inception v4 architecture
longitudinal database (Fig. 5) comprised of 455K rolled fingerprint
(Inception v4 minus Inception modules). Following the base-
images stemming from 38,291 unique fingers [30]. Longitudinal
network are two different branches (Fig. 4) comprised primarily of
fingerprint databases consist of fingerprints from distinct subjects
the three Inception modules (A, B, and C) described in [52]. The
captured over time (Fig. 5) [30]. It is necessary to train DeepPrint
first branch, T (x), completes the Inception v4 architecture 8 as
with a large, longitudinal database so that it can learn compact,
fixed-length representations which are invariant to the differences 7. Fingerprint images in our training dataset vary in size from ≈ 512 × 512
introduced during fingerprint image acquisition at different times to ≈ 800 × 800. As a pre-processing step, we do a center cropping (using
and in different environments (humidity, temperature, user interac- Gaussian filtering, dilation and erosion, and thresholding) to all images to
≈ 448 × 448. This size is sufficient to cover most of the rolled fingerprint
tion with the reader, and finger injuries). The primary task during area without extraneous background pixels.
training is to predict the finger identity label c ∈ [0, 38291] 8. We selected Inception v4 after evaluating numerous other architectures
(encoded as a one-hot vector) of each of the 455K training such as: ResNet, Inception v3, Inception ResNet, and MobileNet.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 6

Fig. 6. Unaligned fingerprint images from NIST SD4 (top row) and corresponding DeepPrint aligned fingerprint images (bottom row).

T (S(It )) and performs the primary learning task of predicting a TABLE 3

finger identity label directly from the cropped, aligned fingerprint Localization Network Architecture
It . It is included in order to learn the textural cues in the fingerprint
image. The second branch (Figs. 4 and 8), M (E(S(It ))), again Output Filter
Type
Size Size, Stride
predicts the finger identity label from the aligned fingerprint It ,
but it also has a related side task (Fig. 8) of detecting the minutiae Convolution 128 × 128 × 24 5 × 5, 1
locations and orientations in It via D(E(S(It ))). In this manner, Max Pooling 64 × 64 × 24 2 × 2, 2
we guide this branch of the network to extract representations Convolution 64 × 64 × 32 3 × 3, 1
influenced by fingerprint minutiae (since parameters between the
Max Pooling 32 × 32 × 32 2 × 2, 2
minutiae detection task and representation learning task are shared
in E(x)). The textural cues act as complementary discriminative Convolution 32 × 32 × 48 3 × 3, 1
information to the minutiae-guided representation. The two 96- Max Pooling 16 × 16 × 48 2 × 2, 2
dimensional representations (each dimension is a float, consuming Convolution 16 × 16 × 64 3 × 3, 1
4 bytes of space) are concatenated into a 192-dimensional repre- Max Pooling 8 × 8 × 64 2 × 2, 2
sentation (768 total bytes). Finally, the floats are truncated from 32
Fully Connected 64
bits to 8 bit integer values, compressing the template size to 200
bytes (192 bytes for features and 8 bytes for 2 decompression Fully Connected 3†
parameters). Note that the minutiae set is not explicitly used †These three outputs correspond to x,y ,θ shown in
in the final representation. Rather, we use the minutiae-map to Fig. 4.
guide our network training. However, for improved accuracy and
interpretability, we can optionally store the minutiae set for use in
a re-ranking scheme during large-scale search operations. speed up the localization estimation); (ii) The parameters of the
In the following subsections, we provide details of the major localization network are tuned to minimize the loss (Eq. 9) of
sub-components of the proposed network architecture. the base-network and representation extraction networks. In other
words, rather than supervising the transformation via reference
3.2 Alignment points (such as the core point), we let the base-network and
representation extraction networks tell the localization network
In nearly all fingerprint recognition systems, the first step is to what a “good” transformation is, so that it can learn a more
perform alignment based on some reference points (such as the discriminative representation for the input fingerprint.
core point). However, this alignment is computationally expensive.
Given an unaligned fingerprint image If , a shallow local-
This motivated us to adopt attention mechanisms such as the
ization network first hypothesizes the translation and rotation
spatial transformers in [51].
parameters (x,y , and θ) of an affine transformation matrix Aθ
The advantages of using the spatial transformer module in
(Fig. 4). A user specified scaling parameter λ is used to complete
place of reference point based alignment algorithms are two-
Aθ (Fig. 4). This scaling parameter stipulates the area of the input
fold: (i) it requires only one forward pass through a shallow
fingerprint image which will be cropped. We train two DeepPrint
localization network (Table 3), followed by bilinear grid sampling.
models, one for rolled fingerprints (λ = 1) and one for slap
This reduces the computational complexity of alignment (we
fingerprints (λ = 285 448 ) meaning a 285 × 285 fingerprint area
resize the 448 × 448 fingerprints to 128 × 1289 to further
window will be cropped from the 448 × 448 input fingerprint
9. We also tried 64 × 64, however, we could not obtain consistent alignment image. Given Aθ , a grid sampler G samples the input image If
f f
at this resolution. pixels (xi , yi ) for every target grid location (xti , yit ) to output the
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 7

3.3 Minutiae Map Domain Knowledge

To prevent overfitting the network to the training data and to
extract interpretable deep features, we incorporate fingerprint do-
main knowledge into DeepPrint. The specific domain knowledge
we incorporate into our network architecture is hereafter referred
to as the minutiae map [20]. Note that the minutiae map is not
explicitly used in the fixed-length fingerprint representation, but
the information contained in the map is indirectly embedded in
the network during training.
A minutiae map is essentially a 6-channel heatmap quan-
tizing the locations (x, y) and orientations θ ∈ [0, 2π] of the
minutiae within a fingerprint image. More formally, let h and
w be the height and width of an input fingerprint image and
T = {m1 , m2 , ..., mn } be its minutiae template with n minutiae
points, where mt = (xt , yt , θt ) and t = 1, ..., n. Then, the
minutiae map H ∈ Rh×w×6 at (i, j, k) can be computed by
summing the location and orientation contributions of each of the
minutiae in T to obtain the heat map (Fig. 7 (b)).

n
X
H(i, j, k) = Cs ((xt , yt ), (i, j)) · Co (θt , 2kπ/6) (2)
t=1

where Cs (.) and Co (.) calculate the spatial and orientation con-
tribution of minutiae mt to the minutiae map at (i, j, k) based
upon the euclidean distance of (xt , yt ) to (i, j) and the orientation
difference between θt and 2kπ/6 as follows:

||(xt , yt ) − (i, j)||22

Cs ((xt , yt ), (i, j)) = exp(− ) (3)
2σs2
dφ(θt , 2kπ/6)
Co (θt , 2kπ/6) = exp(− ) (4)
2σs2
where σs2 is the parameter which controls the width of the
gaussian, and dφ(θ1 , θ2 ) is the orientation difference between
angles θ1 and θ2 :

Fig. 7. Minutiae Map Extraction. The minutiae locations and orientations (

of an input fingerprint (a) are encoded as a 6-channel minutiae map |θ1 − θ2 | −π ≤ θ1 − θ2 ≤ π
(b). The “hot spots” in each channel indicate the spatial location of
dφ(θ1 , θ2 ) = (5)
2π − |θ1 − θ2 | otherwise.
the minutiae points. The kth channel of the hot spots indicate the
contributions of each minutiae to the kπ/3 orientation. An example fingerprint image and its corresponding minutiae map
are shown in Figure 7. A minutiae-map can be converted back to
aligned fingerprint image It in accordance with Equation 1. a minutiae set by finding the local maximums in a channel (loca-
tion), and individual channel contributions (orientation), followed
by non-maximal suppression to remove spurious minutiae10 .
xfi
   t
xi
y f  = Aθ yit  (1)
i 3.4 Multi-Task Architecture
1 1
The minutiae-map domain knowledge is injected into DeepPrint
Once It has been computed, it is passed on to the base-network for via multitask learning. Multitask learning improves generalizabil-
classification. Finally, the parameters for the localization network ity of a model since domain knowledge within the training signals
are updated based upon the loss in Equation 9. of related tasks acts as an inductive bias [53], [54]. The multi-
task branch of the DeepPrint architecture is shown in Figures 4
The architecture used for our localization network is shown
and 8. The primary task of the branch is to extract a representation
in Table 3 and images from before and after the alignment
and subsequently classify a given fingerprint image into its “finger
module are shown in Figure 6. In order to get the localization
identity”. The secondary task is to estimate the minutiae-map.
network to properly converge, (i) the learning rate was scaled
Since parameters are shared between the representation learning
by 0.035 and (ii) the upper bound of the estimated affine matrix
task and the minutiae-map extraction task, we guide the minutiae-
translation and rotation parameters was set to 224 pixels and ±60
branch of our network to extract fingerprint representations that
degrees, respectively. These constraints are based on our domain
knowledge on the maximum extent a user would rotate or translate 10. Code for converting a minutiae set to a minutiae map and vice versa is
their fingers during placement on the reader platen. open-sourced: https://bit.ly/2KpbPxV
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 8

be used to compute the combined cross-entropy loss of the two

branches and an image It :

L1 (It , y) = −log(ŷ1j=y |It , w) − log(ŷ2j=y |It , w) (6)

where j ∈ {1, · · · , c}. To further reduce the intra-class variance
of the learned features, we also employ the widely used center-
loss first proposed in [55] for face recognition. In particular, we
compute two center-loss terms, one for each branch in our multi-
task architecture as:

L2 (It ) = ||R1 − ctr1n ||22 + ||R2 − ctr2n ||22 (7)

where ctrin , are the branch, i, and subject, n, specific centers for
a fingerprint image It .
For computing the loss of the minutiae map estimation side
task, we employ the Mean Squared Error Loss between the
estimated minutiae map H and the ground truth minutiae map 12
H as follows:
X
L3 (It , H) = (Hi,j,k − Hi,j,k )2 (8)
i,j,k

Finally, using the addition of all these loss terms, and a dataset
comprised of N training images, our model parameters w are
trained in accordance with:

N
Fig. 8. The custom multi-task minutiae branch of DeepPrint. The di- X
mensions inside each box represent the input dimensions, kernel size, argmin λ1 L1 (Iti , y i ) + λ2 L2 (Iti ) + λ3 L3 (Iti , H i ) (9)
w
and stride length, respectively. i=1

where {λ1 = 1, λ2 = 0.00125, λ3 = 0.095} are empirically

are influenced by minutiae locations and orientations. At the set to obtain convergence. Note, during the training, we augment
same time, a separate branch in DeepPrint aims to extract a our dataset with random rotations, translations, brightness, and
complementary texture-based representation by directly predicting cropping. We use the RMSProp optimizer with a batch size of 30.
the identity of an input fingerprint without any domain knowledge Weights are initialized with the variance scaling initializer. Regu-
(Fig. 4). DeepPrint extracts minutiae maps of size 128×128×6 11 larization included dropout (before the embedding fully connected
to encode the minutiae locations and orientations of an input layer) with a keep probability of 0.8 and weight decay of 0.00004.
fingerprint image of size 448×448×1. The ground truth minutiae We trained for 140K steps, which lasted 25 hours.
maps for training DeepPrint are estimated using the open source After the multitask architecture has converged, a fixed length
minutiae extractor proposed in [20]. feature representation can be acquired by extracting the fully
Note, we combine the texture branch with the minutiae branch connected layer before the softmax layers in both of the network’s
in the DeepPrint architecture (rather than two separate networks) branches. Let R1 ∈ R96 be the unit-length minutiae representation
for the following reasons: (i) the minutiae branch and the texture and R2 ∈ R96 be the unit-length texture representation. Then, a
branch share a number of parameters (the Inception v4 stem), final feature representation is obtained by concatenation of R1 and
reducing the model complexity that two separate models would R2 into R ∈ R192 , followed by normalization of R to unit length.
necessitate, and (ii) the spatial transformer (alignment module) is
optimized based on both branches (i.e. learned alignment benefits 3.5 Template Compression
both the texture-based and minutiae-based representations) avoid- The final step in the DeepPrint representation extraction is tem-
ing two separate spatial transformer modules and alignments. plate compression. In particular, the 192-dimensional DeepPrint
More formally, we incorporate domain knowledge into the representation consumes a total of 768 bytes. We can compress
DeepPrint representation by computing the network’s loss in this size to 200 bytes by truncating the 32 bit floating point feature
the following manner. First, given R1 and R2 as computed values to 8-bit integer values in the range of [0,255] using min-
in Algorithm 1, fully connected layers are applied for identity max normalization. In particular, given a DeepPrint representation
classification logits, outputting y1 ∈ Rc and y2 ∈ Rc , where c R ∈ R192 , we transfer the domain of R to R ∈ N192 and output
is the number of identities in the training set. Next, y1 and y2 R0 , where we restrict the set of the natural numbers N to the range
are both passed to a softmax layer to compute the probabilities ŷ1 of [0,255]. More formally:
and ŷ2 of R1 and R2 belonging to each identity. Finally, ŷ1 and
255(R − min(R))
 
ŷ2 , the ground truth label y , and the network’s parameters w, can R =0
(10)
max(R) − min(R)
11. We extract maps of 128 × 128 × 6 to save GPU memory during training
(enabling a larger batch size), and to reduce disk space requirements for storage 12. The ground truth minutiae maps are estimated using the open-source
of the maps. minutiae extractor in [20].
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 9

where min(R) and max(R) output the minimum and maximum 5 D EEP P RINT S EARCH
feature values of the vector R, respectively. In order to decompress
Fingerprint search entails finding the top k candidates, in a
the features back to float values for matching, we need to save the
database (gallery or background) of N fingerprints, for an input
minimum and maximum values for each representation. Thus, our
probe fingerprint. The simplest algorithm for obtaining the top
final representation is 200 bytes, 192 bytes for the features, 4 bytes
k candidates is to (i) compute a similarity measure between the
for the minimum value and 4 bytes for the maximum value. To
probe template and every enrolled template in the database, (ii)
decompress the representations (when loading them into RAM),
sort the enrolled templates by their similarity to the probe 13 , and
we simply reverse the min-max normalization using the saved
(iii) select the top k most similar enrollees. More formally, finding
minimum and maximum values. Table 4 shows that compression
the top k candidates Ck (.) in a gallery G for a probe fingerprint
only minimally impacts the matching accuracy.
Rp is formulated as:
TABLE 4
Effect of Compression on Accuracy Ck (Rp ) = Rankk ({s(Rp , Rg )|Rg ∈ G}) (12)

DeepPrint DeepPrint where Rankk (.) returns the k most similar candidates from an
Dataset
Uncompressed Features Compressed Features input set of candidates and s is a similarity function such as
NIST SD4† 97.95% 97.90% defined in Equation 11.
FVC 2004 DB1 A†† 97.53% 97.50%
Since minutiae-based matching is computationally expensive,
†
comparing the probe to every template enrolled in the database
TAR @ FAR = 0.01% is reported.
†† TAR @ FAR = 0.1% is reported.
in a timely manner is not feasible with minutiae matchers. This
has led to a number of schemes to either significantly reduce the
search space, or utilize high-level features to quickly index top
candidates [56], [57], [58], [59], [60]. However, such methods
4 D EEP P RINT M ATCHING have not achieved high-levels of accuracy on public benchmark
datasets such as NIST SD4 or NIST SD14.
Two, unit length, DeepPrint representations Rp and Rg can In contrast to minutiae-matchers, the fixed-length, 200 byte
be easily matched using the cosine similarity between the two DeepPrint representations can be matched extremely quickly using
representations. In particular: Equation 11. Therefore, large scale search with DeepPrint can
be performed by exhaustive comparison of the probe template
s(Rp , Rg ) = R|p · Rg (11) to every gallery template in accordance with Equation 12. The
complexity of exhaustive search is linear with respect to both
Thus, DeepPrint authentication (1:1 matching) requires only the gallery size N and the dimensionality d of the DeepPrint
192 multiplications and 191 additions. We also experimented representation (d = 192 in this case).
with euclidian distance as a scoring function, but consistently
obtained higher performance with cosine similarity. Note that if
5.1 Faster Search
compression is added, there would be an additional d subtrac-
tions and d multiplications to reverse the min-max normaliza- Although exhaustive search can be effectively utilized with Deep-
tion of the enrolled representation. Therefore, the authentication Print representations in conjunction with Equation 12, it may be
time effectively doubles. However, depending on the application desirable to even further decrease the search time. For example,
or implementation, compression does not necessarily effect the when searching against 100 million fingerprints, the DeepPrint
search speed since the gallery of representations could be already search time is still (11 seconds on an i9 processor with 64 GB of
decompressed and in RAM before performing a search. RAM) 14 . A natural way to reduce the search time further with
minimal loss to accuracy is to utilize an effective approximate
nearest neighbor (ANN) algorithm.
4.1 Fusion of DeepPrint Score with Minutiae Score Product Quantization is one such ANN algorithm which has
Given the speed of matching two DeepPrint representations, the been successfully utilized in large-scale face search [29]. Product
minutiae-based match scores of any existing AFIS can also be quantization is still an exhaustive search algorithm, however,
fused together with the DeepPrint scores with minimal loss to representations are first compressed via keys to a lookup table,
the overall AFIS authentication speed (i.e. DeepPrint can be which significantly reduces the comparison time between two
easily used as an add-on to existing minutiae-based AFIS to representations. In other words, product quantization reformulates
improve recognition accuracy). In our experimental analysis, we the comparison function in Equation 11 to a series of lookup oper-
demonstrate this by fusing DeepPrint scores together with the ations in a table stored in RAM. More formally, a given DeepPrint
scores of minutiae-based matchers COTS A, COTS B, and [20] representation Rg of dimensionality d, is first decomposed into m
and subsequently improving authentication accuracy. This indi- sub-vectors as:
cates that the information contained in the compact DeepPrint
13. In our search experiments, we reduce the typical sorting time from
representation is complementary to that of minutiae representa- N log(N ) to N log(k) (where k << N ) by maintaining a priority queue
tions. Note, since DeepPrint already extracts minutiae as a side of size k since we only care about the scores of the top k candidates. This
task, fusion with a minutiae-based matcher requires little extra trick reduces sorting time from 23 seconds to 8 seconds when the gallery size
N = 100, 000, 000 and the candidate list size k = 100.
computational overhead (simply feed the minutiae extracted by
14. Search time for 100 million gallery was simulated by generating 100
DeepPrint directly to the minutiae matcher, eliminating the need million random representations, where each feature was a 32-bit float value
to extract minutiae a second time). drawn from a uniform distribution from 0 to 1.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 10

and Sortk returns a list of k candidates sorted in descending

1 2 m order by similarity score.
Rg = (R , R , ..., R ) (13)
We note that since DeepPrint already outputs a minutiae-map,
Next, each mth sub-vector Ri ∈ Rd/m is mapped to a which can easily be converted to a minutiae-set, fusing DeepPrint
codeword cij in a codebook C i = {cij=1,2,...,z |cij ∈ Rd/m } where with a minutiae matcher is quite seamless. We simply convert the
z is the size of the codebook. The index j of each codeword cij DeepPrint minutiae-maps to minutiae-sets, and subsequently input
can be represented as a binary code of log2 (z) bits. Therefore, the minutiae-sets to a minutiae-matcher such as the open-source
after mapping each sub-vector to its codeword, the original d- minutiae matcher in [20].
dimensional representation Rg (d = 192 for DeepPrint) can be
compressed to only m ∗ log2 (z) bits!
The codewords cij ∈ Rd/m for each codebook C i are com- 6 S ECURE D EEP P RINT M ATCHING
puted offline (before search time) using k-means clustering for
each sub-vector. Thus each codebook C i contains z centroids One of the primary benefits of the fixed-length, 192-dimensional
computed from the corresponding sub-vectors Ri . Given all m DeepPrint representation is that it can be encrypted and matched
codebooks {C 1 , C 2 , C 3 , ..., C m }, the product quantizer of Rg is in the encrypted domain (with 192 bits of security [14]) with
computed as: fully homomorphic encryption (FHE). In particular, FHE enables
performing any number of both addition and multiplication oper-
ations in the encrypted domain. Since DeepPrint representations
q(Rg ) = (q 1 (R1 ), ..., q m (Rm )) (14)
can be matched using only multiplication and addition operations
i i (Eq. 11), they can be matched in the encrypted domain with
where q (R ) is the index of the nearest centroid in the codebook
C i , i = 1, ..., m. minimal loss to system accuracy (only loss in accuracy comes
Finally, given a DeepPrint probe representation Rp , and the from converting floating point features to integer value features,
now quantized gallery template Rg , a match score can be obtained resulting in a loss of precision).
in accordance with Equation 15: In contrast, minutiae-based representations cannot be matched
under FHE, since the matching function cannot be reduced to
m
X simple addition and multiplication operations. Furthermore, ex-
s(Rp , Rg ) = ||Rp − q(Rg )||22 = ||Rip − q i (Ri )||22 (15) isting encryption schemes for minutiae-based templates such as
i=1 the fuzzy-vault, result in a loss of matching accuracy, and are
very sensitive to fingerprint pre-alignment [61]. We demonstrate
Thus matching a probe template to each quantized template
in our experiments that the DeepPrint authentication performance
in the gallery requires a one-time build up of a m × z table
remains almost unaltered following FHE matching. We utilize the
which is stored in RAM, followed by m lookups and additions
Fan-Vercauteren FHE Scheme [62] with improvements from [14]
for each quantized template in the gallery. In our experiments,
for improved speed and efficiency17 .
we set z = 256 and m = 64. A quantized template in the
gallery is compressed to 64 bytes, and search is reduced from 192
additions and multiplications (N times, where N is the gallery
7 DATASETS
size) to a one-time m × z table build up, followed by 64 lookups
and additions for each gallery template (a significant savings on We use four sources of data in our experiments. Our training
memory and search time)15 . data is a longitudinal dataset comprised of 455K rolled fingerprint
images from 38,291 unique fingers taken from [30]. Our testing
data is comprised of both large area rolled fingerprint images taken
5.2 Two-stage DeepPrint Search
from NIST SD4 and NIST SD14 (similar to the training data) and
In addition to increasing the speed of large-scale fingerprint search small area slap fingerprint images from FVC 2004 DB1 A.
using DeepPrint with product quantization, we also propose a
method whereby a negligible amount of search speed can be
sacrificed in order to further improve the search accuracy. In 7.1 NIST SD4 & NIST SD14
particular, we first use the DeepPrint representations to find the The NIST SD4 and NIST SD14 databases are both comprised
top-k 16 candidates for a probe Rp in a gallery G. Then, the of rolled fingerprint images (Fig. 9). Due to the number of
top-k candidates are re-ranked using the scores of a minutiae- challenging fingerprint images contained in both datasets (even
matcher fused together with the DeepPrint similarity scores. More for commercial matchers), they continue to be popular benchmark
formally, given a minutiae-matcher function m(.), the k re-ranked datasets for automated fingerprint recognition algorithms. NIST
candidates can be computed by: SD4 is comprised of 2,000 unique fingerprint pairs (total of 4,000
images), evenly distributed across the 5 fingerprint types (arch,
Sortk ({m(mp , mg ) + s(Rp , Rg )|g ∈ G}) (16) left loop, right loop, tented arch, and whorl). NIST SD14 is a
much larger dataset comprised of 27,000 unique fingerprint pairs.
where mp and mg two varying length minutiae templates, Rp
However, in most papers published on fingerprint search, only the
and Rg are the two fixed-length DeepPrint templates, s(.) is the
last 2,700 pairs from NIST SD14 are utilized for evaluation. To
DeepPrint similarity score (either Equation 11 or Equation 15),
fairly compare DeepPrint with previous approaches, we also use
15. We used the Facebook Faiss PQ implementation: https://github.com/ the last 2,700 pairs of NIST SD14 for evaluation.
facebookresearch/faiss
16. The value of k depends on the gallery size N . For the gallery size of 17. We use the following open-source implementation: https://github.com/
N = 1.1 million, we empirically selected k = 500. human-analysis/secure-face-matching
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 11

Fig. 9. Examples of poor quality fingerprint images from benchmark datasets. Row 1: Rolled fingerprint impressions from NIST SD4. Row 2: Slap
fingerprint images from FVC 2004 DB1 A. Rolled fingerprints are often heavily smudged, making them challenging to accurately recognize. FVC
2004 DB1 A also has several distinct challenges such as small overlapping fingerprint area between two fingerprint images, heavy non-linear
distortions, and extreme finger conditions (wet or dry). Minutiae annotated with COTS A.

TABLE 5
Benchmarking DeepPrint Search Accuracy against Fixed-Length Representations in the Literature and COTS

Template NIST SD41 NIST SD142 Template Size Search Time

Algorithm†
Description Rank-1 Accuracy (%) Rank-1 Accuracy (%) Range (kB) (milliseconds)3
Inception v3 + COTS [25] Fixed-Length 97.80 N.A. 8 175

Finger Patches [28] Fixed-Length 99.27 99.04 1.0 16

DeepPrint Fixed-Length 98.70 99.22 0.2 11

COTS A Minutiae-based4 99.55 99.92 (1.5,23.7) 72

COTS B Minutiae-based4 92.9 92.6 (0.6,5.3) 20

1
Only 2,000 fingerprints are included in the gallery to enable comparison with previous works.
2
Last 2,700 pairs are used to enable comparison with previous works.
3
Search times for all algorithms benchmarked on NIST SD4 with an Intel Core i9-7900X CPU @ 3.30GHz
4
We use the proprietary COTS templates which are comprised of minutiae together with other proprietary features.
† These results primarily show that (i) DeepPrints is competitive with the best fixed-length representation in the literature [28] (with a
smaller template size) and state-of-the-art COTS, but also (ii) the benchmark dataset performances are saturated due to small gallery
sizes. Therefore, in subsequent experiments we compare with state-of-the-art COTS against a background of 1.1 million.

7.2 FVC 2004 DB1 A the latest version of the SDK as of July, 2019). Due to our Non-
The FVC 2004 DB1 A dataset is an extremely challenging bench- disclosure agreement, we cannot provide a link between aliases
mark dataset (even for commercial matchers) for several reasons: COTS A and COTS B and Verifinger or Innovatrics. Both of
(i) small overlapping fingerprint area between fingerprint images these SDKs provide an ISO minutia-only template as well as a
from the same subject, (ii) heavy non-linear distortion, and (iii) proprietary template comprised of minutiae and other features. To
extremely wet and dry fingers (Fig. 9). Another major motivation obtain the best performance from each SDK, we extracted the
for selecting FVC 2004 DB1 A as a benchmark dataset is that it is more discriminative proprietary templates. The proprietary tem-
comprised of slap fingerprint images. Because of this, we are able plates are comprised of minutiae and other features unknown to
to demonstrate that even though DeepPrint was trained on rolled us. We note that both Verifinger and Innovatrics are top performers
fingerprint images similar to NIST SD4 and NIST SD14, our in the NIST and FVC evaluations [10], [63].
incorporation of domain knowledge into the network architecture
enables it to generalize well to slap fingerprint datasets. 9 B ENCHMARK E VALUATIONS
We begin our experiments by comparing the DeepPrint search
8 COTS M ATCHERS performance to the state-of-the-art fixed-length representations
In most all of our experiments, we benchmark DeepPrint against reported in the literature. Then, we show that the DeepPrint rep-
COTS A and COTS B (Verifinger 10.0 or Innovatrics v7.2.1.40, resentation can also be used for state-of-the-art authentication by
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 12

benchmarking against two of the top COTS fingerprint matchers TABLE 6

in the market. We further show that this authentication can be Authentication Accuracy (FVC 2004 DB1 A)
performed in the encrypted domain using fully homomorphic en-
cryption. Finally, we conclude our experiments by benchmarking DeepPrint DeepPrint DeepPrint
the large-scale search accuracy of the DeepPrint representation DeepPrint COTS A COTS B + + +
COTS A1 COTS B1 [20]1
against the same two COTS search algorithms.
97.5%† 96.75% 96.57% 98.93% 98.46% 97.6%
1
9.1 Search (1:N Comparison) Sum score fusion is used.
† TAR @ FAR of 0.1% is reported since there are only 4,950 imposter pairs in
Our first experimental objective is to demonstrate that the fixed- the FVC protocol.
length DeepPrint representation can compete with the best fixed- TABLE 7
length representations reported in the academic literature [25], Authentication Accuracy (Rolled-Fingerprints)
[28] in terms of its search accuracy on popular benchmark datasets
and protocols. In particular, we compute the Rank-1 search accu- NIST SD4 NIST SD14
Algorithm
racy of the DeepPrint representation on both NIST SD4 and the TAR @ FAR = 0.01% TAR @ FAR = 0.01%
last 2,700 pairs of NIST SD14 to follow the protocol of the earlier COTS A 99.70 99.89
studies. COTS B 97.80 97.85
The results, reported in Table 5, indicate that the DeepPrint
representation is competitive with the most accurate search algo- Cao et al. [20]1 96.75 95.96
rithm previously published in [28] (slightly lower performance DeepPrint 97.90 98.55
on NIST4 and slightly higher on NIST14). However, we also
note that the existing benchmarks (NIST SD4 and NISTSD14) DeepPrint +
98.70 99.0
for fingerprint search have now become saturated, making it dif- Minutiae [20]
1
ficult to showcase the differences between published approaches. Minutiae extracted from DeepPrint minutiae-map (H) and fed di-
Therefore, in subsequent experiments, we better demonstrate the rectly into minutiae matcher proposed in [20].
efficacy of the DeepPrint representation by evaluating against a
9.2.1 Fusion with Minutiae-Matchers
background of 1.1 million fingerprints (instead of the ≈ 2K in
existing benchmarks). Another interesting result with respect to the DeepPrint authenti-
We highlight once again that DeepPrint has the smallest cation performance is that of the score distributions. In particular,
template among state-of-the-art fixed length representations (200 we found that minutiae-based matchers COTS A and COTS B
bytes vs 1,024 bytes for the next smallest). have very peaked imposter distributions near 0. Indeed, this is
The search performance on FVC 2004 DB1 A is not reported, very typical of minutiae-matchers. In contrast, DeepPrint, has
since the background is not of sufficient size (only 700 slap prints) a peaked genuine distribution around 1.0, and a much flatter
to provide any meaningful search results. imposter distribution. In other words, COTS is generally stronger
at true rejects, while DeepPrint is stronger at true accepts. This
complementary phenomena motivated us to fuse DeepPrint with
9.2 Authentication minutiae-based matchers to further improve their authentication
We benchmark the authentication performance of DeepPrint performance (Table 6). Indeed, our results (Table 6) indicate that
against two state-of-the-art COTS minutiae-based matchers, the DeepPrint representation does contain features complementary
namely COTS A and COTS B. We note that none of the more to minutiae-based matchers, given the improvement in authentica-
recent works on fixed-length fingerprint representation [25], [26], tion performance under score level fusion. We note that since a
[27], [28] have considered authentication performance, making it DeepPrints score can be computed with only 192 multiplications
difficult for us to compare with these approaches (to the best of and 191 additions, it requires very little overhead for existing
our knowledge, the code for these methods is not open-sourced). COTS matchers to integrate the DeepPrint representation into their
From the experimental results (Tables 6 and 7), we note matcher.
that DeepPrint outperforms COTS B on all benchmark testing
9.2.2 Secure Authentication
protocols. We further note that DeepPrint outperforms both COTS
In addition to being competitive in authentication accuracy with
A and COTS B on the very challenging FVC 2004 DB1 A (Fig. 9).
state-of-the-art minutiae matchers, the fixed-length DeepPrint rep-
The ability of DeepPrint to surpass COTS A and COTS B on the
resentation also offers the distinct advantage of matching in the
FVC slap fingerprint dataset is a very exciting find, given the
encrypted domain (using FHE). Here we verify that the DeepPrint
DeepPrint network was trained on rolled fingerprint images which
authentication accuracy remains intact following encryption. We
are comprised of very different textural characteristics than slap
also benchmark the authentication speed in the encrypted domain.
fingerprint impressions (Fig. 9). In comparison to rolled finger-
Our empirical results (Table 8) demonstrate that the authentication
prints, slap fingerprints often (i) require more severe alignment,
accuracy remains nearly the same following FHE, and that authen-
(ii) can contain heavier non-linear distortion, (iii) and are much
tication between a pair of templates takes only 1.26 milliseconds
smaller with respect to impression area. We posit that our injection
in the encrypted domain.
of domain knowledge (both alignment and minutiae detection) into
the DeepPrint architecture help it to generalize well from the rolled
fingerprints it was trained on to the slap fingerprints comprising 10 L ARGE S CALE S EARCH
FVC 2004 DB1 A. We demonstrate this further in a later ablation Perhaps the most important attribute of the compact DeepPrint
study. representation is its ability to perform extremely fast fingerprint
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 13

TABLE 8 TABLE 9
Encrypted Authentication using DeepPrint Representation DeepPrint + Minutiae Re-ranking Search Accuracy (1.1 million
background)

Algorithm NIST SD42 NIST SD142 FVC 2004 DB1 A3

NIST SD4 NIST SD14
Search Time
DeepPrint 97.9% 98.55% 97.5% Metric Rank-1 Rank-1
(milliseconds)1
Search Accuracy Search Accuracy
DeepPrint
96.9% 97.3% 97.0%
+ FHE1 DeepPrint
98.8% 98.22% 300
1
+ [20]
Fully homomorphic encryption is utilized (match time: 1.26 ms).
2
TAR @ FAR = 0.01%. DeepPrint
3
TAR @ FAR = 0.1% 99.45% 99.48% 11,000
+ COTS A2

DeepPrint
search against large galleries. To adequately showcase this feature, 98.25% 98.41% 13,000
+ COTS B2
we benchmark the DeepPrint search accuracy against COTS A and
COTS B on a gallery of over 1.1 million rolled fingerprint images. COTS A3 98.85% 99.51% 27,472
The experimental results show that DeepPrint is able to obtain COTS B3 89.2% 85.6% 428
competitive search accuracy with the top COTS algorithm, at or- 1
Search times benchmarked on an Intel Core i9-7900X CPU @ 3.30GHz
ders of magnitude faster speeds. Note, we are unable to benchmark 2
COTS only used for re-ranking the top 500 DeepPrint candidates.
other recent fixed-length representations in the literature against 3
COTS used to perform search against the entire 1.1 million gallery.
the large scale background, since code for these algorithms has
not been open-sourced. TABLE 10
DeepPrint + PQ: Search Accuracy (1.1 million background)
10.1 DeepPrint Search
First, we show the search performance of DeepPrints using a sim- NIST SD4 NIST SD14
Search Time
Algorithm Rank 1 Rank1
ple exhaustive search technique previously described. In particular, (milliseconds)1
Search Accuracy Search Accuracy
we match a probe template to every template in the gallery, and
select the k candidates with the highest similarity scores. We use DeepPrint 95.15% 94.44% 160
the NIST SD4 and NIST SD14 databases in conjunction with a
DeepPrint + PQ 94.80% 94.18% 51
gallery of 1.1 million rolled fingerprints. Under this exhaustive
1
search scheme, the DeepPrint representation enables obtaining Search times benchmarked on an Intel Core i9-7900X CPU @ 3.30GHz
Rank-1 identification accuracies of 95.15% and 94.44%, respec-
tively (Table 10) and (Fig. 10). Notably, the search time is only 160 DeepPrint representation to further improve the Rank-1 identifica-
milliseconds. At Rank-100, the search accuracies for both datasets tion accuracy. Following this re-ranking, we obtain competitive
cross over 99%. In our subsequent experiments, we demonstrate search accuracy as the top COTS SDK, but at significantly faster
how we can re-rank the top k candidates to further improve the speeds (Table 9).
Rank-1 accuracy with minimal cost to the search time.
10.3 Product Quantization
We further improve the already fast search speed enabled by the
DeepPrint representation by performing product quantization on
the templates stored in the gallery. This reduces the DeepPrint
template size to only 64 bytes and reduces the search speed down
to 51 milliseconds from 160 milliseconds with only marginal loss
to search accuracy (Table 10) and (Fig. 10).

11 A BLATION S TUDY
Finally, we perform an ablation study to highlight the impor-
tance of (i) the automatic alignment module in the DeepPrint
architecture and (ii) the minutiae-map domain knowledge added
Fig. 10. Closed-Set Identification Accuracy of DeepPrint (with and with- during training of the network. In our ablation study, we report
out Product Quantization (PQ)) on NIST SD4 and NIST SD14 (last 2,700
pairs) supplemented with a gallery of 1.1 Million. Rank-1 Identification the authentication performance of DeepPrint with/without the
accuracies are 95.15% and 94.44%, respectively. Search time is only constituent modules.
160 milliseconds. After adding product quantization, the search time We note that in all scenarios, the addition of domain knowl-
is reduced to 51 milliseconds and the Rank-1 accuracies only drop to
94.8% and 94.2%, respectively.
edge improves authentication performance (Tables 11 and 12).
This is especially true for the FVC 2004 DB1 A database which
is comprised of slap fingerprints with different characteristics
(size, distortion, conditions) than the rolled fingerprints used for
10.2 Minutiae Re-ranking training DeepPrint. Thus we show how adding minutiae domain
Using the open-source minutiae matcher proposed in [20], COTS knowledge enables better generalization of DeepPrint to datasets
A and COTS B, we re-rank the top-500 candidates retrieved by the which are very disparate from its training dataset. We note that
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 14

Fig. 11. Illustration of DeepPrint interpretability. The first row shows three example fingerprints from NIST SD4 which act as inputs to DeepPrint.
The second row shows which pixels the texture branch is focusing on as it extracts its feature representation. Singularity points are overlaid to show
that the texture branch fixates primarily on regions surrounding the singularity points. The last row shows pixels which the minutiae branch focuses
on as it extracts its feature representation. We overlay minutiae to show how the minutiae branch focuses primarily on regions surrounding minutiae
points. Thus, each branch of DeepPrint extracts complementary features which comprise more accurate and interpretable fixed-length fingerprint
representations than previously reported in the literature.

alignment does not help in the case of NIST SD4 and NIST SD14 high. This indicates to us that our guiding the DeepPrint network
(since rolled fingerprints are already mostly aligned), however, with minutiae domain knowledge does indeed draw the attention
it significantly improves the performance on FVC 2004 DB1 A of the network to minutiae points. Since both branches focus on
where fingerprint images are likely to be severely unaligned. complementary areas and features, the fusion of the representa-
We also note that the minutiae-based representation and the tions improves the overall matching performance (Table 11).
texture-based representation from DeepPrint are indeed comple-
mentary, evidenced by the improvement in accuracy when fusing
13 C OMPUTATIONAL R ESOURCES
the scores from both representations. (Table 11).
DeepPrint models and training code are implemented in Tensor-
flow 1.14.0. All models were trained across 2 NVIDIA GeForce
12 I NTERPRETABILITY
RTX 2080 Ti GPUs. All search and authentication experiments
As a final experiment, we demonstrate the interpretability of were performed with an Intel Core i9-7900X CPU @ 3.30GHz
the DeepPrint representation using the deconvolutional network and 32 GB of RAM.
proposed in [64]. In particular, we show in Fig. 11 which pixels
in an input fingerprint image are fixated upon by the DeepPrint
network as it extracts a representation. From this figure, we make 14 C ONCLUSION
some interesting observations. In particular, we note that while We have presented the design of a custom deep network ar-
the texture branch of the DeepPrint network seems to only focus chitecture, called DeepPrint, capable of extracting highly dis-
on texture surrounding singularity points in the fingerprint (core criminative fixed-length fingerprint representations (200 bytes)
points, deltas), the minutiae branch focuses on a larger portion of for both authentication (1:1 fingerprint comparison) and search
the fingerprint in areas where the density of minutiae points are (1:N fingerprint comparison). We showed how alignment and
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 15

TABLE 11 [7] “NIST special database 4.” https://www.nist.gov/srd/nist-special-

DeepPrint Representation Comparison database-4. 1, 3
[8] D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman, and A. K. Jain,
“Fvc2004: Third fingerprint verification competition,” in Biometric Au-
Minutiae Texture Fused thentication, pp. 1–7, Springer, 2004. 2, 4
Metric
Representation1 Representation1 Representation2
[9] A. K. Jain, K. Nandakumar, and A. Nagar, “Biometric Template Secu-
rity,” EURASIP Journal on Advances in Signal Processing, vol. 2008,
FVC 2004 DB1A p. 113, 2008. 1
TAR @ 97.4% 90.0% 97.5%
FAR = 0.1% [10] C. I. Watson, G. Fiumara, E. Tabassi, S. Cheng, P. Flanagan, and W. Sala-
mon, “Fingerprint vendor technology evaluation, nist interagency/internal
NIST SD4 report 8034: 2015,” available at https://dx.doi.org/10.6028/NIST.IR.
TAR @ 97.0% 97.15% 97.9% 8034. 1, 11
FAR = 0.01% [11] D. Yi, Z. Lei, S. Liao, and S. Z. Li, “Learning Face Representation from
Scratch,” arXiv preprint arXiv:1411.7923, 2014. 2
NIST SD14 [12] F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A Unified Embed-
TAR @ 97.29% 98.14% 98.55% ding for Face Recognition and Clustering,” in Proceedings of the IEEE
FAR = 0.01% Conference on Computer Vision and Pattern Recognition, pp. 815–823,
1
2015. 2
Each representation (96 bytes) is extracted from one branch in the DeepPrint [13] Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closing the
architecture. Gap to Human-level Performance in Face Verification,” in Proceedings
2
Scores from the minutiae representation are fused with the texture represen- of the IEEE Conference on Computer Vision and Pattern Recognition,
tation using sum score fusion. pp. 1701–1708, 2014. 2
[14] V. N. Boddeti, “Secure Face Matching using Fully Homomorphic En-
TABLE 12 cryption,” in 2018 IEEE 9th International Conference on Biometrics
DeepPrint Ablation Study Theory, Applications and Systems (BTAS), pp. 1–10, IEEE, 2018. 2,
3, 4, 10
[15] M. Upmanyu, A. M. Namboodiri, K. Srinathan, and C. Jawahar, “Ef-
with alignment ficient Biometric Verification in Encrypted Domain,” in International
Metric w/o all with alignment
+ domain knowledge Conference on Biometrics, pp. 899–908, Springer, 2009. 2
[16] M. Upmanyu, A. M. Namboodiri, K. Srinathan, and C. Jawahar, “Blind
FVC 2004 DB1A Authentication: A Secure Crypto-Biometric Verification Protocol,” IEEE
TAR @ 72.86% 88.0% 97.5% Transactions on Information Forensics and Security, vol. 5, no. 2,
FAR = 0.1% pp. 255–268, 2010. 2
[17] M. Barni, T. Bianchi, D. Catalano, M. Di Raimondo, R. D. Labati,
NIST SD4
P. Failla, D. Fiore, R. Lazzeretti, V. Piuri, A. Piva, et al., “A Privacy-
TAR @ 96.95% 96.65% 97.9%
Compliant Fingerprint Recognition System based on Homomorphic En-
FAR = 0.1%
cryption and Fingercode Templates,” in 2010 Fourth IEEE International
Conference on Biometrics: Theory, Applications and Systems (BTAS),
NIST SD14
pp. 1–7, IEEE, 2010. 2
TAR @ 97.96% 96.52% 98.55%
FAR = 0.1% [18] H. Jegou, M. Douze, and C. Schmid, “Product Quantization for Nearest
Neighbor Search,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 33, no. 1, pp. 117–128, 2010. 3
[19] “Explainable Artificial Intelligence (XAI).” https://www.darpa.mil/
fingerprint domain knowledge could be added to the DeepPrint program/explainable-artificial-intelligence. 3
network architecture to significantly improve the discriminative [20] K. Cao, L. Nguyen, C. Tymoszek, and A. K. Jain, “End to End Latent
Fingerprint Search,” IEEE Transactions on Information Forensics and
power of its representations. Then, we benchmarked DeepPrint Security, 2019. 3, 7, 8, 9, 10, 12, 13
against two state-of-the-art COTS matchers on a gallery of 1.1 [21] “NIST special database 14.” https://www.nist.gov/srd/nist-special-
million fingerprints, and showed competitive search accuracy at database-14. 3
significantly faster speeds (300 ms vs. 27,000 ms against a gallery [22] A. K. Jain, S. Prabhakar, L. Hong, and S. Pankanti, “Fingercode: A Filter-
bank for Fingerprint Representation and Matching,” in Computer Vision
of 1.1 million). We also showed how the DeepPrint representation and Pattern Recognition, 1999. IEEE Computer Society Conference on.,
could be used for matching in the encrypted domain via fully vol. 2, pp. 187–193, IEEE, 1999. 4
homomorphic encryption. We posit that the compact, fixed-length [23] A. K. Jain, S. Prabhakar, L. Hong, and S. Pankanti, “Filterbank-based
DeepPrint representation will significantly aid in large-scale fin- Fingerprint Matching,” IEEE Transactions on Image Processing, vol. 9,
no. 5, pp. 846–859, 2000. 4
gerprint search. Among the three most popular biometric traits
[24] R. Cappelli, M. Ferrara, and D. Maltoni, “Minutia Cylinder-Code: A New
(face, fingerprint, and iris), fingerprint is the only modality for Representation and Matching Technique for Fingerprint Recognition,”
which no state-of-the-art fixed-length representation is available. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32,
This work aims to fill this void. no. 12, pp. 2128–2141, 2010. 4
[25] K. Cao and A. K. Jain, “Fingerprint Indexing and Matching: An Inte-
grated Approach,” in Biometrics (IJCB), 2017 IEEE International Joint
Conference on, pp. 437–445, IEEE, 2017. 4, 5, 11, 12
R EFERENCES [26] D. Song and J. Feng, “Fingerprint Indexing based on Pyramid Deep
[1] F. Galton, Finger prints. Macmillan and Company, 1892. 1 Convolutional Feature,” in Biometrics (IJCB), 2017 IEEE International
[2] D. Maltoni, D. Maio, A. K. Jain, and S. Prabhakar, Handbook of Joint Conference on, pp. 200–207, IEEE, 2017. 4, 12
Fingerprint Recognition. Springer, 2nd ed., 2009. 1 [27] D. Song, Y. Tang, and J. Feng, “Aggregating Minutia-Centred Deep
[3] “Next Generation Identification (NGI).” https://www.fbi.gov/services/ Convolutional Features for Fingerprint Indexing,” Pattern Recognition,
cjis/fingerprints-and-other-biometrics/ngi. 1, 3 vol. 88, pp. 397–408, 2019. 4, 12
[4] “iOS Security.” https://www.apple.com/business/site/docs/iOS Security [28] R. Li, D. Song, Y. Liu, and J. Feng, “Learning Global Fingerprint
Guide.pdf. 1 Features by Training a Fully Convolutional Network with Local Patches,”
[5] “Office of Biometric Identity Management Identification Services.” https: to appear in IEEE ICB, 2019. 4, 5, 11, 12
//www.dhs.gov/obim-biometric-identification-services. 1 [29] D. Wang, C. Otto, and A. K. Jain, “Face Search at Scale,” IEEE
[6] “Unique Identification Authority of India, dashboard summary.” https: Transactions on Pattern Analysis and Machine Intelligence, vol. 39,
//portal.uidai.gov.in/uidwebportal/dashboard.do. 1, 3 no. 6, pp. 1122–1136, 2016. 3, 9
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 16

[30] S. Yoon and A. K. Jain, “Longitudinal Study of Fingerprint Recognition,” [52] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, “Inception-v4,
Proceedings of the National Academy of Sciences, vol. 112, no. 28, Inception-resnet and the Impact of Residual Connections on Learning.,”
pp. 8555–8560, 2015. 5, 10 in AAAI, vol. 4, p. 12, 2017. 5
[31] Y. Sutcu, H. T. Sencar, and N. Memon, “A Geometric Transformation to [53] R. Caruana, “Multitask Learning,” Machine Learning, vol. 28, pp. 41–75,
Protect Minutiae-based Fingerprint Templates,” in Biometric Technology Jul 1997. 7
for Human Identification IV, vol. 6539, p. 65390E, International Society [54] X. Yin and X. Liu, “Multi-task Convolutional Neural Network for Pose-
for Optics and Photonics, 2007. 4 Invariant Face Recognition,” IEEE Transactions on Image Processing,
[32] Y. Sutcu, S. Rane, J. S. Yedidia, S. C. Draper, and A. Vetro, “Feature vol. 27, no. 2, pp. 964–975, 2018. 7
Extraction for a Slepian-Wolf Biometric System using LDPC Codes,” in [55] Y. Wen, K. Zhang, Z. Li, and Y. Qiao, “A Discriminative Feature
2008 IEEE International Symposium on Information Theory, pp. 2297– Learning Approach for Deep Face Recognition,” in European Conference
2301, IEEE, 2008. 4 on Computer Vision, pp. 499–515, Springer, 2016. 8
[33] A. Nagar, S. Rane, and A. Vetro, “Privacy and Security of Features [56] B. Bhanu and X. Tan, “Fingerprint Indexing Based on Novel Features of
Extracted from Minutiae Aggregates,” in 2010 IEEE International Con- Minutiae Triplets,” IEEE Transactions on Pattern Analysis and Machine
ference on Acoustics, Speech and Signal Processing, pp. 1826–1829, Intelligence, vol. 25, no. 5, pp. 616–622, 2003. 9
IEEE, 2010. 4 [57] R. Cappelli, M. Ferrara, and D. Maltoni, “Fingerprint Indexing based
on Minutia Cylinder-Code,” IEEE Transactions on Pattern Analysis and
[34] J. Bringer and V. Despiegel, “Binary Feature Vector Fingerprint Repre- Machine Intelligence, vol. 33, no. 5, pp. 1051–1057, 2010. 9
sentation from Minutiae Vicinities,” in 2010 Fourth IEEE International [58] X. Jiang, M. Liu, and A. C. Kot, “Fingerprint Retrieval for Identification,”
Conference on Biometrics: Theory, Applications and Systems (BTAS), IEEE Transactions on Information Forensics and Security, vol. 1, no. 4,
pp. 1–6, IEEE, 2010. 4 pp. 532–542, 2006. 9
[35] E. Liu, H. Zhao, J. Liang, L. Pang, H. Chen, and J. Tian, “Random [59] M. Liu and P.-T. Yap, “Invariant Representation of Orientation Fields
Local Region Descriptor (RLRD): A New Method for Fixed-length for Fingerprint Indexing,” Pattern Recognition, vol. 45, no. 7, pp. 2532–
Feature Representation of Fingerprint Image and its Application to 2542, 2012. 9
Template Protection,” Future Generation Computer Systems, vol. 28, [60] Y. Su, J. Feng, and J. Zhou, “Fingerprint Indexing with Pose Constraint,”
no. 1, pp. 236–243, 2012. 4 Pattern Recognition, vol. 54, pp. 1–13, 2016. 9
[36] F. Farooq, R. M. Bolle, T.-Y. Jea, and N. Ratha, “Anonymous and Revo- [61] U. Uludag, S. Pankanti, and A. K. Jain, “Fuzzy Vault for Fingerprints,”
cable Fingerprint Recognition,” in 2007 IEEE Conference on Computer in International Conference on Audio-and Video-Based Biometric Person
Vision and Pattern Recognition, pp. 1–7, IEEE, 2007. 4 Authentication, pp. 310–319, Springer, 2005. 10
[37] H. Xu, R. N. Veldhuis, A. M. Bazen, T. A. Kevenaar, T. A. Akkermans, [62] J. Fan and F. Vercauteren, “Somewhat Practical Fully Homomorphic
and B. Gokberk, “Fingerprint Verification using Spectral Minutiae Rep- Encryption.,” IACR Cryptology ePrint Archive, vol. 2012, p. 144, 2012.
resentations,” IEEE Transactions on Information Forensics and Security, 10
vol. 4, no. 3, pp. 397–409, 2009. 4 [63] “Fvc ongoing.” https://biolab.csr.unibo.it/fvcongoing/UI/Form/Home.aspx.
[38] K. Nandakumar, “A Fingerprint Cryptosystem Based on Minutiae Phase 11
Spectrum,” in 2010 IEEE International Workshop on Information Foren- [64] M. D. Zeiler and R. Fergus, “Visualizing and Understanding Convolu-
sics and Security, pp. 1–6, IEEE, 2010. 4 tional Networks,” in European Conference on Computer Vision, pp. 818–
[39] Z. Jin, M.-H. Lim, A. B. J. Teoh, B.-M. Goi, and Y. H. Tay, “Generating 833, Springer, 2014. 14
Fixed-length Representation from Minutiae using Kernel Methods for
Fingerprint Authentication,” IEEE Transactions on Systems, Man, and Joshua J. Engelsma graduated magna cum
Cybernetics: Systems, vol. 46, no. 10, pp. 1415–1428, 2016. 4 laude with a B.S. degree in computer science
[40] Y. Zhu, X. Yin, X. Jia, and J. Hu, “Latent Fingerprint Segmentation from Grand Valley State University, Allendale,
based on Convolutional Neural Networks,” in Information Forensics and Michigan, in 2016. He is currently working to-
Security (WIFS), 2017 IEEE Workshop on, pp. 1–6, IEEE, 2017. 5 wards a PhD degree in the Department of
Computer Science and Engineering at Michigan
[41] X. Dai, J. Liang, Q. Zhao, and F. Liu, “Fingerprint Segmentation via
State University. His research interests include
Convolutional Neural Networks,” in Chinese Conference on Biometric
pattern recognition, computer vision, and image
Recognition, pp. 324–333, Springer, 2017. 5
processing with applications in biometrics. He
[42] J. Ezeobiejesi and B. Bhanu, “Latent Fingerprint Image Segmentation won the best paper award at the 2019 IEEE
using Deep Neural Network,” in Deep Learning for Biometrics, pp. 83– International Conference on Biometrics (ICB).
107, Springer, 2017. 5
[43] D.-L. Nguyen, K. Cao, and A. K. Jain, “Automatic Latent Fingerprint
Segmentation,” in 2018 IEEE 9th International Conference on Biometrics Kai Cao received the Ph.D. degree from the
Theory, Applications and Systems (BTAS), pp. 1–9, IEEE, 2018. 5 Key Laboratory of Complex Systems and Intel-
[44] K. Cao and A. K. Jain, “Latent Orientation Field Estimation via Con- ligence Science, Institute of Automation, Chi-
volutional Neural Network,” in Biometrics (ICB), 2015 International nese Academy of Sciences, Beijing, China, in
Conference on, pp. 349–356, IEEE, 2015. 5 2010. He was a Post Doctoral Fellow in the De-
[45] P. Schuch, S.-D. Schulz, and C. Busch, “Deep Expectation for Estimation partment of Computer Science & Engineering,
of Fingerprint Orientation Fields,” in Biometrics (IJCB), 2017 IEEE Michigan State University. His research interests
International Joint Conference on, pp. 185–190, IEEE, 2017. 5 include biometric recognition, image processing
[46] Z. Qu, J. Liu, Y. Liu, Q. Guan, R. Li, and Y. Zhang, “A Novel System and machine learning.
for Fingerprint Orientation Estimation,” in Chinese Conference on Image
and Graphics Technologies, pp. 281–291, Springer, 2018. 5
[47] D.-L. Nguyen, K. Cao, and A. K. Jain, “Robust Minutiae Extractor:
Integrating Deep Networks and Fingerprint Domain Knowledge,” in 2018 Anil K. Jain is a University Distinguished Pro-
International Conference on Biometrics (ICB), pp. 9–16, IEEE, 2018. 5 fessor in the Department of Computer Science
[48] Y. Tang, F. Gao, J. Feng, and Y. Liu, “Fingernet: An Unified Deep at Michigan State University. He is a Fellow of
Network for Fingerprint Minutiae Extraction,” in Biometrics (IJCB), the ACM, IEEE, IAPR, AAAS and SPIE. His
2017 IEEE International Joint Conference on, pp. 108–116, IEEE, 2017. research interests include pattern recognition
5 and biometric authentication. He served as the
[49] L. N. Darlow and B. Rosman, “Fingerprint Minutiae Extraction using editor-in-chief of the IEEE Transactions on Pat-
Deep Learning,” in Biometrics (IJCB), 2017 IEEE International Joint tern Analysis and Machine Intelligence, a mem-
Conference on, pp. 22–30, IEEE, 2017. 5 ber of the United States Defense Science Board
[50] K. Cao and A. K. Jain, “Automated Latent Fingerprint Recognition,” and the Forensics Science Standards Board. He
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018. has received Fulbright, Guggenheim, Alexander
5 von Humboldt, and IAPR King Sun Fu awards. He is a member of
[51] M. Jaderberg, K. Simonyan, A. Zisserman, et al., “Spatial Transformer the United States National Academy of Engineering, a member of The
Networks,” in Advances in Neural Information Processing Systems, World Academy of Science, and foreign members of the Indian National
pp. 2017–2025, 2015. 5, 6 Academy of Engineering and the Chinese Academy of Sciences.