IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO.

9, SEPTEMBER 2019 4413

Two-Layer QR Codes
Tailing Yuan , Yili Wang, Kun Xu, Member, IEEE, Ralph R. Martin,
and Shi-Min Hu , Senior Member, IEEE

Abstract— A quick-response code (QR code) is a two- encompass all applications of QR codes needing multiple
dimensional code akin to a barcode that encodes a message of messages.
limited length. In this paper, we present a variant of QR code, Various kinds of art exhibit different effects when
a two-layer QR code. Its two-layer structure can display two
alternative messages when scanned from two different directions. viewed or illuminated from different angles or distances,
We propose a method to generate such two-layer QR codes such as lenticular images [3] which can change the dis-
encoding two given messages in a few seconds. We also demon- played image depending on the viewing angle, hybrid
strate the robustness of our method on both synthetic and images [4] which have different interpretations depending on
fabricated examples. All source code will be made publicly the viewing distance, and shadow art sculptures [5] which
available (https://github.com/yuantailing/two-layer-qrcode).
cast different meaningful shadows under different lighting
Index Terms— QR code, two-layer QR code, barcode. directions.
I. I NTRODUCTION Inspired by such work [3]–[5], we present a variant of QR
code called two-layer QR code, in which two messages are
A QUICK response code (QR code) is a two-dimensional
barcode, consisting of black and white squares, called
modules. It encodes information such as a URL or a text
stored and can be retrieved separately from two different view-
ing directions. It is a specially designed two-layer structure,
with a top layer and a bottom layer. Each layer contains
message. By scanning a QR Code using a mobile phone, a user
a matrix of modules. The bottom layer modules are black
can get immediate access to its content. QR codes have been
or white, while the top layer modules may be transparent,
widely used in numerous applications such as advertising and
allowing the code to appear differently from different viewing
digital payments, as they are easy to create and scan.
directions. The two-layer QR code can thus display two valid
A QR code can encode only a single message. However,
QR codes, encoding two different messages, by changing the
encoding two or more messages is useful in many scenarios.
view.
For example, a store may provide alternative payment meth-
Our method is thus more generally applicable than [1], [2]
ods, an app may need different download links for different
and could be used in almost any real-world scenarios where
operating systems, and an exhibition item may have descrip-
two messages are needed. These might include providing
tions in multiple languages. While previous work [1], [2]
alternative payment methods, giving different app download
has considered combining two messages in one QR code,
links for different operating systems, providing explanatory
they target particular applications. The former targets sepa-
text in different languages, or giving multiple stories for an
rate public and private messages, and requires a specialized
exhibit in a museum. Another possible use is in gaming,
reader for the latter. The second uses device characteristics
where two or more people sit at a table opposite each other.
(such as its operating system) to determine which alternative
A two-layer QR code on the table would naturally be scanned
message to retrieve; the user is not involved in selecting which
from different directions, e.g., to provide separate messages
alternative message is desired. Such specific usages do not
to a card dealer and other card players. For puzzle games,
Manuscript received August 12, 2018; revised February 21, 2019 and scanning from the left could show puzzle questions, and
March 24, 2019; accepted March 24, 2019. Date of publication May 8, 2019; scanning from the right could give the answers. Using a
date of current version July 1, 2019. This work was supported in part by the
National Natural Science Foundation of China under Project 61822204 and single code ensures that the questions will always be correctly
Project 61521002, in part by the Beijing Higher Institution Engineering associated with the answers, which may not always occur if
Research Center, and in part by the Tsinghua-Tencent Joint Laboratory for two separate QR codes are used (one may get lost, become
Internet Innovation Technology. The associate editor coordinating the review
of this manuscript and approving it for publication was Dr. Zhengguo Li. defaced, etc.).
(Corresponding author: Kun Xu.) As well as explaining the structure of our two-layer QR
T. Yuan, K. Xu, and S.-M. Hu are with the Department of Computer code, we also give a fully automatic algorithm to create a
Science and Technology, Tsinghua University, Beijing 100084, China (e-mail:
[email protected]). two-layer QR code encoding two given messages. Specifically,
Y. Wang is with the Department of Foreign Languages and Literatures, we optimize the module values of both layers, with the aim of
Tsinghua University, Beijing 100084, China. maximizing its error correction capacity while also encoding
R. R. Martin, retired, was with the School of Computer Science and
Informatics, Cardiff University, Cardiff CF10 3AT, U.K. He resides in Cardiff the two messages. For speed, we have developed a two-step
CF24 3AA, U.K. optimization scheme and merge and reduce operators to reduce
This paper has supplementary downloadable material available at the solution space. The optimization problem is then solved
http://ieeexplore.ieee.org, provided by the authors. The material includes a
supplementary document and a supplementary video. by simulated annealing in just a few seconds. We also show
Digital Object Identifier 10.1109/TIP.2019.2908490 how to physically fabricate a two-layer QR code.
1057-7149 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
4414 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 9, SEPTEMBER 2019

Our two-layer QR codes can also be combined with exist- in 2D barcodes. Colored QR codes [31]–[33] have been pro-
ing QR code beautification approaches, to display a visual posed to increase data capacity by using more color channels.
logo within the QR code display area. We demonstrate the Yang et al. [34] proposed ARTcode, which look like an image,
robustness of our method on a variety of examples, including retaining aesthetic content and decoding feasibility in one
both synthetic and fabricated ones. visual pattern. However, a common drawback of all such codes
is that additional software is needed when decoding.
II. R ELATED W ORK
A. QR Code Beautification
C. QR Codes Combining Two Messages
Although a QR code is machine readable, it appears as
a random pattern of black and white modules and is not Other previous works [1], [2] have considered how to com-
visually meaningful. To address this, various works have con- bine two messages into one QR code. The two-level QR code
sidered QR code beautification, allowing e.g. the embedding (2LQR) [1] stores a public message and an additional private
of visual information such as a logo within a QR code. QR message. The messages are of different kinds. The public
code beautification approaches are generally based on one of message can be read by any QR code reader, but the private
3 techniques: direct superimposition, module modification and message can only be read by a specialized reader. This
padding editing. method is designed specifically for private message sharing
1) Direct Superimposition: One way to achieve QR code and document authentication. However, it is not suited to
beautification is to directly superimpose a logo or a pattern general scenarios; users are required to install a specialized
onto a QR code [6]–[9]. Superimposing a logo will damage decoding App for the private message.
some codewords but as long as this is within the error In [2], two App download links can be combined into one
correction capacity, the QR code can still be correctly decoded. QR code, including e.g. one link for iOS and another for
Different schemes have been proposed to find appropriate posi- Android. This work does not modify QR code design or
tions, orientations, and scales of superimposed logos without structure: the QR code simply stores only a single link to an
damaging the code. Samretwit and Wakahara [10] measured external server. The linked webpage is redirected to the desired
the reliability of a superimposed QR code at different positions download link as determined by the operating system of the
and scales. As an alternative to superimposition, Baharav device scanning the QR code. While working well for this
and Kakarala [11] achieved QR code beautification by image target usage, it has limited use in general scenarios. An Internet
blending. connection is required for message selection, while standard
2) Module Modification: A QR code remains decodable QR code decoding can be done offline. Users are not involved
as long as the values in the central pixels of each module in choosing the message decoded, which is determined solely
are unchanged. Several beautification approaches [12]–[19] by the device characteristics (unless additional information is
have been proposed, in which carefully crafted modules, input). For example, this approach cannot readily use a single
usually of a smaller size and only covering the central pixels, QR code for the puzzle game scenario (provide either the
replaces the original modules. For example, Chu et al. [12] puzzle questions or answers).
generated halftone QR codes by subdividing modules into While the above methods [1], [2] are specialized for partic-
3 × 3 submodules. Lin et al. [13] embellished QR codes by ular applications, our method is more general and flexible.
superimposition as well as stylizing module shape based on We allow two arbitrary messages, and the user can freely
a specified exemplar. Gao et al. [14] showed how to embed choose which message to read simply by viewing our code
QR codes into color images, making QR code perceptually from different viewing directions, using any standard QR code
invisible to humans but still machine-readable. reader. Changing viewing angles to obtain different messages
3) Padding Editing: If the length of the encoded message provides a natural, simple interface which is easy to learn and
is less than the code’s capacity, padding bits follow in each to perform, and follows standard QR code practice. We do not
data codeword. The values of these padding bits can be freely require an Internet connection, external server, or specialized
changed without altering the message. By taking advantage readers.
of this property, Fujita [20] use non-systematic encoding of A disadvantage of our two-layer QR codes is that they
the Reed-Solomon code to allow free editing of a selected require fabrication rather than simple printing. However, this
area without reducing the code’s error correction capability. only causes more work for the content provider (who creates
Other methods [21]–[27] have been proposed for generating the code), while users (who scan the code) can use existing
colored QR codes by combining padding editing and module QR code readers. The only difference to them is that the code
modification. is scanned from the left or right directions instead of from the
front. We believe this is a reasonable trade-off for the potential
benefits obtained.
B. QR Code Alternatives The remainder of the paper is organized as follows.
Various alternative 2D barcode formats have been pro- We briefly review standard QR codes in Section III, then
posed for different purposes. Picture-embedding 2D barcodes explain the basis of our two-layer QR codes in Section IV.
like 2D image code [28], PiCode [29], and RA (Robust and We present an algorithm for their generation in Section V.
Aesthetic) code [30] have been proposed to embed images We provide experimental results in Section VI and an
YUAN et al.: TWO-LAYER QR CODES 4415

Fig. 1. QR code structure. Finder patterns and alignment patterns are located
at fixed positions. The format information records the error correction level
and data masking pattern. A codeword consists of 8 modules and represents Fig. 2. A two-layer QR code consists of a top layer and a bottom layer
8 bits of data. with a small gap between them. The bottom layer is composed of black and
white modules, while the modules of the top layer may also be transparent.
When scanned from the left view, this QR code appears as a standard QR
evaluation in Section VII. Finally, we conclude the paper in code encoding the string “IEEE”; when scanned from the right, it appears
as a QR code encoding a different string, “TIP”. In all figures, transparent
Section VIII. modules are shown in translucent green to make them visible; in the actual
QR code they are transparent.
III. QR C ODE BASICS
To aid understanding, and introduce terminology, we briefly the data capacity of a QR code, i.e., the length of the
remind the reader of the structure and operation of an original message it can represent.
QR code. Further details can be found in the specification ISO/IEC
1) Structure: A QR code is a 2D image consisting of 18004:2015 [36].
black and white squares: see Figure 1. Its parameters and 2) Message Encoding: To encode a given message as a
components include: QR code, it is first converted to a sequence of bits using one of
• A module is the smallest element (black or white square) four encoding modes: numeric, alphanumeric, byte, and Kanji.
of the QR code. Each module represents 1 bit of data, Different modes use different character sets, with different
i.e., a black module represents 1 while a white module numbers of bits for each character. Suitable values of version
represents 0. and EC level are selected, based on message length and desired
• A codeword consists of 8 modules and represents 8 bits reliability. Next, the sequence is put into data codewords
of data. in each block in a particular order, and corresponding error
• An error correction block, or block for short, consists of correction codewords are computed. Finally, the modules are
a sequence of codewords. A block represents a message bit-wise XORed with one of the eight predefined data masking
with a specific length. The codewords within it comprise patterns, using the one which best avoids adjacent modules
data codewords and error correction codewords. The appearing in the same color.
former represent the underlying message, while the latter 3) Message Decoding: To decode a message in a scanned
are Reed-Solomon [35] checksums computed from the QR code, the code is first located by use of the finder and
data codewords, helping to make the QR code robust to alignment patterns. The value of each module is obtained by
noise and damage. A block with p codewords in total, out binarization. Next, the EC level and data masking pattern are
of which k are data codewords, can be correctly recovered obtained from the format information. The bit sequence of
if it has no more than ( p − k)/2 incorrect codewords. the message may then be recovered block by block, the error
• Finder patterns and alignment patterns are located correction codewords helping to repair noise and damage.
at fixed positions and are used to locate and orient the
QR code when scanning. IV. T WO -L AYER QR C ODE
• Format information stores parameters describing the In this section, we introduce our two-layer QR codes, which
QR code, including its error correction level (EC level) aim to display two valid QR codes when scanned from two
and data masking pattern. The EC level determines different views. It has a specially designed structure, which
the level of error tolerance, via the numbers of data consists of a top layer and a bottom layer. The bottom layer
codewords and error correction codewords in each block. is composed of black and white modules, which is similar to
Four EC levels (L, M, Q, and H ) exist, with increasing a standard QR code. The top layer has a similar number of
reliability but decreasing data capacity. One of 8 prede- modules (see later), but some of them may be transparent. The
fined data masking patterns must be selected. A further space between the two layers allows our proposed structure
parameter, version, defines the size of the QR code: to have different appearances when scanned from different
a QR code having version V has (4V + 17) × (4V + 17) directions. Figure 2 shows the structure of a two-layer QR
modules. The version and EC level together determine code.
4416 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 9, SEPTEMBER 2019

V. G ENERATION A LGORITHM
A. Problem Definition
Given two target messages, our goal is to generate top
and bottom layer patterns (L t and L b ) such that the left and
right viewing patterns (Ol and Or ) encode the two desired
messages. We first use conventional QR code generation
algorithms [36] to convert each message into a standard QR
code, which we refer to as left and right target patterns
(Tl and Tr ), respectively. They must have the same version
(i.e. size) but the EC levels and data masking patterns may
Fig. 3. Each module on the bottom or top layer affects two different modules differ. For now, we restrict their EC levels and data masking
in the viewing patterns. The color of module L b (i, j) affects the colors of
Ol (i, j) and Or (i, j). patterns to be the same. We will discuss how to handle cases
when they are different in Section VI-A2.
A strict requirement would demand that the viewing patterns
should exactly match the target patterns (i.e., Ol = Tl ,
Or = Tr ), but this is in practice impossible to achieve.
Instead, we require that the same messages can still be decoded
from the viewing patterns as from the target patterns. At the
same time, we should maximize the reliability of the viewing
patterns, i.e. their ability to still be correctly decodable when
Fig. 4. A generated two-layer QR code. Left to right: bottom layer, top layer, damaged. To measure the reliability of a two-layer QR code,
left view, and right view. These two views show QR codes representing the we define its effective recovery ratio E to be:
strings “Two-Layer QR Code” and “Computer Science” respectively.
E = min D(Osi , Tsi ), (2)
i,s
The bottom and top layers are designed as follows.
where Osi denotes the i -th block in one pattern Os (Os ∈
The bottom layer has N × N modules. Each module on the
{Ol , Or }, either the left viewing pattern Ol or the right viewing
bottom layer is colored white (for 0) or black (for 1). The
pattern Or ), Tsi denotes the i -th block in the corresponding
top layer has (N + 1) × N modules, with N + 1 and N in
target pattern (which covers the same area as Osi ), and D(·)
x and y directions respectively. Each of its modules may be
is the remaining recovery ratio for each block. The effective
white (0), black (1), or transparent (t). The top and bottom
recovery ratio E is thus computed as the minimum of the
layers are aligned along the y-axis but symmetrically offset
remaining recovery ratios over all blocks of the two viewing
by 0.5 units along the x-axis (where the width of a module
patterns. The remaining recovery ratio D for a block Osi is
is 1 unit). A typical case is a QR code with version 5, which
defined as:
has N = 37.  
When viewed from a carefully chosen direction from the left D(Osi , Tsi ) = ( p − k)/2 − δ(Osi , Tsi ) / p, (3)
side, the projected offset between top and bottom layers will
change exactly to 0. Similarly, when viewed from a specific where p and k respectively denote the number of codewords in
direction from the right side, the projected offset changes total, and of data codewords in this block. Here, ( p − k)/2
exactly to 1. See Figure 3. Let L t (i, j ) be the value of module is the maximum allowed number of mismatched codewords
(i, j ) in the top layer (1 ≤ i ≤ N + 1, 1 ≤ j ≤ N), and that still allows correct decoding from the QR code structure,
L b (i, j ) be the value of module (i, j ) in the bottom layer while δ(Osi , Tsi ) denotes the number of codewords in block Osi
(1 ≤ i, j ≤ N). The left and right viewing patterns, Ol and which mismatch those in block Tsi (a codeword consists of 8
Or respectively, can be computed to be: modules: if any of its modules mismatches, then the codeword
 mismatches). The remaining recovery ratio D represents the
L b (i, j ) if L t (i, j ) = t (transparent) remaining error capacity of a block.
Ol (i, j ) =
L t (i, j ) otherwise, Formally, we may now consider generation of a two-layer
 QR code as an optimization problem. Given two target patterns
L b (i, j ) if L t (i + 1, j ) = t (transparent) Tl and Tr , the goal is to find top and bottom layer patterns L t
Or (i, j ) =
L t (i + 1, j ) otherwise. and L b by minimizing an objective function:
(1)
min −E, (4)
L t ,L b
Without further constraints, the left and right viewing patterns
are not necessarily valid QR codes. In Section V, we describe where E is the effective recovery ratio defined in Equation 2.
a generation algorithm for two-layer QR codes which ensures It is computed by comparing the module values in target
both left and right viewing patterns are valid QR codes. patterns (Tl and Tr ) with those in the viewing patterns (Ol
An example of a generated two-layer QR code is shown and Or ), which are computed from the top and bottom layer
in Figure 4. patterns (L t and L b ) according to Equation 1.
YUAN et al.: TWO-LAYER QR CODES 4417

TABLE I
M ODULE VALUES W ITH D IFFERENT C OMBINATIONS OF VALUES
FOR L t AND L b W HEN Tl (i, j) = Tr (i, j) = 1

Fig. 5. Fixed modules and segments.

Equation 4 represents a combinatorial optimization problem.

Since the top layer L t and the bottom layer L b contain N(N +
1) and N 2 modules, with 3 and 2 possible module values,
2
respectively, the size of the solution space is 2 N 3 N(N+1) ,
which is far too large to explore exhaustively. We need a more
efficient approach.
Recall the definitions of the effective recovery ratio E and
remaining recovery ratio D in Equations 2 and 3. They are
computed from the set of all mismatched codewords in the
viewing patterns, which we call the full mismatched codeword
set. We observe that E can be computed in two steps, firstly
obtaining the full mismatched codeword set by comparing
module values of viewing and target patterns, and secondly,
computing E from the full mismatched codeword set. We
will show how to make use of this separation property in the L b (i, j ) may be fixed as L b (i, j ) = 1, without further
following subsections. optimization.
We determine all fixed modules and fix their values accord-
B. Fixed Modules and Segments ingly.
We next introduce concepts of fixed modules, interleaved 2) Segments: Consider the j -th rows of L t and L b . We may
rows, segments, and partial mismatched codeword sets. Fixed interleave their modules to form an interleaved row:
modules are modules whose value can be directly set without {L t (0, j ), L b (0, j ), L t (1, j ), . . . , L b (N − 1, j ), L t (N, j )},
optimization. An interleaved row is formed by interleaving
modules in a row of the top layer with those in the same in which top layer modules and bottom layer modules alter-
row of the bottom layer. A segment is an array of contiguous nate. The value of a viewing pattern module is determined
modules in an interleaved row terminated by fixed modules. by two neighboring modules in an interleaved row (i.e., one
See Figure 5. Each segment may contain mismatched modules, top layer module and one bottom layer module). Therefore,
resulting in mismatched codewords, which are grouped into after dividing the interleaved row into segments, we could
the partial mismatched codeword set for the segment. The determine each module value in the viewing pattern from
full mismatched codeword set is the union of the partial just one corresponding segment instead of the whole row.
mismatched codeword sets of all segments. This allows us to analyze the reliability of each segment
We now explain how to compute all of the above. separately.
1) Fixed Modules: We observe that there are two cases 3) Partial Mismatched Codeword Set: Each segment may
when we can directly find the values of some modules, contain mismatched modules, resulting in mismatched code-
the fixed modules, in L t and L b without the need for opti- words. These form its partial mismatched codeword set.
mization. These two cases are: We obtain the full mismatched codeword set by computing the
Case 1: If Tl (i, j ) = Tr (i, j ) = c (where c = 0 or 1), union of the partial mismatched codeword sets of all segments.
L b (i, j ) can be fixed likewise: L b (i, j ) = c.
Case 2: if Tl (i, j ) = Tr (i − 1, j ) = c, L t (i, j ) can be fixed
likewise: L t (i, j ) = c. C. Two-Step Optimization
Proof: Consider Case 1 (the proof for Case 2 is similar). By making use of segments, the optimization problem in
Without loss of generality, suppose Tl (i, j ) = Tr (i, j ) = 1. Equation 4 can be solved using a two-step scheme.
According to Equation 1, the values of Ol (i, j ) and Or (i, j ) 1) Candidate Family: A candidate family is the set con-
(and these two modules only) depend on the value of L b (i, j ). taining all possible partial mismatched codeword sets for a
In Table I, we provide the values of Ol (i, j ), Or (i, j ) for segment (a set of sets). In the first step, we obtain a candidate
different combinations of values for L t and L b . The viewing family for each segment, by iterating through all combinations
patterns should have the same values as the target patterns of a segment’s module values. Note that different combinations
wherever possible, i.e. we desire Ol (i, j ) = Tl (i, j ) and of module values may result in duplicate partial mismatched
Or (i, j ) = Tr (i, j ). It is clear that in all cases, setting codeword sets, which lead to the same full mismatched
L b (i, j ) = 1 is always better than L b (i, j ) = 0. Hence, codeword set, and thus the same effective recovery ratio E.
4418 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 9, SEPTEMBER 2019

Hence, in a candidate family, we only store unique partial 1) Reduce Operator: Given a candidate family S, let P1
mismatched codeword sets, and remove any duplicates. and P2 be two different partial mismatched codeword sets
Below we show an example of computing a candidate in S. If P1  P2 , P2 is a redundant partial mismatched
family. Consider a very simple segment A with two modules, codeword set in S, since choosing P1 would always give
one from the top layer and one from the bottom layer. fewer mismatches than choosing P2 . Hence P2 can safely
There are 6 possible combinations of module values: {c1 , c2 } be ignored. Reducing a candidate family means remov-
(c1 ∈ [0, 1, t], c2 ∈ [0, 1]). Suppose the partial mismatched ing all such redundant mismatched codeword sets from
codeword set for each value combination is as below: it. An example of reducing a candidate family is given
below:
P({0, 0}): {Cl1 }, P({1, 0}): {Cr2 }, P({t, 0}): {Cl1 },
reduce
P({0, 1}): {Cl1 }, P({1, 1}): {Cr2 }, P({t, 1}): ∅, S = {{Cl1 }, {Cl1 , Cr2 }} −−−→ S = {{Cl1 }}.

where P(·) is the partial mismatched codeword set for a Reducing a candidate family decreases its size and hence the
specific combination of module values, Cli is the i -th codeword size of the solution space.
j
of the left target, and Cr is the j -th codeword of the right 2) Merge Operator: The merge operator takes two candi-
target; P({0, 0}) = {Cl1 } means that setting the module values date families as input and combines them into one candidate
of the segment to {0, 0} causes one codeword Cl1 to be family. Given two candidate families S1 and S2 , the merged
mismatched. The candidate family S for segment A is, after candidate family S is given by:
removing duplicates: S = {P1 ∪ P2 |P1 ∈ S1 , P2 ∈ S2 }.
S = {∅, {Cl1 }, {Cr2 }}. At a first glance, the merge operator does not seem to decrease
the size of the solution space, since it converts two candidate
This candidate family is a set of 3 different partial mismatched
families (S1 with size m 1 and S2 with m 2 ) into a candidate
codeword sets.
family with size m 1 m 2 . However, the merged candidate family
We compute the candidate families for all segments accord-
may possibly be further reduced. For example, consider two
ingly.
candidates S1 and S2 both with size 2:
2) Optimization on Candidate Families: In the second step,
we solve the optimization problem in Equation 4. Instead of S1 = {{Cl1 }, {Cr2 }}, S2 = {{Cl1 }, {Cr3 }}. (7)
simultaneously determining values for all modules in L t and
2
L b , a problem of size 2 N 3 N(N+1) , we operate on segments, The merged candidate family S is:
and aim to determine a combination of partial mismatched S = {{Cl1 }, {Cl1 , Cr2 }}, {Cl1 , Cr3 }, {Cr2 , Cr3 }}.
codeword sets, i.e. to select one partial mismatched codeword
set for each segment from its candidate family. These give the Its size is 4 but it can be decreased to 2 by applying the reduce
full mismatched codeword set and hence the effective recovery operator:
ratio E. reduce
S −−−→ S = {{Cl1 }, {Cr2 , Cr3 }}.
Let u be the number of segments, Ai be the i -th segment
(1 ≤ i ≤ u), Si be its candidate family as computed in the first For convenience, we define two candidate families to be
step, and Si  be the size of Si . The optimization problem in mergeable if at least one identical codeword is contained in
Equation 4 is reduced to: both. For example, S1 and S2 in Equation 7 both contain
min −E, (5) the same codeword Cl1 , so S1 and S2 are mergeable. If two
s candidate families are not mergeable, there is no benefit in
where s = {s1 , . . . , su } is an unknown index vector to merging them.
be found, and si gives the index of the selected partial 3) Merge and Reduce Procedure: We now show how to
mismatched codeword set from the i -th candidate family Si systematically make use of the merge and reduce operators,
(1 ≤ si ≤ Si ). starting from the initial set of all candidate families computed
For each candidate family Si , there are Si  choices for the as in Section V-C, i.e.,  = {S1 , . . . , Su }.
partial mismatched codeword set. Hence, the complexity of We repeatedly apply reduce and merge operators to the
the solution space of the problem in Equation 5 is: candidate families in , as follows:
 • Reduce all candidate families in .
Si . (6) • Repeat
1≤i≤u
– Shuffle the ordering of all candidate families in .
This two-step scheme greatly reduces the size of the solution – For each mergeable pair S1 , S2 of candidate families
space. ∗ If S1  × S2  < threshold m then
· S ← Merge(S1, S2 )
D. Merge and Reduce · Remove S1 and S2 from 
To further reduce the solution space and accelerate opti- · S ← Reduce(S)
mization, we introduce two operators on candidate families: · Add S to 
the reduce operator and the merge operator. · Goto Repeat
YUAN et al.: TWO-LAYER QR CODES 4419

∗ End If 
where E old 
and E new are the energies before and after
– End For transition, respectively; T is the temperature. Iteration stops
• Until there are no more pairs in  that can be merged.
when there has been no further energy decrease for the last
140,000 iterations (i.e. the temperature has decayed by about
A naive implementation of enumerating over all mergeable 10−3 ). Note that E  is only used in the acceptance probability
pairs of candidate families takes u(u − 1)/2 pairwise tests, function. We record the solutions in all iterations and adopt
where u is the number of candidate families. We improve the the solution with the highest actual energy E.
algorithm performance by building an index from codewords 2) Module Determination: Having obtained the partial mis-
to candidate families, which quickly allows us to find all matched codeword set for each candidate family, the next
mergeable pairs. Furthermore, the merge size threshold m is step is to determine corresponding module values. For each
used to avoid very large candidate families: applying reduce segment, there may be multiple module value combinations
and merge operators to a very large candidate family may even which result in the same partial mismatched codeword set. We
reduce overall performance. In our implementation, the merge prefer the combination with the highest number of transparent
size threshold is set to m = 300. We will evaluate different modules on the top layer, because more transparent modules
choices of m in Section VII-A1. will lead to fewer occlusions, producing fewer shadows and
potentially fewer scanning errors.
E. Simulated Annealing After determining all module values for the segments,
we further change the values of fixed modules on the top
The solution space of the problem in Equation 5 is still too
layer to transparent if doing so does not introduce additional
large to be fully explored even after the merge and reduce
mismatched codewords.
procedure. To quickly obtain an approximate solution, we use
3) Data Masking Patterns: The above process is carried
simulated annealing [37].
out 8 times for the 8 different data masking patterns, and
1) Simulated Annealing: We aim to simultaneously max-
the solution with the highest recovery ratio from all 8 data
imize the minimum of the remaining recovery ratio D for
masking patterns is selected; it is done in parallel using
all blocks. Optimization techniques like simulated annealing
8 threads.
which implicitly rely on the use of derivatives E are unlikely
to work well in this case, since E does not necessarily change
if only small changes are made to the index vector s. Hence, F. Beautification
we slightly modified the optimization target in Equation 5 to: Two-layer QR codes also support beautification, e.g. to add
a given logo. Beautification is done during preprocessing.
min −E  , (8) Given a reference image, we first beautify the input left and
s
right target patterns using an existing QR code beautification
where E  is defined as: method [20]. We then optimize the top and bottom layers as
 usual.
E  = min D(Osi , Tsi ) + λ D(Osi , Tsi ). (9)
i,s However, since we use the same reference image to beautify
i,s
the left and right target patterns, many modules in the left and
E  is composed of two terms. The first term is just E: the right target patterns are identical, resulting in a large number
minimum of the remaining recovery ratio D for all blocks. of fixed modules. As a result, beautification reduces the search
The second term is the sum of the remaining recovery ratio space and also potentially increases the recovery ratio.
D for all blocks.
We incorporate the second term to improve the derivative VI. R ESULTS AND FABRICATION
properties of the overall optimization target, making it more A. Results
suitable for simulated annealing. Recall that our target is
to maximize E (minimize −E) as given in Equation 5. We tested our method on a PC with a 3.5 GHz Intel Core i7-
Maximizing the second term (by definition, the sum of all 5930K CPU and 32 GB RAM. We implemented our algorithm
D) shares a similar goal to maximizing E (by definition, in C++. All two-layer QR codes shown in the paper were
the minimum of all D): a small sum cannot, in general, give a generated within a few seconds; the time needed to generate
large minimum. In practice, we set the weight parameter λ to and beautify the input target patterns was less than 5 ms and
a small value (λ = 0.1) to ensure that the second term makes can be ignored.
only a small contribution, and E  is still dominated by E. 1) Diverse Examples: To show the robustness of our algo-
We start with a random choice for s. On each iteration, rithm in handling diverse examples, we generated 12 examples
a neighborhood is computed by reassigning the indices of of two-layer QR codes: see Figures 6 and 7. Their versions and
the selected mismatched codeword set for several candidate EC levels range from (2, M) to (17, H ); 6 of the 12 examples
families (i.e. we change the values of si for some i , 1 ≤ i ≤ u). are beautified.
The temperature is initialized to 1.0 and decays by 0.999 every We show 4 examples in Figure 6. Messages are encoded in
20 iterations. The acceptance probability function is set to: alphanumeric mode. The top two rows of Figure 6 show two
   examples encoding the same messages, with different versions
 
E new − E old and different EC levels. As EC level increases, the results
min 1, exp ,
T become more reliable with larger effective recovery ratio, but
4420 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 9, SEPTEMBER 2019

Fig. 6. Four examples of two-layer QR codes with different versions and different EC levels. Left to right: version, EC level, string lengths of the two
encoded messages, generated bottom and top layers, left and right views, and the effective recovery ratio. Messages are encoded in alphanumeric mode.
Messages encoded in the top 2 rows: left view: “A B C D E F G H I J K L M N”, right view: “O P Q R S T U V W X Y Z”. Messages encoded in the
bottom 2 rows: left view: “HTTP://WWW.TSINGHUA.EDU.CN/”, right view: “TSINGHUA UNIVERSITY. BEIJING. CHINA”.

a larger version value is required to encode the message. As explained in Section V-A, our generation algorithm
However, a larger version value requires smaller modules (for restricts the left and right views to have the same EC levels and
the same overall physical size of QR code), which makes the masks. However, Our algorithm could be slightly modified to
codes harder to scan reliably. There is thus a trade-off between support different EC levels, and different masks, by taking
the EC level and version. format information into account. To do so, care must be
The bottom two rows of Figure 6 provide two more taken in dealing with format information, since differences
beautified examples encoding the same messages, with the in EC levels and masks can introduce errors in format
same versions but different EC levels. It shows that better information.
reliability can also be achieved for beautified examples by To achieve this, we slightly modify our algorithm in two
increasing EC level. However, a higher EC level requires more ways. Firstly, we add a precomputation stage. For all combi-
error correction codewords, resulting in fewer modules being nations of different EC levels and different masks (42 × 82 =
available to meet beautification design requirements (if we 1024 combinations in total), we precompute the optimal values
keep the version value unchanged). Therefore, an appropriate for those top and bottom layer modules which affect format
EC level should be chosen to balance aesthetics and reliability. information, using exhaustive search. For each combination,
Further 8 examples are shown in Figure 7. Here, the exam- the associated module values which give the fewest error
ples are encoded in byte mode. Our method can robustly bits are considered as optimal and kept. These precomputed
handle these diverse examples. module values are considered as fixed modules and their values
2) Different EC Levels and Different Data Masking Pat- are not changed during later optimization. Secondly, during
terns: The EC level and data masking pattern (short as ‘mask’) the run-time stage, after EC levels are provided by users,
of a QR code are stored in the format information. As shown we need to determine which combination to use. We iterate
in Figure 1, format information is located next to the finder over combinations of the selected EC levels and all masks.
patterns. It contains 15 bits of data and can tolerate up to 3 bit For each combination, we run the optimization and obtain
errors. Format information is identified by a scanner before an effective recovery ratio. The combination with the highest
message decoding, and its correctness is important. effective recovery ratio is selected. If multiple combinations
YUAN et al.: TWO-LAYER QR CODES 4421

Fig. 7. Further examples of two-layer QR codes. Left to right: version, EC level, string lengths of the two encoded messages, generated bottom and top
layers, left and right views, and the effective recovery ratio. Messages are encoded in byte mode.
4422 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 9, SEPTEMBER 2019

Fig. 8. Example whose views have different EC levels. EC level: H (left view), M (right view). Messages are encoded in byte mode. Left:
“https://www.iso.org/standard/62021.html”, right: “Information technology -- Automatic identification and data capture techniques -- QR Code bar code
symbology specification”. Time taken: 0.635 s.

Fig. 9. Examples whose views have (top row) the same data masks (pattern 010), and (bottom row) different masks (left: 100, right: 011). Both encode the
same messages, randomly generated strings of length 50, in alphanumeric mode. Time taken: 0.412 s (same masks), 2.726 s (different masks).

result in the same effective recovery ratio, the one with the To assess scanning robustness, we conducted a synthetic
fewest error bits in format information is selected. scanning comparison between the different mask scheme and
Figure 8 shows an example using different EC levels. In this the same mask scheme. See Figure 18. The success rates of
example, the EC levels of the left and right views are user- these two schemes were nearly the same.
specified to be H and M, respectively, to allow different error We believe the reasons are twofold. Firstly, the padding
correction capacities. The resulting effective recovery ratio is codewords are probably the same when we apply the same
2/39 (left 2/39, right 3/49). mask scheme, while different masks may cause potential
To allow left and right views to have different masks, conflicts in padding codewords. Secondly, different masks may
we have to iterate over 8 × 8 combinations of masks and make scanning less robust because the use of different masks
select the combination with the highest effective recovery may cause errors in the format information.
ratio. Using such a different mask scheme thus requires approx- Overall, since the different mask scheme has limited benefits
imately 8× as much computation as the original same mask but greatly increases the computation required, we suggest
scheme described in Section V-E3, where both views use the using the same masks for both left and right QR codes. Unless
same mask. explicitly stated otherwise, all results in the paper thus use the
Figure 9 (bottom) shows an example of using different same mask scheme.
masks, while the same masks are used in Figure 9 (top); other
settings and encoded messages are unchanged. In this example,
while the result with different masks achieves a better effective B. Fabrication
recovery ratio (3/25) than when using the same mask (2/25), 1) Manufacture: We now consider the fabrication of a real
it additionally introduces 1 bit error in format information. two-layer QR code. We print the top layer pattern on a
We tested the different mask scheme described above for transparent plastic sheet and the bottom layer pattern on paper.
all 14 examples in Figures 6–9. Perhaps surprisingly, except We then paste these two patterns onto the two sides of a
for one example in Figure 9, the different mask scheme transparent acrylic plate with a thickness of h = 3 mm. We set
performed no better than the same mask scheme, i.e. the the width of the top layer modules to wt = 1.5 mm.
optimal combination turned out to be a pair of identical An example of a fabricated two layer QR code is shown
masks. in Figure 10.
YUAN et al.: TWO-LAYER QR CODES 4423

Fig. 10. A fabricated two-layer QR code. Fig. 12. Synthetic images of two-layer QR codes with corrected (left)
and uncorrected bottom module widths (right). Both are taken from the
optimal camera directions. Left: the bottom module width is computed using
Equation 11. Right: the bottom module width is set equal to the top module
width. The modules in the left image are better aligned.

where n and h are the refractive index and thickness of

the transparent acrylic plate, respectively. Given the camera
direction, for best alignment between the top and bottom
y
layers, the widths of bottom layer modules (wbx and wb ) are
given by:
⎛   2 3/2 ⎞
h 2 + wbx /2 cos2 θ
⎜ ⎟
wbx = wt ⎝1 + √ ⎠,
nd h
Fig. 11. Camera position.  
y wbx
wb = wt 1 + . (11)
2) Module Widths: Intuitively, it is clear that the bottom 2d sin θ
layer modules should be a little larger than the top layer
modules to achieve the best alignment, due to the foreshort- While Equations 10 and 11 show how to compute the optimal
y
ening effect of perspective projection. We next consider how θ , wbx and wb , they are nonlinear. We initialize wbx = wt =
to set the size of the bottom layer modules in terms of camera 1.5 mm and compute these values iteratively. The resulting
distance and position. optimal values are: θ ≈ ±21.57◦ for version 5 QR code, and
Figure 11 illustrates a typical use case and the camera θ ≈ ±21.49◦ for version 10 QR code.
position. Suppose the field of view angle of the camera is Figure 12 shows two synthetic images of two-layer QR
30◦ , and it has an aspect ratio of 1:1. Let the distance from the codes. The left image takes perspective projection into account
camera to the center of the top layer pattern be d, the width of and sets the bottom module width according to Equation 11
the top layer pattern be w, and the polar and azimuthal angles (referred to as corrected bottom module width), while the
of the camera direction be θ and φ, respectively. We denote the right image ignores perspective projection and simply sets
refractive index of the acrylic plate by n, which has a typical bottom module width equal to top module width (referred to as
value of n = 1.5. Since the QR code pattern usually covers uncorrected bottom module width). The close-ups in Figure 12
the majority of the captured image when scanning, the camera help to verify that the modules in the left image are better
distance d is likely to satisfy 2.4w ≤ d ≤ 3.6w, and so we aligned.
may assume d = 3w. (In computing module widths, without 3) Scanning Time: We printed and fabricated five different
loss of generality, we also assume the camera is to the right two-layer QR codes. We then invited 5 users to scan the fab-
of the QR plate, i.e. θ > 0) ricated two-layer QR codes with 4 widely used mobile phone
The best camera direction corresponds to a viewing QR code reader applications (NeoReader, ZXing, QR Droid,
direction in which the central point of a top layer mod- and i-nigma). The mobile phone used was a MEIZU PRO
ule just occludes the central point of the corresponding 6s, running Android 7.1.1 (Flyme 6.3.0.2A). Each user was
bottom layer module. Given the widths of the bottom asked to scan each code twice: the user was asked to align
layer modules along x- and y-axes, respectively, denoted the camera with the center of the two-layer QR code first, and
y then turn left (first scan) or right (second scan) to scan the
wbx , wb , the best camera direction is (see the supplemen-
tary document for detailed derivations of formulae in this two-layer QR code. The scanning process was recorded by a
section): screen capture plugin, and the scanning interaction time for
⎛ ⎞ each view was also recorded. In the experiment, the two-layer
n(wbx /2) QR codes were scanned successfully and decoded correctly in
θ = arcsin ⎝  ⎠
 x 2 , φ = 0 ,
◦
(10) every case. Figure 13 gives, for each example, for each reader
h + wb /2
2
app, the median scanning times for the 5 users. The results
4424 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 9, SEPTEMBER 2019

TABLE II
E VALUATION OF M ERGE AND R EDUCE O PERATORS (T OP 7 ROWS ), AND E VALUATION OF M ODIFIED O PTIMIZATION TARGET (B OTTOM 4 ROWS ).
W E G IVE A CCEPTABILITY R ATE (A BOVE ) AND AVERAGE RUN T IME (B ELOW, IN S ECONDS ), FOR E ACH E XAMPLE ,
FOR E ACH S ETTING . ‘*’ R EFERS TO A B EAUTIFIED T WO -L AYER QR C ODE

Fig. 13. Median scanning times using each reader app for 5 fabricated
examples with varying parameters of version and EC level. ‘*’ refers to a Fig. 14. Two synthetic images of the same two-layer QR code, rendered
beautified two-layer QR code. from different camera directions with different noise levels. Left: θ = −15◦ ,
φ = 2◦ , σ = 32, d = 2.4w, right: θ = 18◦ , φ = −10◦ , σ = 16, d = 2.4w.
demonstrate that our fabricated two-layer QR codes can easily
be scanned in practice.
The features include the merge and reduce operators, and the
The supplementary video illustrates the fabricated examples
parameters are the merge size threshold m and the weight λ
and the scanning process.
in simulated annealing.
4) Instructions for Users: Given familiarity with standard
For each experiment, for each example in Figures 6 and 7,
QR codes, users might at first try to scan two-layer QR
we ran our algorithm 100 times. The run time and effective
codes from the front. Therefore, we recommend placing clear
recovery ratio were recorded. In each case, if the effective
guidance besides such new QR codes to explain how to use
recovery ratio was not less than the desired effective recov-
them. For example, a two-layer QR code describing an exhibit
ery ratio (i.e., the median effective recovery ratios over all
in a museum could be accompanied by text labels and arrows
experiments for an example), we considered the output to
saying ‘English (scan this way)’ pointing from the left, and
be acceptable. Finally, we give the acceptability rate and
‘Chinese (scan this way)’ pointing from the right.
average run time for each example for a given experimental
setup. In some cases, no additional merging occurred on
VII. E VALUATION
increasing the threshold m, so the acceptability rate and
A. Parameter Evaluation run time remained unchanged. We mark these cases as
In this section, we evaluate how features and parameters ‘same’.
in our algorithm affect correction capability and run time. Next, we explain each experiment in detail.
YUAN et al.: TWO-LAYER QR CODES 4425

Fig. 15. Scanning robustness with respect to camera direction. Orange curves: success rate (η) versus camera direction for two-layer QR codes. Blue curves:
success rate for standard QR codes for comparison.

1) Merge and Reduce Operators: Table II (top 7 rows) show original optimization target (λ = 0) for complex examples.
results concerning the merge and reduce process, including: However, for λ > 0, we observe that increasing λ improves
whether or not merge operators and reduce operators were computational efficiency but reduces the acceptability rate.
used, and the value of the merge size threshold m. We suggest setting λ = 0.1 provides a good trade-off.
The results indicate that use of reduce and merge operators
significantly increases the acceptability rate and decreases
run time. Without reduce and merge operators, it is hard to B. Robustness for Arbitrary Input Messages
generate acceptable results for complex examples (see Table II We conducted an experiment to evaluate the robustness of
(rows 1 and 2)). For the merge size threshold, m = 300 is our algorithm for arbitrary input messages. We tested different
a reasonable compromise, since it achieves a nearly optimal settings of version (1, 5, 10, 15, and 20) and EC level (L, M,
acceptability rate, and larger m does little but increase run Q, and H ).
time. In fact, the results show that varying m has little effect on For each parameter setting, we first generated 1,000 pairs
the algorithm. Varying m between 30 and 3000 always results of random messages in alphanumeric mode with maximal
in an acceptability rate over 99%, while run time changes by encodable message length for this setting. We then ran our
under 10%. There is little to be gained by adjusting m for algorithm. The generation was considered to be successful
different cases. if the effective recovery ratio E was non-negative when
2) Simulated Annealing Process: Table II (bottom 4 rows) simulated annealing terminated.
shows the effect of changing the simulated annealing weight Table III shows the success rate of our algorithm and
parameter λ in the modified optimization target (Equation 9). the corresponding message length for each parameter setting.
The results show that using the modified optimization target It demonstrates that our algorithm is robust for arbitrary input
(λ > 0) leads to much larger acceptability rates than the messages if version and EC level are chosen appropriately.
4426 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 9, SEPTEMBER 2019

Fig. 16. Scanning robustness with respect to camera distance. Orange curves: success rate (η) versus camera distance for two-layer QR codes. Blue curves:
success rate for standard QR codes for comparison.

Fig. 17. Scanning robustness with respect to bottom module width. Orange curves: success rate for two-layer QR codes with corrected bottom module
width computed by Equation 11. Green curves: success rate for two-layer QR codes with uncorrected bottom module width (equal to top module width). Blue
curves: success rate for standard QR codes.

TABLE III the transparent plate, respectively. We assume the plate has a
ROBUSTNESS FOR A RBITRARY I NPUT M ESSAGES : S UCCESS R ATE OF transmittance of 0.75. Figure 14 shows two synthetic images
O UR A LGORITHM , AND C ORRESPONDING M AXIMAL A LPHANUMERIC
M ODE M ESSAGE L ENGTH FOR D IFFERENT S ETTINGS
of the same code rendered from different camera directions
OF V ERSION AND EC L EVEL with different noise levels.
To test scanning robustness, we conducted experiments with
two settings of version and EC level: (5, H ) and (10, H ).
For each setting, we generated one thousand two-layer QR
codes encoding randomly generated messages with appropriate
length. We used the ray tracer to render synthetic images of
these two-layer QR codes from different camera directions
with different noise levels. The synthetic images were rendered
with a resolution of 960×960 pixels, and were resized to 5 dif-
Our algorithm is always successful for EC levels Q and H . ferent resolutions (160, 240, 320, 400 and 480 pixels) using
In contrast, the success rate is low for EC levels L and M, bilinear interpolation. Scanning was considered successful if
which provide smaller error correction capability. The table one of the 5 resized images could be correctly decoded by the
suggests that it is more robust to use EC levels Q or H . open-source QR code reader ZXing.1
We provide some sample two-layer QR code examples 1) Camera Direction: Figure 15 shows how the success
which encode randomly generated messages in the supple- rate changes with camera direction. Rows 1 and 3 show
mentary document. how success rate η varies with polar angle θ , while rows
2 and 4 show how success rate η varies with azimuthal
angle φ. When computing success rate η for a polar angle
C. Scanning Robustness
θ , we randomly selected azimuthal angle φ ∈ [−2◦ , 2◦ ]
We implemented a simple ray tracer to render synthetic two- accordingly to a uniform distribution. Similarly, when
layer QR codes. Gaussian noise with standard deviation σ
was added to each pixel on the top layer, bottom layer, and 1 ZXing: https://code.google.com/p/zxing
YUAN et al.: TWO-LAYER QR CODES 4427

Fig. 18. Scanning robustness with respect to mask schemes. Blue curves: success rate (η) for two-layer QR codes generated using the same mask scheme.
Orange curves: success rate (η) for two-layer QR codes generated using the different mask scheme. Their scanning success rates are almost identical.

computing η for an azimuthal angle φ, the polar angle θ was VIII. C ONCLUSION
randomly sampled from ±[20◦, 22◦ ]. In both cases, d was
randomly sampled from [2.4w, 3.6w]. The results show that This paper has presented two-layer QR codes which can
encode two independent messages. They can be decoded
scanning from directions with polar angle θ ∈ ±[10◦ , 30◦ ]
separately by changing scanning directions. We have given
and azimuthal angle [−15◦, 15◦ ] is relatively successful.
an automatic algorithm for generating such codes, and demon-
2) Camera Distance: Figure 16 shows how the success strated the robustness of the proposed approach. Beautification
rate changes with camera distance for different noise levels. can be used with these codes, as for standard QR codes.
When computing success rate η for a camera distance d, In future, we hope to extend this method to allow more than
we randomly selected polar angle θ ∈ ±[20◦, 22◦ ] and two messages to be encoded in such a specially designed
azimuthal angle φ ∈ [−2◦ , 2◦ ]. The results indicate that two-layer structure, and to incorporate more types of code
restricting camera distance d ∈ [2.4w, 3.6w] is a good choice beautification techniques.
for scanning in the presence of noise (see Figure 16, right two
columns). As d further increases, the success rate drops as the
QR code becomes smaller and harder to recognize. R EFERENCES
3) Bottom Module Width: In Figure 17, we plot success [1] I. Tkachenko, W. Puech, C. Destruel, O. Strauss, J.-M. Gaudin, and
rate η against polar angle θ and azimuthal angle φ, using the C. Guichard, “Two-level QR code for private message sharing and
document authentication,” IEEE Trans. Inf. Forensics Security, vol. 11,
corrected bottom module width (orange curve) and uncorrected no. 3, pp. 571–583, Mar. 2016.
bottom module width (green curve), respectively. The results [2] Hotapp: ‘Zhima’ QR Code Generator. Accessed: Apr. 16, 2019.
show that using corrected bottom module width (Equation 11) [Online]. Available: https://www.hotapp.cn/
[3] Lenticular Printing. Accessed: Apr. 16, 2019.
increases success rate and improves scanning robustness. [Online]. Available: https://en.wikipedia.org/wiki/
In all plots of Figures 15–17, we also provide scanning Lenticular_printing
success rates for standard QR codes (blue curves). Compared [4] A. Oliva, A. Torralba, and P. G. Schyns, “Hybrid images,” ACM
to standard QR codes, two-layer QR codes have a lower Trans. Graph., vol. 25, no. 3, pp. 527–532, Jul. 2006. doi: 10.1145/
1141911.1141919.
success rate due to damage introduced during the generation [5] N. J. Mitra and M. Pauly, “Shadow art,” ACM Trans. Graph., vol. 28,
process. However, they still retain a high success rate if camera no. 5, pp. 156:1–156:7, Dec. 2009. doi: 10.1145/1618452.1618502.
distance and direction are appropriately set. [6] S. Ono, K. Morinaga, and S. Nakayama, “Two-dimensional barcode
decoration based on real-coded genetic algorithm,” in Proc. IEEE Congr.
4) Same Versus Different Masks: While the different mask Evol. Comput. (CEC), IEEE World Congr. Comput. Intell., Jun. 2008,
pp. 1068–1073.
scheme described in Section VI-A2 can slightly increase
[7] S. Ono, K. Morinaga, and S. Nakayama, “Animated two-dimensional
the effective recovery ratio, it can also introduce errors in barcode generation using optimization algorithms—Redesign of formu-
format information. To demonstrate how it affects scanning lation, operator, and quality evaluation,” J. Adv. Comput. Intell. Intell.
robustness overall, in Figure 18, we plot success rate η against Inform., vol. 13, no. 3, pp. 245–254, 2009.
[8] S. Ono and S. Nakayama, “Fusion of interactive and non-interactive
polar angle θ using the different mask scheme (orange curve) evolutionary computation for two-dimensional barcode decoration,” in
and the same mask scheme (blue curve), respectively. Their Proc. IEEE Congr. Evol. Comput., Jul. 2010, pp. 1–8.
scanning success rates are almost identical. Since the different [9] M. Kamizono, K. Shimomura, M. Tajiri, and S. Ono, “Two-dimensional
barcode decoration using module-wise non-systematic coding and coop-
mask scheme brings few benefits but requires 8 times the erative evolution by user and system,” in Proc. ACM Genetic Evol.
computation, we prefer the same mask scheme. Comput. Conf., 2016, pp. 245–252.
4428 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 9, SEPTEMBER 2019

[10] D. Samretwit and T. Wakahara, “Measurement of reading characteristics [36] Information Technology—Automatic Identification and Data Cap-
of multiplexed image in QR code,” in Proc. 3rd Int. Conf. Intell. Netw. ture Techniques—QR Code Bar Code Symbology Specification, Stan-
Collaborative Syst., Nov./Dec. 2011, pp. 552–557. dard ISO/IEC 18004:2015, 2015, pp. 1–117. [Online]. Available:
[11] Z. Baharav and R. Kakarala, “Visually significant QR codes: Image https://www.iso.org/standard/62021.html
blending and statistical analysis,” in Proc. IEEE Int. Conf. Multimedia [37] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by
Expo (ICME), Jul. 2013, pp. 1–6. simulated annealing,” Science, vol. 220, no. 4598, pp. 671–680, 1983.
[12] H.-K. Chu, C.-S. Chang, R.-R. Lee, and N. J. Mitra, “Halftone QR
codes,” ACM Trans. Graph., vol. 32, no. 6, pp. 217:1–217:8, Nov. 2013.
[13] Y.-S. Lin, S.-J. Luo, and B.-Y. Chen, “Artistic QR code embellishment,”
Comput. Graph. Forum, vol. 32, no. 7, pp. 137–146, Oct. 2013. Tailing Yuan received the bachelor’s degree from
[14] Z. Gao, G. Zhai, and C. Hu, “The invisible QR code,” in Proc. 23rd Tsinghua University in 2016, where he is currently
ACM Int. Conf. Multimedia, 2015, pp. 1047–1050. pursuing the Ph.D. degree with the Department of
[15] G. J. Garateguy, G. R. Arce, D. L. Lau, and O. P. Villarreal, “QR images: Computer Science and Technology. His research
Optimized image embedding in QR codes,” IEEE Trans. Image Process., interests include computer graphics and computer
vol. 23, no. 7, pp. 2842–2853, Jul. 2014. vision.
[16] C. Fang, C. Zhang, and E.-C. Chang, “An optimization model for
aesthetic two-dimensional barcodes,” in Proc. 20th Int. Conf. Multimedia
Modeling, Jan. 2014, pp. 278–290.
[17] S. Wang, T. Yang, J. Li, B. Yao, and Y. Zhang, “Does a QR code
must be black and white?” in Proc. Int. Conf. Orange Technol. (ICOT),
Dec. 2015, pp. 161–164.
[18] A. Mittal, “Generating visually appealing QR codes using colour image
embedding,” Imag. Sci. J., vol. 65, no. 1, pp. 1–13, 2017.
[19] L. Lin, S. Wu, S. Liu, and B. Jiang, “Interactive QR code beautification Yili Wang is currently a Senior Undergraduate Stu-
with full background image embedding,” in Proc. 2nd Int. Workshop dent with the Department of Foreign Languages and
Pattern Recognit., vol. 10443, 2017, Art. no. 1044317. [Online]. Avail- Literatures, Tsinghua University. She plans to pursue
able: http://spie.org/Publications/Proceedings/Volume/10443 Ph.D. degree in computer science in the future.
Her research interests include computer graphics and
[20] K. Fujita, M. Kuribayashi, and M. Morii, “Expansion of image dis-
image editing.
playable area in design QR code and its applications,” (in Japanese), in
Proc. 10th Forum Inf. Technol., vol. 4, Sep. 2011, pp. 517–520. [Online].
Available: https://www.ipsj.or.jp/event/fit/fit.html
[21] S.-S. Lin, M.-C. Hu, C.-H. Lee, and T.-Y. Lee, “Efficient QR code
beautification with high quality visual content,” IEEE Trans. Multimedia,
vol. 17, no. 9, pp. 1515–1524, Sep. 2015.
[22] Y. Zhang, S. Deng, Z. Liu, and Y. Wang, “Aesthetic QR codes based
on two-stage image blending,” in Proc. 21st Int. Conf. Multimedia
Modeling, Jan. 2015, pp. 183–194. Kun Xu received the bachelor’s and Ph.D. degrees
[23] M. Kuribayashi and M. Morii, “Enrichment of visual appearance of from Tsinghua University in 2005 and 2009, respec-
aesthetic QR code,” in Proc. 14th Int. Workshop Digit. Watermarking, tively. He is currently an Associate Professor with
Oct. 2015, pp. 220–231. the Department of Computer Science and Tech-
[24] S. Qiao, X. Fang, B. Sheng, W. Wu, and E. Wu, “Structure-aware QR nology, Tsinghua University. His research interests
code abstraction,” Vis. Comput., vol. 31, no. 6, pp. 1123–1133, Jun. 2015. include realistic rendering and image/video editing.
[25] L. Li, J. Qiu, J. Lu, and C.-C. Chang, “An aesthetic QR code solu-
tion based on error correction mechanism,” J. Syst. Softw., vol. 116,
pp. 85–94, Jun. 2016.
[26] M. Kuribayashi, E.-C. Chang, and N. Funabiki, “Watermarking with
fixed decoder for aesthetic 2D barcode,” in Proc. 15th Int. Workshop
Digit. Watermarking, Sep. 2016, pp. 379–392.
[27] M. Kuribayashi and M. Morii, “Aesthetic QR code based on modified
systematic encoding function,” IEICE Trans. Inf. Syst., vol. E100-D, Ralph R. Martin recently retired after a long career
no. 1, pp. 42–51, 2017. in visual computing. He is currently an Emeritus
[28] J.-C. Liu and H.-A. Shieh, “Toward a two-dimensional barcode with Professor with Cardiff University. He is enjoying his
visual information using perceptual shaping watermarking in mobile life in the Welsh countryside.
applications,” Opt. Eng., vol. 50, no. 1, 2011, Art. no. 017002.
[29] C. Chen, W. Huang, B. Zhou, C. Liu, and W. H. Mow, “PiCode: A new
picture-embedding 2D barcode,” IEEE Trans. Image Process., vol. 25,
no. 8, pp. 3444–3458, Aug. 2016.
[30] C. Chen, B. Zhou, and W. H. Mow, “RA code: A robust and aesthetic
code for resolution-constrained applications,” IEEE Trans. Circuits Syst.
Video Technol., vol. 28, no. 11, pp. 3300–3312, Nov. 2017.
[31] M. E. V. Melgar, A. Zaghetto, B. Macchiavello, and
A. C. A. Nascimento, “CQR codes: Colored quick-response codes,” in
Proc. IEEE 2nd Int. Conf. Consum. Electron.-Berlin (ICCE), Sep. 2012, Shi-Min Hu received the Ph.D. degree from
pp. 321–325. Zhejiang University in 1996. He is currently a Pro-
[32] H. Blasinski, O. Bulan, and G. Sharma, “Per-colorant-channel color bar- fessor with the Department of Computer Science and
codes for mobile applications: An interference cancellation framework,” Technology, Tsinghua University, Beijing. He has
IEEE Trans. Image Process., vol. 22, no. 4, pp. 1498–1511, Apr. 2013. published more than 100 papers in journals and ref-
[33] H. J. Galiyawala and K. H. Pandya, “To increase data capacity ereed conferences. His research interests include dig-
of QR code using multiplexing with color coding: An example of ital geometry processing, video processing, render-
embedding speech signal in QR code,” in Proc. IEEE Annual India ing, computer animation, and computer-aided geo-
Conf. (INDICON), Dec. 2014, pp. 1–6. metric design. He is also the Editor-in-Chief of Com-
[34] Z. Yang et al., “ARTcode: Preserve art and code in any image,” in Proc. putational Visual Media and on the Editorial Boards
Int. Joint Conf. Pervasive Ubiquitous Comput., 2016, pp. 904–915. of several journals, including the IEEE T RANSAC -
[35] I. S. Reed and G. Solomon, “Polynomial codes over certain finite fields,” TIONS ON V ISUALIZATION AND C OMPUTER G RAPHICS , Computer-Aided
J. Soc. Ind. Appl. Math., vol. 8, no. 2, pp. 300–304, Jun. 1960. Design and Computer & Graphics.