2842 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO.

7, JULY 2014

QR Images: Optimized Image

Embedding in QR Codes
Gonzalo J. Garateguy, Student Member, IEEE, Gonzalo R. Arce, Fellow, IEEE,
Daniel L. Lau, Senior Member, IEEE, and Ofelia P. Villarreal, Member, IEEE

Abstract— This paper introduces the concept of QR images, to the advantage they provide in tracking the performance of
an automatic method to embed QR codes into color images publicity campaigns.
with bounded probability of detection error. These embeddings An important problem of QR codes is its impact on the
are compatible with standard decoding applications and can
be applied to any color image with full area coverage. The aesthetics of publicity designs. The square shapes and limited
QR information bits are encoded into the luminance values of the color tolerance, severely impairs their integration into billboard
image, taking advantage of the immunity of QR readers against designs or printed materials. This challenge has generated
local luminance disturbances. To mitigate the visual distortion great interest for algorithms capable of embedding QR codes
of the QR image, the algorithm utilizes halftoning masks for into images without loosing decoding robustness. There have
the selection of modified pixels and nonlinear programming
techniques to locally optimize luminance levels. A tractable model been several efforts to improve the appearance of such embed-
for the probability of error is developed and models of the dings [4]–[15] which can be classified in two categories,
human visual system are considered in the quality metric used to methods that modify the luminance or color of image pixels
optimize the luminance levels of the QR image. To minimize the and methods that replace QR modules.
processing time, the optimization techniques proposed to consider The methods presented in [6] and [7] base the strategy on
the mechanics of a common binarization method and are designed
to be amenable for parallel implementations. Experimental finding the best group of QR modules to substitute by the
results show the graceful degradation of the decoding rate and image or logo in the QR code. Other methods take advantage
the perceptual quality as a function the embedding parameters. of unused modules in the padding regions [8]–[10] to introduce
A visual comparison between the proposed and existing methods the image without affecting the decoding robustness. In [7]
is presented. the impact of replacing code modules with image pixels was
Index Terms— QR codes, image embedding, halftoning. studied. The authors concluded that to retain high rate of
decodability, the ratio between image and code area should
be approximately proportional to the correction capacity of
I. I NTRODUCTION
the code. It was also found that superimposing images over

Q UICK response (QR) codes [1], [2] have rapidly

emerged as a widely used inventory tracking and identi-
fication method in transport, manufacturing, and retail indus-
finder or alignment patterns severely decrease the probability
of correct decoding. As a consequence it is common for logos
or images to be located at the center of the code for these
tries [3]. Their popularity is due to the proliferation of smart methods as depicted in Fig. 1(a). In general these approaches
phones, capable of decoding and accessing on line resources do not take advantage of the codeword generation process and
as well as its high storage capacity and speed of decoding. this imposes restrictions in the location of modified modules.
QR codes are used in a variety of applications, such as This problem was addressed by recently developed techniques
accessing websites, download personal card information, post [4], [11], [12] which manipulate the Reed Solomon encoding
information to social networks, initiate phone calls, reproduce procedure to maximize the area coverage without reducing the
videos or open text documents. This versatility makes them a correction capacity.
valuable tool in any industry that seeks to engage mobile users The second category of embedding algorithms is based on
from printed materials. Not surprisingly QR codes have been the modification of the pixel’s luminance. The approach in [5]
widely adopted in the marketing and publicity industry thanks chooses central pixels of each module to modify its luminance
since this is the area usually sampled by the decoder. This
Manuscript received September 6, 2013; revised December 31, 2013 and
April 14, 2014; accepted April 16, 2014. Date of publication May 2, 2014; date approach provides an adequate trade off between robustness
of current version May 20, 2014. The associate editor coordinating the review and visual distortion but the number of center pixels modified
of this manuscript and approving it for publication was Dr. Karsten Mueller. is in general a large proportion of the module area and creates
G. J. Garateguy is with Mathworks, Natick, MA 01760 USA (e-mail:
[email protected]). undesirable low pass artifacts [see Fig. 1(c)]. A method which
G. R. Arce and O. P. Villarreal are with the Department of Electrical and implements a similar idea is LogoQ [13]. In [16] a method
Computer Engineering, University of Delaware, Newark, DE 19711 USA to introduce colors into QR codes was proposed with the
(e-mail: [email protected]; [email protected]).
D. L. Lau is with the Department of Electrical and Computer Engi- goal of increasing its data capacity and a detailed study of
neering, University of Kentucky, Lexington, KY 40506 USA (e-mail: the interference between different color layers was presented.
[email protected]). Two embedding methods which are related to the method
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org. presented here are [14] and [15]. In [14] the author chooses the
Digital Object Identifier 10.1109/TIP.2014.2321501 modified pixels in the code based on the average luminance of
1057-7149 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
GARATEGUY et al.: OPTIMIZED IMAGE EMBEDDING IN QR CODES 2843

Fig. 2. QR code regions with the location of finder and alignment patterns
highlighted in red.

distortion metric subject to a constraint in the probability of

error. This algorithm can be applied to any color image and
QR code with full area coverage and bounded probability of
error. A novel contribution of this paper is the use of halftone
masks to distribute the modified pixels and the introduction
of a probabilistic model to predict the distortion generated
by the embedded image. This allows to control the trade off
between image quality and decoding robustness by setting
a few parameters such as the number of modified pixels at
Fig. 1. (a) Substitutions of QR modules by a logo, (b) QArt code the center of each QR module and the maximum allowed
designed using the method in [4], (c) luminance modification algorithm [5], probability of error. Fig. 1(d) depicts the results of embedding
J = 0.4149 (d) proposed algorithm, J = 0.3989. The distortion metric J is
defined in (16). a QR code using the method proposed in this paper which
is visually more pleasant than the embedding in Fig. 1(d)
and presents a lower visual distortion according to the metric
image pixels and two user defined thresholds that regulate the defined in (16).
reliability of the embedding. QR blocks are still visible but Sections II and III presents the structure and decoding
the visual impact is reduced with respect to [5]. The method procedures for QR codes since they are central to understand
presented in [15] generates a binary embedding by subdividing the possible modification strategies. Section IV presents the
each QR module in a set of 3 × 3 pixels and setting the concept of QR images and the luminance manipulation algo-
central pixel to the value of the QR code while the remaining rithm and Section V describes the probability of error models
pixels are selected to minimize the probability of error and as a function of the embedding parameters. Finally Section VI
generate a halftone of the image. In this case the probability of and Section VIII present the optimization algorithm and the
error is evaluated empirically using a large database of random embedding results of applying the algorithm to several images
embeddings. and logos.
The main challenge of any embedding method is the fact
that they should be decodable by standard applications. The
embedding of image pixels introduce changes in the lumi- II. C HARACTERISTICS OF QR C ODES
nance of the code, distorting the binarization thresholds and The patterns and structures inside a QR code have well
thus increasing the probability of detection error. The second defined functions which include symbol alignment, sampling
challenge concerns the problem of using the entire area of grid determination, and error correction. The information is
the code in which the image or logo is to be embedded. This encoded in square black and white modules of several pixels
cannot be achieved by simply replacing information modules wide. Finder patterns play a central role in the speed and
with the desired image since the number of modules that can success of decoding and are located in three corners of the
be replaced is at most proportional to the correction capacity symbol. Fig. 2 shows the three main regions in the QR symbol
of the code [7]. A good embedding method, should minimize structure: function pattern region, encoding region and the
the number of corrupted modules and use the greatest possible quiet zone which is a guard region located on the outside
area while keeping visual fidelity to the original image. of the symbol.
This paper aims at the above cited goals by introducing 1) Function Pattern Region: Finder and alignment struc-
QR images, an optimization based approach for embedding tures are essential to locate, rotate and align the QR code.
color images into QR codes. The algorithm proposed is The former ones are designed to have the same ratio of black
based on the selection of a set of modified pixels using and white pixels when intersected by a line at any angle,
a halftoning mask. The concentration of pixels and its allowing to easily detect rotated or inverted codes. Alignment
corresponding luminance are optimized to minimize a visual patterns are used to determine the sampling grids from which
2844 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

codewords are extracted and they are easily identifiable as

concentric square structures evenly distributed along the code
area.
2) Encoding Region: The code area delimited by finder
patterns is denoted as the encoding region, where data, parity
modules and decoding information is stored. This area is
divided into codewords consisting of blocks of 8 QR modules.
Two dimensional shapes of these codewords depend on the
version of the code and are designed to optimize area coverage.
3) Data Capacity and Error Correction: Different types of Fig. 3. (a) Image of a QR module showing the length of the module and the
QR codes defined in the standard [1], are identified by their center region in pixels. (b) Diagram of a module showing possible sampling
version and error correction level. The version of the QR code points.
determines its size and goes from 21×21 modules for version 1
B. Mean Block Binarization Method
up to 177×177 for version 40. QR codes use Reed Solom code
for error correction and there are 4 types of error correction One of the most popular libraries for QR code generation
L, M, Q and H that allow to correct up to 7%, 15%, 20% and and reading is the open source Zxing library [22]. The
30% of codewords in error respectively. thresholds used in the binarization functions of this library,
are calculated through a hybrid method that use the average
luminance in a set of overlapping square windows. The
III. D ECODING A LGORITHM captured image is divided into non-overlapping blocks Bm,n
After acquiring the image and calculating its luminance of 8 × 8 pixels and then the average luminance in overlapping
from the RGB components, the decoding process continues sub windows of 5 × 5 blocks is calculated according to
with three basic stages: binarization, detection, and decoding p=m+2
 q=n+2
of the bit stream. In the binarization stage, the gray scale 1  
Tm,n = Y [k, l]. (2)
image captured by the camera is segmented into black and 25 × 64
p=m−2 q=n−2 (k,l)∈B p,q
white pixels. This binary image is used to determine the QR
The averages calculated for each block Bm,n are assigned
modules centers and the sampling grid from which the code-
to the pixels in the block as ti, j = Tm,n for [i, j ] ∈ Bm,n .
words are extracted. After this process, detected codewords are
Segmentation performance is inferior to the method presented
corrected using the Reed Solomon algorithm and then decoded
in [19] but this method has the advantage of being computa-
according to its format and version.
tionally less expensive. This is the technique assumed in the
following sections to develop the probability of error model
A. Threshold Calculation for Binarization and the QR embedding algorithm.
A salient feature of QR codes which plays a central role
in their decoding speed is the use of binarization as the first C. Sampling Grid and Probability of Sampling Error
step in the decoding process. Binary images are obtained by Once the binary image I B is obtained, codewords are
thresholding the gray scale image as extracted by sampling on a grid estimated using finder and
 alignment patterns. The points in this grid are generated by
1 if Yi, j > ti, j drawing parallel lines between the estimated centers of finder
I B [i, j ] = (1)
0 if Yi, j ≤ ti, j and alignment patterns and the spacing between lines is set
where Y is the image captured by the camera, ti, j is the to the estimated width of a QR module Wa . For larger code
threshold assigned to pixel [i, j ] and I B [i, j ] is the binary sizes, multiple sampling grids are used to compensate for local
output. Determining the values of the thresholds ti, j for an geometric distortions.
optimal binarization is a challenging task due to the variability In order to safely detect the binary value, the luminance
in lighting conditions. The QR code standard does not define around the center of the module should be clearly defined.
specific binarization methods and these are usually selected If we define a region of size da × da pixels centered in the
making a trade off between speed and quality. Thresholds QR module, the probability of sampling outside this region
can be calculated globally using all the pixels in the image can be obtained by assuming a Gaussian distribution of the
or locally considering only reduced windows. Global seg- sampling point around the center and integrating outside the
mentation methods as [17], [18] have been tried and proved region as depicted in Fig. 3. In the following section we model
ineffective since they require high level of contrast in the the sampling distribution as a Gaussian with σ = Wa /4.
acquired image. Local thresholding strategies on the other The probability of sampling error denoted by ps can be
hand are better suited to illumination variations and have precomputed for different sizes of Wa and da to be used in
been successfully applied to QR code binarization [19], [20]. the embedding algorithm.
Adaptive methods such as the one presented in [21] show
better binarization accuracy but at the cost of increased com- IV. A ESTHETIC E NHANCEMENT OF QR C ODES
putational complexity which is not always possible to satisfy The QR code embedding technique introduced here,
in embedded devices. encodes the QR code value on the luminance values of the
GARATEGUY et al.: OPTIMIZED IMAGE EMBEDDING IN QR CODES 2845

image in such a way that the average luminance is increased a halftone mask. The size of the square of center pixels
for light regions in the code and decreased for dark regions. is selected by the user and regulates the robustness of the
In fact any embedding algorithm tailored for a standard embedding but also affects its visual quality. The distribution
decoder must be a variation of this type since the binarization of modified pixels in non central pixels is generated by
thresholds are usually calculated as local averages of pixels thresholding a blue or green noise mask to generate a binary
luminance. The parameters of the modification consist of pattern with a concentration of dots of pc . To simplify the
selecting the location of modified pixel and its luminance or notation we denote by I pc the binary pattern generated by the
to keep the QR module pixels unchanged. In this paper we halftoning mask and by M a mask that is 1 on the set of
consider the last possibility as a particular case in which the central pixels and 0 otherwise.
concentration of modified pixels in a region is close to 1.
The embedding method proposed consists of two components. C. Luminance Modification
The first is the use of halftoning techniques for the selection
After selecting the pixels, its luminance is modified to one
of modified pixels allowing to break and reduce the coarse
of four possible levels α, β, αc , βc . The luminance of the
square structures of the code. The second component is the
embedded image Yi,out j at (i, j ) is selected as a function of
modification of the luminance levels to minimize the image
the QR code value qi, j and the luminance of the original
distortion. This modification parameters are optimized inde-
image Yi, j as described in equation (3). This transformation
pendently in local windows as described in Section VI in
changes the luminance of the pixels that are selected according
order to accelerate the embedding and leverage the correlation
to the halftone distribution and keep the remaining pixels
between the luminance of the image and the code.
in the image unchanged. The pixels at the center of the
QR module are assigned different luminance levels, since they
A. Halftoning Techniques play a central role in the detection of binary values when the
The method proposed to select modified pixels is based on sampling accuracy is high.
halftoning techniques in order to minimize the appearance of ⎧
blocks while preserving high frequency details. If modified ⎪β
⎪
⎪ if Mi, j = 0, qi, j = 1, I pc ,(i, j ) = 1
⎪α
⎪
pixels are randomly but uniformly distributed in space, the
⎪
⎨ if Mi, j = 0, qi, j = 0, I pc ,(i, j ) = 1
visual impact of the embedding is minimized since these Yi,out
j = β c if Mi, j = 1, qi, j = 1 (3)
patterns concentrate most of their energy at higher frequencies
⎪
αc if Mi, j = 1, qi, j = 0
⎪
⎪
⎪
where the human visual system is less sensitive. This effect
⎪
⎪
⎩Y
i, j otherwise.
is commonly used in digital halftoning [23] where different
algorithms to generate even distributions of points with par-
ticular spectral properties have been proposed. Examples of D. Color Optimization
such algorithms are, error diffusion [24], [25], blue noise Since the goal is to embed the codes into color images,
masks [26]–[28], green noise masks [26], or direct binary we need to establish a rule to select the corresponding color
search [29]. Error diffusion has a long history in the printing vector for each modified luminance since there is an associated
industry where it has been used for the last 30 years; however, subspace of colors in the RG B space for each luminance
being a recursive algorithm, the processing time increase value. To determine the best possible color that fulfills the
considerably for very large images. Blue and green noise luminance constraint, it is necessary to measure the color
masking techniques in contrast, generate high frequency binary differences into a perceptually uniform color space [33]. The
patterns by thresholding a carefully designed multilevel array. H S L color space is selected because it involves simpler
These patterns can be computed in parallel which greatly computations than other color spaces while it is still capable
increase the speed of generation. Many algorithms to design of representing perceptual difference using euclidean metrics.
halftoning masks were presented in the literature such as, To obtain the RG B values of a modified pixel given the
Void and Cluster [30], direct binary search [29], green noise luminance target lt ∈ [0, 1], the original color is transformed
masks [26], [31], blue noise multitone dithering [32] and into the H S L color space and then the L component is
techniques based on centroidal voronoi tessellations [28]. Each optimized keeping S and H fixed until reaching the desired
of these techniques have different computational complexities, luminance Y = lt . The relationship between the luminance
but since the mask design process is performed offline the defined as Y = 0.2989R + 0.5870G + 0.114B and the
speed of pattern generation does not change. min(R, G, B) + max(R, G, B)
L=
2
B. Pixel Selection component in the H S L color space is a piecewise, linear and
Ideally, only the pixels at the center of the QR module are monotone function Y = f (L). If the weight luminance vector
relevant for a correct decoding, however, due to errors in the is defined as w = [0.298, 0.587, 0.1140]T , then f (L) is given
determination of the sampling grid, adjacent pixels play an by
important role in the decoding process. To account for this
Y = f (L) = w T T −1 (H, S, L), (4)
we make a distinction between the pixels in the QR module.
A square of size da × da is always selected for modification where (R, G, B) = T −1 (H, S, L)
is the backward transfor-
and the remaining pixels in the module are selected using mation from H S L to RG B color spaces. The optimal value
2846 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

Algorithm 1 Color Modification for Luminance Target where p0 = P(qi, j = 0) and p1 = P(qi, j = 1) are the
probabilities of having a black or white pixel in the code
respectively. The detector makes a binary decision on the value
of the code qi, j based on the threshold ti, j and the embedding
luminance Yi,out
j . Taking this into account, the probabilities of
miss detection in (6) can be expressed as
P(decide qi, j = 1|qi, j = 0) = P(Yi,out
j > ti. j |qi, j = 0)
P(decide qi, j = 0|qi, j = 1) = P(Yi,out
j < ti. j |qi, j = 1). (7)

The gray scale image Y is modeled as a bi-dimensional

of L is obtained from the luminance target Y out = lt as the random field, and the thresholds as correlated random vari-
solution of ables. The exact calculation of the probabilities in (7) involves
the knowledge of the joint distribution of the image and
L ∗ = argmin| f (L) − lt |. (5) threshold values which in practice can be quite challenging
L to calculate. Instead of using this general relation a simpli-
Once the optimal value L ∗ is determined, the new RG B fied model is proposed here, assuming independence of the
components of the pixels are obtained by using the forward components of the embedding luminance Yi,out j . The value
transformation between the H S L and RG B color spaces of Yi,out
j at location (i, j ) can be decomposed as the com-
(R ∗ , G ∗ , B ∗ ) = T (H, S, L ∗ ). Summarizing the different bination of several random variables that are related to the
steps, the transformations to obtain the RG B value of a luminance parameters of the transformation. In this model we
modified pixel, given the target luminance value lt is described don’t differentiate between central and non-central pixels and
in algorithm 1. consider that the contribution of central pixels to the threshold
is minimal.
V. P ROBABILITY OF E RROR M ODELS The expression for the output image embedding under these
assumptions is given by
Pixels of the embedded code are a combination of pixels
from the image and the QR code as described in (3). This Yi,out
j = [βqi, j +α(1 − qi, j )]I pc ,(i, j ) +Yi, j (1 − I pc ,(i, j ) ). (8)
mixture distorts the binarization thresholds with respect to the
black and white QR code, and in general increases the proba- The first component of this expression is the binary random
bility of binarization error. The problem can be minimized variable I pc ,(i, j ) that selects the pixels being modified and
by optimizing the luminance parameters, but this presents models the stochastic nature of the blue binary pattern. Despite
a trade off between decoding robustness and image quality. the fact that the location of modified pixels are correlated
Efficient optimization methods require a tractable model for by the binary pattern constraints, it is reasonable to model
the probability of error defined in local neighbourhoods to them as independent Bernoulli random variables given that a
leverage the parallel nature of the problem. The probability sufficiently large window is considered. Furthermore, indepen-
model presented here is based on reasonable assumptions dence is also assumed between QR code values, qi, j and lumi-
and yields an expression for the probability of error that nance values Yi, j in the original image. Finally the variables
is smooth and continuous and can be easily integrated into Yi, j are modeled as independent random variables following a
different optimization algorithms. This model considers two local distribution based on the size and location of the window
regimes, which correspond to the precise sampling of QR in the image. Combining these assumptions and the expression
modules centers and the case when all the pixels in the module for the binarization thresholds, it is possible to derive (see
are indistinguishable. Two independent probability models are Appendix A) a model for the probability distribution of the
constructed and then combined to yield a general model. thresholds given by
n 
 n k n−k
P(ti, j = t) = f (t − tk ) p p . (9)
A. Probability of Binarization Error Model k 1 0
k=0
The probability of binarization error is defined as the
where n = ⌊ pc (64 × 25)⌋ is the total number of modified
probability of sampling the incorrect binary value at any pixel
pixels in the the window of blocks defined in (2),
in the QR module. This probability is influenced by different
factors such as the local distribution of luminance values in kβ (n − k)α
the image, the distribution of pixels in the QR code and the tk = +
25 × 64 25 × 64
parameters of the luminance transformation. To formulate a
model of the probability of error, consider the detected binary are the possible outcomes from the sum of n modified lumi-
value at pixel (i, j ). The probability of error at this location nance pixels βqi, j + α(1 − qi, j ) and f is the convolution of
is given by f Y (x) corresponding to the mean value of unmodified pixels
and f n (x) corresponding to the distribution of the Gaussian
PBerr = P(decide qi, j = 1|qi, j = 0) p0 noise at the detector. This model is central for the tractability
+P(decide qi, j = 0|qi, j = 1) p1. (6) of the decision probabilities in (7) and yields a smooth and
GARATEGUY et al.: OPTIMIZED IMAGE EMBEDDING IN QR CODES 2847

Considering the luminance transformation in (3) and using

the four luminance levels. The binarization thresholds at the
detector are given by t = µ + b + η where
αn α + βn β + αc n αc + βc n βc
µ=
N
and b is the mean value of unmodified pixels in the window.
The number of pixels in each modified luminance level is
denoted by, n α , n β , n αc , n βc and N is the total number of pix-
els. By the considerations made in Section IV-C the variable b
can be regarded as a Gaussian random variable with mean
µb = E[Y1, j ](1 − pc ) and variance
V ar [Yi, j ]
Fig. 4. Probability of binarization error (PBE) for a randomly selected σb2 = (1 − pc )
window as a function of α for fixed β and pc . The variance of the noise N
used was ση = 0.1. independent of the noise η. Therefore the probability distribu-
tion of t is also Gaussian with mean µt = µ+µb and variance
continuous expression for the probability of binarization error σt2 = ση2 + σb2 and its cdf can be expressed as a function of
given by the Gaussian normal cdf,
n

x − µt
P(t < x) =  .

PBerr = pc wk ( p0 F(α − tk ) − p1 F(β − tk )) σt
k=0
n
 Finding the probability of detection error then reduces to
+(1 − pc ) wk ( p0 − p1 )F(Yi, j − tk ) + p1 (10) combining this result with the expressions in (7) and (11)
k=0 yielding
 
where the coefficients αc − µt β c − µt
PDerr =  p0 −  p1 + p1 . (13)
 σt σt
n k n−k
wk = p p
k 1 0 C. Global Probability of Error
are introduced for compactness. Fig. 4 depicts the agreement The previous probability models define two clear regimes
between the probability model and the empirical probability of in the detection of binary values, however, there are cases in
binarization error calculated by adding Gaussian noise with the in which only a faction of the sampled pixels lie inside the
appropriate variance to the embedding and then calculating the center regions with modified luminance αc or βc . However,
probability of error by counting the number of discrepancies since the probability of sampling error can be precomputed
between the QR code and the binarized embedding. for a given module (Wa ) and center size da , it is possible
to find the global probability of error by conditioning on the
event of successful sampling of the center region. The global
B. Probability of Detection Error probability of error is then given by
If we assume that the pixels at the center of each module can
Perr = PBerr ps + PDerr (1 − ps ). (14)
be sampled accurately, then the probability of error is given
by
VI. Q UALITY M ETRICS
PDerr = P(decide qc = 1|qc = 0) p0 Based on the probability models developed, is possible to
+P(decide qc = 0|qc = 1) p1 (11) accurately determine the probability of error for a given set
of luminance parameters and guarantee the decodability of
where qc is the value of the QR code module at its center the embedding. However, it is necessary to define appropriate
and p0 = P(qc = 1), p1 = P(qc = 1) are the probabilities of metrics to quantify also its visual quality. In spite of the
having the corresponding QR module inside the local window. complexities of the human visual system (HVS) a largely
There is a slight difference with respect to the expression in used quality metric to measure image similarity is the mean
(6) since here we consider the probability of detecting the squared error (MSE) since it has many desirable mathematical
incorrect value but only at the center position of the module. properties such as convexity and differentiability, however it
This is a much relaxed requirement than the one in (6) since does not always correlates with visual quality [34]. There have
only the mean value of non-central pixels is important. This been efforts to overcome the limitations of the MSE such
naturally leads to a different probability model where the miss as the SSIM index [35] which explicitly considers different
detection probabilities in (11) reduce to luminance, contrast and structure measures. For the case of
halftones the most commonly used metric is based on the
P(decide qc = 1|qc = 0) = P(αc > t) MSE between two filtered versions of the original image and
P(decide qc = 0|qc = 1) = P(βc < t). (12) the halftone where the filter usually models the low pass
2848 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

Fig. 5. Different stages of the QR embedding procedure. The inputs are the original image, the QR code, the halftone mask, the masks used for image
prioritization and global value of Pmax and da . These images are divided in local windows and then optimized independently and in parallel. Finally the
results are interpolated to combine them in the final result.

characteristic of the HVS. One of the most used models the number of pixels in the window. Only the L component
for the HVS was proposed by Näsänen [36] and consist is considered here because S and H are unchanged in the
of an exponential model H ( f r ) = e−k fr where fr is the color optimization. The filter h used in all the experiments
radial frequency in cycles/degree. The corresponding discrete is the one defined in (15) with parameters D = 9 inches
impulse response considering the viewing distance D in inches and R = 150 dpi. Considering all the requirements, the final
and the resolution R in dots/inch is given by expression for the distortion metric used in the optimizations
k 1 is then given by
h[m, n] = 3/2
(15)
2π k
+ ( π180m 2 + ( π180n 2 1
2π RD ) RD ) J (α, β, αc , βc , pc ) = ||h ∗ (L out − L)|| F
N
where the constant k is determined by fitting empirical data 1
to the model [36]. + (1 − M SS I M(Y out , Y )). (18)
2
To evaluate the similarity between the image and the embed-
ding, we propose a variation of the metric presented in [37]
originally introduced to design structure aware halftones. The VII. PARAMETER O PTIMIZATION
rationale for this selection is that blocks of the QR code In order to take advantage of local correlations between
should be less visible when viewed from a distance where the luminance of the image and the values of the QR code,
the low pass characteristics dominate but at the same time the the optimization of the transformation parameters is performed
embedding should preserver important details when viewed independently using local overlapping windows. In our imple-
at closer distances. This metric considers both the halftone mentation we choose to set the size of the windows to
visibility and image structure and allows to select the trade 40×40 pixels centered around each image block of 8×8 pixels.
off between both by adjusting two weighting parameters. The This respects the proportions of the window defined in (2)
general form of this metric is given by and based on our experimental tests, yields a good trade off
J (Y out , Y ) = wl G(Y out , Y ) + wh (1 − M SS I M(Y out , Y )). between robustness and quality for typical scanning distances.
The optimization of luminance parameters in local win-
(16)
dows has other advantages besides leveraging the correlation
where the term G(Y out , Y ) gives more importance to the low between image and code luminance. For example it is pos-
pass error between the embedding and the image while the sible to specify regions of the image which require higher
term 1 − M SS I M(Y out , Y ) measures the structure dissimi- visual quality or higher decoding robustness and set different
larity considering high frequency components and the overall probability bounds for each case.
structure of the image. A block diagram of the optimization process is depicted in
The weighting factors selected for all the simulations in this Fig. 5. After the initial subdivision of the image, the code
paper are wl = 1 and wh = 1/2 and its determination was and the different masks, the optimization of each window is
based on several experiments with different images from which performed in parallel. The combination of the global lumi-
the best combination of parameters was selected. To make a nance parameters is performed by low pass filtering the array
faithful prediction of tone similarity the term G includes a of solutions and interpolating to match the size of the original
model of the HVS and the difference between the images in image. This global map is then applied to the original image
calculated in the HSL color space since color similarity is an to obtain the QR embedding.
important component of the perceived similarity. The general The local optimizations are formulated based on the quality
expression for G is then given by metrics defined in (18) and the probability of error model in
1 (14) as
G(Y out , Y ) = ||h ∗ (L out − L)|| F (17)
N argmin Ji (α, β, αc , βc , pc )
where L out and L are the lightness components of the embed- α,β,αc ,βc , pc
ding and original image in the HSL color space and N is subject to Perr < Pmax,i (19)
GARATEGUY et al.: OPTIMIZED IMAGE EMBEDDING IN QR CODES 2849

Fig. 6. (Top row) Image embeddings for (a) da = 2 pixels (J = 0.4), (b) da = 3 pixels (J = 0.4053), (c) da = 4 pixels (J = 0.4194). The QR modules in
this example had a size of Wa = 8 pixels, and the global probability of error used was Pmax = 0.2. A noise level of ση = 0.1 was assumed. (Bottom row)
Images acquired using a Google Nexus 7 tablet at 12in from the target. The images were printed in a resolution of 150d pi.

The input parameters to the optimizations are the global for decreasing values of the regularization parameter µ. The
parameter Pmax which determines the maximum admissible advantage of this approach is that the starting points can be
probability of error and the size of the center region in pixels, infeasible at the beginning and as the regularization parameter
da . The parameter Pmax,i in the local optimization (19) is is decreased the sequence of solutions converges towards a
determined as a function of the local features of the image. feasible local minimum of the original problem in Eqn. (19).
In particular here, we define a mask M P which has value The solver used is based on a sequential quadratic programing
−2 in regions where the image presents an edge, 1 in the with trust region method presented in [39] and implemented
function pattern regions and 0 elsewhere. The value of Pmax,i by the f mi ncon function of MATLAB [40].
is calculated as
Pmax,i = (1 − mean(M P,i ))Pmax (20) VIII. I MAGE AND L OGO E MBEDDINGS
This Section presents the result of embedding different
where M P,i is the corresponding patch of M P in the local
images into a QR code and the experimental decoding rates
window.
obtained with mobile devices in order to evaluate the effec-
The edges are detected using a canny edge detector and
tiveness of the embedding method on a real setting.
lowpass filtering to generate the edge regions. This is a
common practice in other embedding algorithms such as [12]
or [15]. The values of Pmax,i can be distributed based on a A. Decoding Robustness
probability of error budget consistent with the user preferences To determine the robustness of the embedding against non
and type of error correction provided by the QR code (i.e. modeled disturbances such as blur and uneven illumination we
L, M, Q or H). conducted an scanning experiment. The experiment consisted
of scanning the embeddings in Fig. 6 from a printed version
A. Nonlinear Optimization Methods in which each QR pixels was render at 150 dpi and also
The optimization problem in Eqn. (19) is solved by means from a laptop screen with a smart phone and a tablet. The
of a logarithmic barrier method [38]. The solutions are found embeddings generated had different sizes of the central region
by solving the sequence of problems corresponding to da = 2, da = 3 and da = 4 pixels with a
argmin J (α, β, αc , βc , pc ) + µ log(s) corresponding module size of Wa = 8 pixels and the halftone
α,β,αc ,βc , pc ,s mask used was a green noise mask [41] to minimize the dot
subject to Perr (α, pc ) − Pmax + s = 0 (21) gain effect introduced by the printer. The devices used to scan
2850 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

Fig. 7. Comparison of the embedding of different images and logos into a QR code. (a) Embedding of an image for a center region size of da = 2 pixels
and a global probability of error of Pmax = 0.2 using a blue noise mask (b) Embedding of a logo for da = 1 and Pmax = 0.15 using a clustered dot
mask aligned with the centers of the QR code modules. (c) Embedding of a logo for da = 3 and Pmax = 0.15 using a blue noise mask. The embedding in
(d) and (e) where generated using the method in [5] while the embedding in (f) corresponds to [14].

the codes were a smart phone Samsung Galaxy S I897 and a TABLE I
Google Nexus 7 tablet and the app used was Barcode Scanner1 T HE MSE AND MSSIM A RE C ALCULATED OVER THE L UMINANCE
which is based on the Zxing Library. The recognition times OF THE E MBEDDING AN THE O RIGINAL I MAGE . T HE MSE
where in general in the order of 5 to 10 seconds for the P RESENTED H ERE I S N ORMALIZED BY THE N UMBER OF
embeddings in Fig. 6(a) and Fig. 6(b) and in the order of 1 to P IXELS IN THE I MAGE , M S E = √1 ||(Y out − Y )|| F
N
2 seconds for the embedding of Fig. 6(c) comparable to the
method in [5]. The effective scanning distance depends on the
resolution of the device, and in general is necessary to have
multiple image pixels per QR module pixel for a successful
decoding. This is in line with the assumption that QR pixels
can be individually resolved, used to develop the probability
models. Examples of the images captured by the tablet are
depicted in Figs. 6(d), 6(e), and 6(f). These pictures were
taken at an average distance of 9 inches from the target with
approximately 10 pixels per QR module. The embeddings with
smaller center regions where more susceptible to error and the
scanning distance limited to 1 or 2 inches around the distance The values of Pmax selected should be smaller than the
in which a QR module pixel is represented by a pixel in the corresponding decoding robustness of the code in order to
captured image. For images with larger central regions such have a successful decoding. In the case of Fig. 6, the code
as 6(c). The range of successful scanning is extended since used had a correction level H (30% correction capacity) and
the assumption about correct sampling of the centers applies Pmax = 0.2. It is important to note that this embedding
to a broader range of conditions. One possible explanation for method always reduces the robustness of the code, however,
the reduced robustness of the embedding is 6(a) and Fig. 6(b) this reduction can be precisely controlled by changing the
is that the noise level used to design the embedding underes- value of the Pmax parameter and the center size da . Smaller
timates the contributions of poor illumination or camera blur. center sizes imply a unifrom tone change similar to the results
1 https://play.google.com/store/apps/details?id=com.google.zxing.client. in [14] while larger center regions yield similar results to the
android method in [5]. The main advantage here is that the user can
GARATEGUY et al.: OPTIMIZED IMAGE EMBEDDING IN QR CODES 2851

select which is best in each case given more versatility for the luminance of modified pixels to achieve particular error limits.
integration in different designs. In a similar way as in [15] or [12] different regions of the
image are prioritized by using local probability limits Pmax,i .
B. Visual Quality of the Embeddings These bounds can be used to select the codewords in error
allowing to distribute the errors spatially to regions where the
Fig. 7 depicts the results of embedding a QR code into
image presents non relevant features.
different images and logos. Each embedding is compared with
The main limitation of this method is the assumption that
their corresponding counterparts using the methods in [5]
single pixels in the QR code can be resolved in the acquired
or [14]. Different center sizes and halftone masks are used
image and that they are only corrupted by Gaussian noise.
for each embedding. For an objective evaluation of the quality
This is a simple yet powerful model but it adds a constraint
different metrics are compared in Table I, including the metric
on the minimum resolution of the image. Different trials using
J (18) used in the optimization. An interesting feature of
mobile devices and printed codes however have shown that for
the proposed embedding method is the possibility to generate
common scanning distances the models are indeed applicable
embeddings with different textures by changing the halftone
and the decoding rates are reasonable in comparison with
mask as in Fig. 7(b) where a clustered dot mask aligned with
similar methods while having a noticeable improvement in
the center of the QR modules was used generating pseudo
visual quality with respect to the metric defined. An important
random cluster of dots. This feature also allows to solve
factor for the success of the decoding is the correct estimation
the problem of dot gain generated when the embeddings are
of the variance of the noise to accommodate different illumi-
printed simply by using appropriate halftoning masks.
nations or camera blur. The values of ση used in this paper are
between 0.1 and 0.2 an were selected by experimentation with
IX. D ISCUSSION AND C ONCLUSION
different images and standard fluorescent illumination in an
A method to embedded QR codes into color images was office environment. It is important to note that the embedding
presented, allowing to automatically generate embeddings with method might fail to decode if the reading application uses
limited probability of detection error for a given noise power. a different binarization method than the one assumed here,
This method shares different elements with previous methods however since the Zxing library is widely adopted this is
such as [5], [14], and [15] since it allows to optimize the not a severe limitation. Finally as a consequence of the
luminance values of individual pixels in the image at locations elaborate models for the visual quality and the probability of
selected via halftoning techniques and by setting the centers error the implementation of the method is significantly more
of the QR modules to specific values. The method presented computationally intensive than other methods. The present
in [14] change the luminance of modified pixels into two implementation in Matlab takes 30 minutes to optimize a color
luminance levels analogous to the α and β used here. However image of size 350×350 pixels, but more efficient implementa-
these levels are global and the selection of its optimal values tions that take advantage of parallelism and especial processors
is based on the assumption that the image is uniformly like GPUs might reduce the optimization time to practical
distributed and that the binarization threshold is fixed. The levels.
method developed here is much more general and includes
A PPENDIX A
these as particular cases. We make an accurate approximation
of the image statistics using its local mean and variance and P ROBABILITY OF B INARIZATION E RROR
develop a model for the probability of error using a common If we assume that all pixels in the window are indis-
binarization algorithm widely used in real applications. The tinguishable then we can treat them as non-central pixels
proposed method also introduces a novel technique to distrib- and assume that they can only take two independent values
ute modified pixels based on halftoning methods which help to α or β. The binarization threshold at location (i, j ) is in
reduce the visual impact of the modification. The embedding general given by (2) and it can be divided in three components
of halftones into QR codes was proposed in [15] where the ti, j = ai, j + bi, j + ηi, j with
location of binary pixels in the QR modules is optimized to
1  
maximize visual quality and decoding robustness. This method ai, j = (k,l)∈B p,q βqk,l + α(1 − qk,l )
however is only suitable to generate binary halftones and no 25 × 64 p,q
(k,l)∈C
modification for the embedding of color or multilevel halftones 1  
bi, j = (k,l)∈B p,q Y [k, l]. (22)
was proposed. 25 × 64 p,q
(k,l) ∈C
In addition to the use of halftones to diminish the visual
distortion of the embedding, the method presented here defines Given the set of modified pixels these components are indepen-
a quality metric which considers color, tone and structural dent and is possible to evaluate the distribution of the threshold
similarity used to select the optimal luminance of modified as the convolution of the distributions of its components. The
pixels. To fully leverage the characteristics of QR decoders, component ai, j can be modeled as a binomial distribution
central pixels in the QR modules play an important role in assuming independence of the code pixels given by
our algorithms similarly to [5] and [15]. However in contrast n  
 kβ (n − k)α n k n−k
to these methods which fix the ratio between central and non P(ai, j = x) = δ x− + p p
central pixels, the method proposed here allows to choose the 25 × 64 25×64 k 1 0
k=0
number of central pixels and then optimize the location and (23)
2852 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 7, JULY 2014

where k is the number of pixels with modified luminance β R EFERENCES

in the local window and n is the total number of modified
[1] Information Technology—Automatic Identification and Data Capture
pixels. Since the variables Yi, j are assumed to be independent Techniques—Bar Code Symbology QR Code, Int. Org. Standardization,
and identically distributed, the distribution of bi, j can be Geneva, Switzerland, ISO/IEC 18004, 2006.
approximated by a Gaussian distribution denoted by P(bi, j = [2] M. Hara, M. Watabe, T. Nojiri, T. Nagaya, and Y. Uchiyama, “Optically
readable two-dimensional code and method and apparatus using the
y) = fY (y) with mean same,” U.S. Patent 5 726 435, Mar. 10, 1998.
[3] T. J. Soon, and Q. R. Code. Synthesis Journal, 2008, section 3
E[Yk,l ](25 × 64 − n) [Online]. Available: http://www.itsc.org.sg/pdf/synthesis08/Three_QR_
µy = = E[Yk,l ](1 − pc )
25 × 64 Code.pdf
[4] R. Cox. (2012, Apr.). Qart Codes [Online]. Available:
1− pc
and variance σ y2 = V ar [Yk,l ] 25×64 . Combining these expres- http://research.swtch.com/qart
sions and using the fact that ai, j , bi, j and ηi, j are independent [5] Visualead company, Herzelia, Israel. (2013, Jan.). Free Visual QR Code
Generator [Online]. Available: http://www.visualead.com/
give C, the probability distribution of the threshold ti, j is [6] S. Ono, K. Morinaga, and S. Nakayama, “Two-dimensional barcode
given by decoration based on real-coded genetic algorithm,” in Proc. IEEE CEC,
 Jun. 2008, pp. 1068–1073.
n n k n−k [7] D. Samretwit and T. Wakahara, “Measurement of reading characteristics
P(ti, j = t) = ( f Y ∗ fn )(t − tk ) p p of multiplexed image in QR code,” in Proc. 3rd Int. Conf. Intell. Netw.
k=0 k 1 0 and Collaborative Syst., 2011, pp. 552–557.

n n k n−k [8] T. Wakahara and N. Yamamoto, “Image processing of 2-dimensional
= f (t − tk ) p p (24) barcode,” in Proc. 14th Int. Conf. Network-Based Inform. Syst., 2011,
k=0 k 1 0 pp. 484–490.
[9] Program, information storage medium, two-dimensional code gen-
where eration system, image generation system and printed material, by
kβ (n − k)α S. Oouchi. (2007, Mar. 1). Patent 7 950 589 [Online]. Available:
tk = + http://www.google.com/patents/US7950589
25 × 64 25 × 64 [10] M. Hagiwara. (2013). Qrjam [Online]. Available: http://qrjam.jp/
[11] K. Fujita, M. Kuribayashi, and M. Morii, “Expansion of image dis-
are the possible outcomes of the sum of n modified pixels. playable area in design QR code and its applications,” in Proc. Forum
The distribution f η (n) corresponds to the Gaussian noise and Inform. Technol. Papers, 2011, vol. 10, no. 4, pp. 517–520.
f = f Y ∗ f n is the convolution of the average image values [12] Y. Lin, Y. Chang, and J. Wu, “Appearance-based QR code beautifier,”
IEEE Trans. Multimedia, vol. 15, no. 8, pp. 2198–2207, Dec. 2013.
and noise distributions. Since Y and n are independent and [13] D. W. Inc., Elkhart, IN, USA. (2013). Logoq [Online]. Available:
Gaussian, f is also Gaussian with mean µ = µY and variance http://www.qrcode.com/en/codes/logoq.html
 [14] Z. Baharav and R. Kakarala, “Visually significant QR codes: Image
blending and statistical analysis,” in Proc. IEEE ICME, Jul. 2013,
σ = σY2 + σn2 . pp. 1–6.
[15] H.-K. Chu, C.-S. Chang, R.-R. Lee, and N. J. Mitra, “Halftone QR
Finally the conditional probabilities on (7) are obtained by codes,” ACM Trans. Graph., vol. 32, no. 6, p. 217, 2013.
[16] O. Bulan, H. Blasinski, G. Sharma, “Color QR codes: Increased capacity
conditioning on the set of modified pixels I pc as follows via per-channel data encoding and interference cancellation,” in Proc.
19th CIC, San Jose, CA, USA, 2011.
P(Yi,out
j > ti. j |qi, j = 0) = P(α > ti. j ) pc [17] N. Otsu, “A threshold selection method from gray-level histograms,”
+P(Yi, j > ti. j )(1 − pc ) (25) Automatica, vol. 11, nos. 285–296, pp. 23–27, 1975.
[18] A. Brink and N. Pendock, “Minimum cross-entropy threshold selection,”
Pattern Recognit., vol. 29, no. 1, pp. 179–188, 1996.
and [19] J. Sauvola and M. Pietikäinen, “Adaptive document image binarization,”
Pattern Recognit., vol. 33, no. 2, pp. 225–236, 2000.
P(Yi,out
j < ti. j |qi, j = 1) = P(β < ti. j ) pc [20] J. Zhou, Y. Liu, and P. Li, “Research on binarization of QR code image,”
in Proc. ICMT, 2010, pp. 1–4.
+P(Yi, j < ti. j )(1 − pc ). (26)
[21] H. Yang, A. Kot, and X. Jiang, “Binarization of low-quality barcode
images captured by mobile phones using local window of adaptive
Using (25), (26) and substituting in (6) we obtain the proba- location and size,” IEEE Trans. Image Process., vol. 21, no. 1,
bility of binarization error pp. 418–425, Jan. 2012.
[22] S. Owen. (2012). Zxing, Multi-Format 1d/2d Barcode Image Processing
n
 Library With Clients for Android, Java and C++ [Online]. Available:
Perr = pc wk ( p0 F(α − tk ) − p1 F(β − tk )) https://code.google.com/p/zxing/
[23] D. L. Lau and G. R. Arce, Modern Digital Halftoning. New York, NY,
k=0
USA: Marcel Dekker, 2001.
n
 [24] R. Floyd and L. Steinberg, “An adaptive technique for spatial grayscale,”
+(1 − pc ) wk ( p0 − p1 )F(Yi, j − tk ) + p1 in Proc. Soc. Inform. Display, vol. 17. 1976, pp. 78–84.
k=0 [25] R. L. Stevenson and G. R. Arce, “Binary display of hexagonally
sampled continuous-tone images,” J. Opt. Soc. Amer. A, vol. 2, no. 7,
where F is the cumulative distribution function of f and pp. 1009–1013, 1985.
 [26] D. L. Lau, G. R. Arce, and N. C. Gallagher, “Digital halftoning by
n k n−k means of green-noise masks,” J. Opt. Soc. Amer. A, vol. 16, no. 7,
wk = p p . pp. 1575–1586, 1999.
k 1 0
[27] D. L. Lau, G. R. Arce, and N. C. Gallagher, “Green-noise dig-
ital halftoning,” Proc. IEEE, vol. 86, no. 12, pp. 2424–2444,
The probability of error depends on the distribution of the Dec. 1998.
image, the QR code, the transformation parameters α, β, pc [28] G. J. Garateguy, G. R. Arce, and D. L. Lau, “Voronoi tessellated halftone
and the noise power level ση . masks,” in Proc. 17th IEEE ICIP, Sep. 2010, pp. 529–532.
GARATEGUY et al.: OPTIMIZED IMAGE EMBEDDING IN QR CODES 2853

[29] M. Analoui and J. P. Allebach, “Model-based halftoning using direct Gonzalo R. Arce (F’00) received the M.S. and
binary search,” in Proc. SPIE/IS&T Symp. Electron. Imag., Sci. Technol., Ph.D. degrees in electrical engineering from Pur-
Int. Soc. Opt. Photon., 1992. due University, West Lafayette, IN, USA. He is
[30] R. Ulichney, “The void-and-cluster method for generating dither arrays,” the Charles Black Evans Professor of Electri-
Proc. SPIE, Human Vis., Vis. Process., Digit. Display IV, vol. 1913, cal and Computer Engineering with the Univer-
pp. 332–343, 1993. sity of Delaware, Newark, DE, USA. He was the
[31] D. L. Lau, R. Ulichney, and G. R. Arce, “Blue and green noise halfton- Fulbright-Nokia Distinguished Chair of Information
ing models,” IEEE Signal Process. Mag., vol. 20, no. 4, pp. 28–38, and Communications Technologies at Aalto Uni-
Jul. 2003. versity, Helsinki, Finland, in 2010. His research
[32] J. B. Rodriguez, G. R. Arce, and D. L. Lau, “Blue-noise multitone interests include statistical signal processing and
dithering,” IEEE Trans. Image Process., vol. 17, no. 8, pp. 1368–1382, computational imaging. He has co-authored the texts
Aug. 2008. Nonlinear Signal Processing (Wiley, 2004), Modern Digital Halftoning (CRC
Press, 2008), and Computational Lithography (Wiley, 2010). He has served
[33] H. J. Trussell, E. Saber, and M. Vrhel, “Color image processing [basics
as an Associate Editor of several journals of the IEEE, OSA, and SPIE.
and special issue overview],” IEEE Signal Process. Mag., vol. 22, no. 1,
pp. 14–22, Jan. 2005.
[34] Z. Wang and A. C. Bovik, “Mean squared error: Love it or leave it?
A new look at signal fidelity measures,” IEEE Signal Process. Mag.,
vol. 26, no. 1, pp. 98–117, Jan. 2009.
[35] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image
quality assessment: From error visibility to structural similarity,” IEEE
Daniel L. Lau (SM’09) received the B.Sc. (Hons.)
Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004.
degree from Purdue University, West Lafayette, IN,
[36] R. Nasanen, “Visibility of halftone dot textures,” IEEE Trans. Syst., Man, USA, in 1995, and the Ph.D. degree in electri-
Cybern.,Syst., vol. SMC-14, no. 6, pp. 920–924, Nov./Dec. 1984. cal engineering from the University of Delaware,
[37] W.-M. Pang, Y. Qu, T.-T. Wong, D. Cohen-Or, and P.-A. Heng, Newark, DE, USA, in 1999. He is currently an
“Structure-aware halftoning,” ACM Trans. Graph., vol. 27, no. 3, p. 89, Associate Professor with the Department of Electri-
2008. cal and Computer Engineering, University of Ken-
[38] D. P. Bertsekas, Nonlinear Programming. Belmont, MA, USA: Athena tucky, Lexington, KY, USA, where his early research
Scientific, 1999. interest focused on digital printing and halftoning.
[39] R. H. Byrd, J. C. Gilbert, and J. Nocedal, “A trust region method He has authored Modern Digital Halftoning. His
based on interior point techniques for nonlinear programming,” Math. current research interests include 3-D measurements
Program., vol. 89, no. 1, pp. 149–185, 2000. by means of structured light, with his contributions having been featured in
[40] Mathworks, Natick, MA, USA. Fmincon Reference Page numerous machine vision trade magazines, and the creation of a startup cor-
Documentation [Online]. Available: http://www.mathworks.com/help/ poration, i.e., Seikowave Inc., Lexington. Apart from the research, he is also
optim/ug/fmincon.html an Avid Digital Photographer primarily interested in lenticular photography.
[41] D. L. Lau, G. R. Arce, and N. C. Gallagher, “Digital color halftoning He has authored papers in the P ROCEEDINGS OF THE IEEE.
with generalized error diffusion and multichannel green-noise masks,”
IEEE Trans. Image Process., vol. 9, no. 5, pp. 923

Gonzalo J. Garateguy (S’06) received the B.E.E. Ofelia P. Villarreal received the B.E.E. degree at
degree from the Catholic University of Uruguay, the University of Valle in Cali, Colombia, in 2006.
Montevideo, Uruguay, in 2007, and the Ph.D. degree She is currently a Master student at the Depart-
from the University of Delaware, Newark, DE, USA, ment of Electrical and Computer Engineering at the
in 2014. He is currently a Signal Processing Quality University of Delaware, Newark, DE, USA. Her
Engineer with The MathWorks, Inc., Natick, MA, current research interests include signal and image
USA. His research interests include signal and image processing.
processing, halftoning, and convex optimization.