Three-dimensional profiling with binary fringes

using phase-shifting interferometry algorithms

Gastón A. Ayubi,1 J. Matías Di Martino,1 Julia R. Alonso,1 Ariel Fernández,1

César D. Perciante,2 and José A. Ferrari1,*
1
Instituto de Física, Facultad de Ingeniería, UdelaR, J. Herrera y Reissig 565, Montevideo, Uruguay
2
Facultad de Ingeniería y Tecnologías, UCUDAL, Av. 8 de Octubre 2738, Montevideo, Uruguay
*Corresponding author: [email protected]

Received 23 September 2010; revised 12 November 2010; accepted 17 November 2010;

posted 17 November 2010 (Doc. ID 135569); published 6 January 2011

Three-dimensional shape measurements by sinusoidal fringe projection using phase-shifting interfero-

metry algorithms are distorted by the nonlinear response in intensity of commercial video projectors and
digital cameras. To solve the problem, we present a method that consists in projecting and acquiring a
temporal sequence of strictly binary patterns, whose (adequately weighted) average leads to a sinusoidal
fringe pattern with the required number of bits. Since binary patterns consist of “ones” and “zeros”—and
no half-tones are involved—the nonlinear response of the projector and the camera will not play a role,
and a nearly unit contrast gray-level sinusoidal fringe pattern is obtained. Validation experiments are
presented. © 2011 Optical Society of America
OCIS codes: 120.0120, 120.2650, 100.5070.

1. Introduction tially hinders the application of calibration and com-

A simple technique for three-dimensional (3-D) pro- pensation schemes.
filing is to project a series of equally displaced In the literature, there are also references about
(fringe-shifted) sinusoidal patterns on the test object, the use of binary patterns to elude the nonlinearity
and apply phase-shifting interferometry (PSI) algo- problem [10–14]. But, to achieve a sinusoidal pat-
rithms to retrieve its 3-D shape (see, e.g., [1,2] and tern, some of the methods require blurring to a cer-
the references therein). A typical setup for digital tain degree by means of either defocusing the
fringe projection consists in a commercial video pro- projector [10–12] or digitally smoothing the acquired
jector for projecting fringes and a digital camera for patterns [13], which has the disadvantage of lower-
acquiring the distorted patterns [3]. ing the image contrast.
Any of these devices has a nonlinear response in The purpose of the present paper is to present a
the light intensity (i.e., which is often called the new approach for overcoming the nonlinear response
“gamma problem”). To get a strictly sinusoidal pat- of commercial projectors and cameras. Specifically,
tern, the nonlinearity must be overcome. A variety the proposed method consists of two stages.
of techniques have been proposed with this purpose,
e.g., projection of predistorted patterns [4], creation First, we project on the test object a temporal se-
of a lookup table for error compensation [5], use of a quence of binary fringes whose superposition (with
calibration board [6], and other techniques [7–9]. the adequate weights) reproduces a sinusoidal fringe
However, it has been found [10] that the response pattern, as required for applying PSI-reconstruction
in intensity of projectors actually changes over time, algorithms. [In order to generate the adequate num-
and thus needs frequent recalibration, which par- ber of frames (depending on the specific PSI algo-
rithm), this stage should be repeated using shifted
fringes.]
0003-6935/11/020147-08$15.00/0 Second, each image of the sequence is acquired and
© 2011 Optical Society of America a thresholding technique to generate a sequence of

10 January 2011 / Vol. 50, No. 2 / APPLIED OPTICS 147

Fig. 1. Binary sequence M 1i ðx; yÞ (with i ¼ 1; 2; …8) representing the sinusoidal pattern shown at the bottom (right corner) of the figure.

binary images is applied, i.e., emulating a 1 bit cam- 2. Theory

era acquisition. In principle, this last procedure
mitigates the problem of the nonlinear response of
A. Projection of Binary Fringes
the camera, because the obtained images will contain
only “zeros” and “ones” (similarly to the projected In the following we will work with 256
patterns), so that the camera’s nonlinear response gray-level images (i.e., 8 bit), with intensity levels
will not play an important role. Iðx; yÞ running from 0 to 255 at each point ðx; yÞ.
The technique described in the present article (Of course, the proposed method can be used
resembles some previous works on structured illumi- with images having an arbitrary number of gray
nation found in the literature (see, e.g., [14] and the levels.)
references therein). However, to the best of our One can imagine a gray-level image as the super-
knowledge, there are no references to the application position of eight (time-multiplexed) binary images
of PSI techniques by simultaneous projection and fM i g (with i ¼ 1; 2; …8), where M 1 is the most signif-
acquisition of binary patterns. icant bit of the binary representation of I. That is,
each M i ðx; yÞ takes the values 1 or 0 depending on
In Section 2, we describe the theory, and in
the integer number Iðx; yÞ.
Section 3 we will present validation experiments.

148 APPLIED OPTICS / Vol. 50, No. 2 / 10 January 2011

 The function Iðx; yÞ can be reconstructed through sinðθÞz0 ðx0 ; y0 Þ; y0 Þ, it is possible to reconstruct the
the well-known formula different frames I k ðx0 ; y0 Þ required to apply a PSI
algorithm and retrieve z0 ðx0 ; y0 Þ.
X
8 In practice, the reflectivity of the test surface will
Iðx; yÞ ¼ 2ð8−iÞ M i ðx; yÞ: ð1Þ appear as a factor multiplying the distorted binary
i¼1 patterns, so that the actually acquired image will
be of the form
In the present work, we deal with the issue of gen-
erating sequences of binary images to reproduce X
8

high-quality phase-shifted sinusoidal fringe pat- I k ðx0 ; y0 Þ ¼ Rðx0 ; y0 Þ 2ð8−iÞ M ki ðcosðθÞx0

terns, i.e., ideally we have to generate a series of i¼1

phase-shifted intensity frames given by þ sinðθÞz0 ; y0 Þ; ð6Þ

I k ðx; yÞ ¼ ð255=2Þ½1 þ cosð2πf 0 x þ αk Þ
; ð2Þ where Rðx0 ; y0 Þ (with 0 ≤ R ≤ 1) denotes the reflectiv-
ity of the surface.
where f 0 is the fringe frequency, and αk is the kth The binarization (thresholding) procedure applied
phase step required for the application of PSI algo- to the acquired images tends naturally to solve the
rithms. Each gray-level image I k ðx; yÞ has to be gen- problem of the reflectivity variations because, if
erated through a formula similar to (1) , the threshold is set low enough, the factor involving
the surface reflectivity has no effect on the final re-
X
8 sult, and the dominant role will be played by M ki,
I k ðx; yÞ ¼ 2ð8−iÞ M ki ðx; yÞ; ð3Þ which ideally already takes the value “one” or “zero.”
i¼1 The usual procedure to minimize the role of the
surface reflectivity (see, e.g., [15]) is to acquire an ad-
where fM 1i ðx; yÞg is the sequence of eight binary ditional image I 0 ðx0 ; y0 Þ under homogeneous illumi-
images corresponding to k ¼ 1, and fM 2i ðx; yÞg is a nation, so that I 0 ðx0 ; y0 Þ ∼ Rðx0 ; y0 Þ, and normalize
sequence similar to fM 1i ðx; yÞg but with the binary the frames I k ðx0 ; y0 Þ by I 0 ðx0 ; y0 Þ, to finally recover
fringes shifted a quantity ðα2 − α1 Þ=2πf 0 along the images unaffected by the reflectivity factor.
x axis, and so on.
3. Experimental Results
The binary image sequence fM 1i ðx; yÞg to be used
in our experiments (see Section 3) was generated We performed a series of validation experiments
using a standard procedure: Fig. 1 shows the image using a commercial LCD projector (Model CP-RX80,
sequence generated using MATLAB function Hitachi) with 1024 × 768 pixels. The images were ac-
dec2bin.m. (For simplicity, only a couple of periods quired with a monochromatic 8 bit digital camera
have been reproduced in the figure.) At the bottom (Model DC310, Thorlabs) with 1024 × 768 pixels, un-
(right corner) of Fig. 1, the sinusoidal pattern recon- der a viewing angle θ ≈ 33°.
structed using Eq. (3) is shown. First, to characterize the response of the camera/
projector system, we projected a digital image of a
B. Binarization of the Acquired Fringes
In 3-D profiling by fringe projection, the visual direc-
tion of the acquiring camera (CCD) makes an angle θ
with the direction of the light beam illuminating the
test surface (T), as schematically shown in Fig. 2. It is
easy to verify that

x ¼ cosðθÞx0 þ sinðθÞz0 ðx0 ; y0 Þ; ð4Þ

where ðx0 ; y0 Þ represents a point on the detector array

of the camera, and z0 ðx0 ; y0 Þ is the corresponding sur-
face profile to be retrieved.
Substituting x → cosðθÞx0 þ sinðθÞz0 ðx0 ; y0 Þ and y →
0
y in Eqs. (2) and (3), one obtains

I k ðx0 ;y0 Þ ¼ ð255=2Þf1þcos½2πf 0 ðcosðθÞx0

þsinðθÞz0 Þþαk
g
X
8
¼ 2ð8−iÞ M ki ðcosðθÞx0 þsinðθÞz0 ðx0 ;y0 Þ;y0 Þ: ð5Þ
i¼1

This expression tells us that, by acquiring Fig. 2. Experimental setup. T is the test surface and CCD is a
the (distorted) binary patterns M ki ðcosðθÞx0 þ digital camera.

10 January 2011 / Vol. 50, No. 2 / APPLIED OPTICS 149

Fig. 3. (a) Image of a digital sinusoidal pattern projected over a
Fig. 4. (a) Gray-level pattern reconstructed from Eq. (5) using the
plane orthogonal to the projection direction and acquired by the
proposed method; (b) representative intensity cut along the pat-
camera (without a binarization procedure); (b) representative in-
tern horizontal direction; (c) FFT of the intensity cut.
tensity cut along the pattern horizontal direction; (c) FFT of the
intensity cut.
In a second series of experiments, we tested the
performance of the proposed method for reconstruct-
gray-level sinusoidal pattern of (arbitrary) frequency ing ideal sinusoidal fringes: we projected a binary se-
f 0 ≈ 0:2 cm−1 over a plane surface orthogonal to the quence fM 1i ðx; yÞg (with the same fundamental
projection direction, and acquired the gray-level frequency f 0 as before) over a plane orthogonal to
images with the camera. [Clearly the lateral view the projection direction [see Fig. 1 (i ¼ 1; 2; …8)],
of the projected pattern should have a fundamental and the images acquired by the camera were binar-
frequency f 00 ¼ f 0 = cosðθÞ.] In this experiment, we did ized applying Otsu’s thresholding procedure (see,
not perform the binarization procedure. e.g., [16]).
The acquired image is shown in Fig. 3(a) (after Figure 4(a) shows the gray-level pattern recon-
applying a 3 × 3 median filter). Figure 3(b) shows a structed using Eq. (5), and applying a 3 × 3 median
representative intensity cut along the pattern hori- filter afterward.
zontal direction, while Fig. 3(c) shows its fast Fourier Figure 4(b) shows a representative intensity
cut along the pattern horizontal direction, while
transform (FFT). The FFT shows that the pattern
Fig. 4(c) shows its FFT. Figures 4(b) and 4(c) show
has large spurious frequency components in addition that the reconstructed intensity pattern is a quasi-
to the expected fundamental frequency f 00. From ideal sinusoidal pattern of frequency f 00, with con-
Figs. 3(b) and 3(c), it is clear that the response of trast very close to the unit.
the camera/projector system is nonlinear and, thus, To illustrate the performance of our method in re-
the resultant image is far from being an ideal sinu- constructing a complex surface, we make experi-
soidal pattern. ments using a plaster mask as a test surface.

150 APPLIED OPTICS / Vol. 50, No. 2 / 10 January 2011

Fig. 5. Deformed binary sequence M 1i ði ¼ 1; 2; …8Þ after binarization of the acquired images. The pattern reconstructed using Eq. (5) is
shown at the bottom (right corner) of the figure.

First, to minimize the role of the surface reflectiv- median filter at a posterior stage), the reconstructed
ity, we acquire a normalization image, I 0 ðx0 ; y0 Þ, image is a high-quality image with unit contrast.
under homogeneous illumination (see Subsection 2.B In general, one has to acquire additional images to
above). perform 3-D profiling by applying a three- or four-
Second, we projected the sequence of eight binary frame PSI algorithm. In the present work, we chose
patterns shown in Fig. 1 (i ¼ 1; 2; …8) on the test sur- a phase step of π=2 rad. Thus, we repeated the fringe
face. The images acquired by the camera were nor- projection procedure using a shifted sequence
malized by I 0 ðx0 ; y0 Þ, and then binarized using fM 2i ðx; yÞg, which was obtained from fM 1i ðx; yÞg
Otsu’s thresholding method to generate the distorted [see Fig. 1 (i ¼ 1; 2; …8)] by shifting the fringes a
binary sequence fM 1i g, which is shown in Fig. 5 quarter-period along the x axis. (Of course, the pro-
(i ¼ 1; 2; …8). cedure can be generalized to a different phase-step
By means of Eq. (5), from the distorted fM 1i g one and to arbitrary numbers of frames.)
obtains a reconstructed gray-level image I 1 ðx0 ; y0 Þ, After normalizing and binarizing the images ac-
which is shown at the bottom (right corner) of quired by the camera, we achieved the (distorted)
Fig. 5. Despite a small amount of “salt and pepper” binary sequence fM 2i g shown in Fig. 6 (i ¼ 1; 2; …8).
noise (which can be easily filtered out by a 3 × 3 By using Eq. (5), from fM 2i g a reconstructed

10 January 2011 / Vol. 50, No. 2 / APPLIED OPTICS 151

Fig. 6. Deformed binary sequence M 1i ði ¼ 1; 2; …8Þ after binarization of the acquired images. The pattern reconstructed using Eq. (5) is
shown at the bottom (right corner) of the figure.

gray-level image I 2 ðx0 ; y0 Þ is obtained, which is shown Finally, for comparison purposes, we projected on
at the bottom (right corner) of Fig. 6. the test surface a sequence of four “sinusoidal” pat-
Since the reconstructed images I 1 ðx0 ; y0 Þ and terns [similar to that shown in Fig. 3(a)] with a phase
I 2 ðx0 ; y0 Þ have unit contrast, in principle it would be shift of π=2 rad between consecutive patterns. We ac-
not necessary to acquire additional frames for re- quired the gray-level images with the camera and ap-
trieving the 3-D surface shape. From Eq. (5) it is clear plied a standard four-frame algorithm to reconstruct
that the 3-D profile will be given by the surface profile. (In this experiment, we did not

   perform the binarization procedure.)
1 255 − 2I 2 ðx0 ; y0 Þ The reconstructed profile is shown in Figs. 7(c) and
z0 ðx0 ; y0 Þ ¼ arctan
2πf 0 sinðθÞ 2I 1 ðx0 ; y0 Þ − 255 7(d) (after unwrapping and filtering as before).
x0 The reconstructed surface profile is corrupt by a
− : ð7Þ spurious periodic undulation, which is clearly visible
tanðθÞ
in Figs. 7(c) and 7(d) as a periodic pattern of vertical
Different views of the reconstructed 3-D surface fringes superimposed to the smooth surface of
(after unwrapping and filtering) are shown in the mask. This kind of phase error is a character-
Figs. 7(a) and 7(b). istic feature due to the nonlinear response of the

152 APPLIED OPTICS / Vol. 50, No. 2 / 10 January 2011

 duced by the nonlinear response of the system
camera/projector.
The advantage of the method with respect to other
approaches (e.g., [4–7]) is that we do not require ca-
librating or compensating the response of the
system, which could change over time. Another im-
portant advantage—in comparison with classical
methods using multiple-step PSI algorithms [8,9]
—is that we are not assuming that the nonlinearity
is necessarily associated to a power law, i.e., a one-
parameter gamma function. {Actually, the intensity
responses of projectors and cameras are considerably
complex; they show plateaus near the threshold- and
saturation-intensity levels, which cannot be ade-
quately described by a simple power law (see, e.g.,
Fig. 12 of [7]).}
A drawback of the proposed procedure is that
blurred fringes may cause some coding errors in
the final 3-D result, so that the measuring range will
be limited by the depth of field of the system. [In our
experiment, the depth of field of the projector/camera
system was larger than the depth of the mask to be
reconstructed (which was of the order of 6–7 cm).]
Another drawback of this technique is that we
Fig. 7. Different views of the reconstructed 3-D surface (after un-
wrapping and filtering). (a), (b) Smooth surface profile recon-
have a sequence of images (so many images as bits
structed using the proposed method; (c), (d) reconstructed we would like to have) for each frame to be used
profile using standard four-frame PSI algorithm by gray-level in conjunction with the PSI algorithm. In our experi-
fringe projection and acquisition. ments we used a commercial LCD projector and a ca-
sually available standard camera, which work at
video rates (i.e., ∼15–30 frames=s). Since we re-
camera/projector system [see, e.g., Fig. 2 in [9]). On quired two shifted 8 bit frames (i.e., two image se-
the contrary, the profile reconstructed using our quences, each of them consisting of 8 images) for
method [Figs. 7(a) and 7(b)] presents a comparatively achieving a 3-D surface profile, with the projector
smoother surface without spurious features. and camera used, we can achieve a maximum recon-
struction rate of 1–2 profiles per second.
4. Discussion and Conclusions In principle, it could appear difficult to use the pro-
In general, 3-D profiling by sinusoidal fringe projec- posed approach in real-time applications requiring
tion using PSI algorithms may be distorted by the higher reconstruction rates. However, with the intro-
nonlinear response of commercial video projectors duction of the modern digital micrometer device
and digital cameras. To solve these problems, we (DMD) digital light processing technology, the projec-
have suggested a new approach that consists in tion of binary patterns at high rates (i.e., 1 kHz) does
the simultaneous projection and acquisition of se- not represent a problem. Of course, the implementa-
quences of strictly binary patterns. tion of the method in real-time applications will also
The key issue underlying the technique is a well- require a camera with acquisition synchronized with
established principle of communication engineering, the DMD projector. An alternative to reduce the
which states that, when a signal consists only of high-acquisition-rate requirement could be to codify
“ones” and “zeros,” the distortion due to the nonlinear the binary images in the three fundamental colors.
response of the involved devices may be minimized. This issue will be investigated in future work.
Thus, the projection of binary fringes allows over-
coming the nonlinear response of the projector J. A. Ferrari and J. M. Di Martino acknowledge fi-
and, simultaneously, the binarization of the acquired nancial support from Programa de desarrollo de
Ciencias Básicas (PEDECIBA) (Uruguay) and the
images (which ideally are already binary) minimizes
Comisión Sectorial de Investigación Científica
the problem of the nonlinear response of the camera.
(CSIC, UdelaR, Uruguay).
The utilization of binary patterns in 3-D shape mea-
surements has some precedents in the literature, but References
as yet it has not been used in conjunction with PSI
1. K. Creath, J. Schmit, and J. C. Wyant, “Moiré and fringe pro-
algorithms, as in this work, where we do not require jection techniques,” in Optical Shop Testing, 3rd ed.,
any degree of defocusing. D. Malacara, ed. (Wiley, 2007), Chap. 16.
Experimentally we demonstrated that the applica- 2. H. Schreiber and J. H. Bruning, “Phase shifting interferome-
tion of our technique allows a high-quality 3-D profile try,” in Optical Shop Testing, 3rd ed., D. Malacara, ed. (Wiley,
reconstruction without the spurious features intro- 2007), Chap. 14.

10 January 2011 / Vol. 50, No. 2 / APPLIED OPTICS 153

 3. S. Zhang, “Recent progresses on real-time 3D shape measure- 11. G. Sansoni, S. Lazzari, S. Peli, and F. Docchio, “3-D imager
ment using digital fringe projection techniques,” Opt. Lasers for dimensional gauging of industrial workpieces: state-of-
Eng. 48, 149–158 (2010). the-art of the development of a robust and versatile system,”
4. P. S. Huang, C. Zhang, and F.-P. Chiang, “High-speed 3-D in First International Conference on Recent Advances in 3-D
shape measurement based on digital fringe projection,” Opt. Digital Imaging and Modeling (3DIM ’97) (IEEE, 1997), pp.
Eng. 42, 163–168 (2003). 19–26.
5. S. Zhang and P. S. Huang, “Phase error compensation for a 3-D 12. G. A. Ayubi, J. A. Ayubi, J. M. Di Martino, and J. A. Ferrari,
shape measurement system based on the phase-shifting “Pulse-width modulation in defocused three-dimensional
method,” Opt. Eng. 46, 063601 (2007). fringe projection,” Opt. Lett. 35, 3682–3684 (2010).
6. S. Zhang and S.-T. Yau, “Generic nonsinusoidal phase 13. G. Sansoni, M. Carocci, and R. Rodella, “Three-dimensional
error correction for three-dimensional shape measurement vision based on a combination of gray-code and phase-shift
using a digital video projector,” Appl. Opt. 46, 36–43 (2007). light projection: analysis and compensation of the systematic
7. H. Guo, H. He, and M. Chen, “Gamma correction for digital errors,” Appl. Opt. 38, 6565–6573 (1999).
fringe projection profilometry,” Appl. Opt. 43, 2906–2914 14. J. Salvi, J. Pagès, and J. Batlle, “Pattern codification strategies
(2004). in structured light systems,” Patt. Recog. 37, 827–849
8. B. Pan, Q. Kemao, L. Huang, and A. Asundi, “Phase error (2004).
analysis and compensation for nonsinusoidal waveforms in 15. G. Sansoni, S. Corini, S. Lazzari, R. Rodella, and F. Docchio,
phase-shifting digital fringe projection profilometry,” Opt. “Three-dimensional imaging based on gray-code light projec-
Lett. 34, 416–418 (2009). tion: characterization of the measuring algorithm and devel-
9. T. Hoang, B. Pan, D. Nguyen, and Z. Wang, “Generic gamma opment of a measuring system for industrial applications,”
correction for accuracy enhancement in fringe-projection pro- Appl. Opt. 36, 4463–4472 (1997).
filometry,” Opt. Lett. 35, 1992–1994 (2010). 16. N. Otsu, “A threshold selection method from gray level
10. S. Lei and S. Zhang, “Flexible 3-D shape measurement histograms,” IEEE Trans. Syst. Man Cybern. SMC-9, 62–66
using projector defocusing,” Opt. Lett. 34, 3080–3082 (2009). (1979).

154 APPLIED OPTICS / Vol. 50, No. 2 / 10 January 2011