sensors

Article
A Novel Simulation Method for 3D Digital-Image Correlation:
Combining Virtual Stereo Vision and Image
Super-Resolution Reconstruction
Hao Chen , Hao Li, Guohua Liu * and Zhenyu Wang

College of Civil Engineering and Architecture, Zhejiang University, Hangzhou 310058, China;
[email protected] (H.C.); [email protected] (H.L.); [email protected] (Z.W.)
* Correspondence: [email protected]

Abstract: 3D digital-image correlation (3D-DIC) is a non-contact optical technique for full-field shape,
displacement, and deformation measurement. Given the high experimental hardware costs associated
with 3D-DIC, the development of high-fidelity 3D-DIC simulations holds significant value. However,
existing research on 3D-DIC simulation was mainly carried out through the generation of random
speckle images. This study innovatively proposes a complete 3D-DIC simulation method involving
optical simulation and mechanical simulation and integrating 3D-DIC, virtual stereo vision, and
image super-resolution reconstruction technology. Virtual stereo vision can reduce hardware costs
and eliminate camera-synchronization errors. Image super-resolution reconstruction can compensate
for the decrease in precision caused by image-resolution loss. An array of software tools such as
ANSYS SPEOS 2024R1, ZEMAX 2024R1, MECHANICAL 2024R1, and MULTIDIC v1.1.0 are used
to implement this simulation. Measurement systems based on stereo vision and virtual stereo
vision were built and tested for use in 3D-DIC. The results of the simulation experiment show that
when the synchronization error of the basic stereo-vision system (BSS) is within 10−3 time steps,
the reconstruction error is within 0.005 mm and the accuracy of the virtual stereo-vision system
is between the BSS’s synchronization error of 10−7 and 10−6 time steps. In addition, after image
super-resolution reconstruction technology is applied, the reconstruction error will be reduced to
within 0.002 mm. The simulation method proposed in this study can provide a novel research path
for existing researchers in the field while also offering the opportunity for researchers without access
Citation: Chen, H.; Li, H.; Liu, G.;
Wang, Z. A Novel Simulation Method
to costly hardware to participate in related research.
for 3D Digital-Image Correlation:
Combining Virtual Stereo Vision and Keywords: 3D-DIC simulation; virtual stereo vision; image super-resolution
Image Super-Resolution
Reconstruction. Sensors 2024, 24, 4031.
https://doi.org/10.3390/s24134031
1. Introduction
Academic Editor: Stefania Perri
Digital image correlation (DIC) is a non-contact image-based optical method for full-
Received: 14 May 2024 field shape, displacement and deformation measurements [1]. DIC techniques acquire
Revised: 14 June 2024 digital images of an object at different loadings using digital-imaging devices and then per-
Accepted: 18 June 2024
form image analysis with correlation-based matching (tracking or registration) algorithms
Published: 21 June 2024
and numerical differentiation approaches to quantitatively extract full-field displacement
and strain responses of materials, components, structures, or biological tissues [2,3]. DIC
approaches can be classified as 2D-DIC or 3D-DIC according to the measurement dimen-
Copyright: © 2024 by the authors.
sions. In 2D-DIC, typically the light axis of the camera is expected to be perpendicular to
Licensee MDPI, Basel, Switzerland. the component surface during the measurements, but it is often challenging to maintain this
This article is an open access article orientation in practice. Furthermore, 2D-DIC can measure only the in-plane displacement
distributed under the terms and of the specimen and is unable to measure out-of-plane displacement. However, when
conditions of the Creative Commons specimens are under stress, they can exhibit out-of-plane displacement due to the Poisson
Attribution (CC BY) license (https:// effect, which at times can cause the 2D-DIC measurements to deviate severely from reality.
creativecommons.org/licenses/by/ Therefore, researchers combined DIC with stereoscopic vision principles, extending the
4.0/). measurement range of DIC from 2D to 3D. Compared to 2D-DIC, 3D-DIC possesses greater

Sensors 2024, 24, 4031. https://doi.org/10.3390/s24134031 https://www.mdpi.com/journal/sensors

Sensors 2024, 24, 4031 2 of 26

capabilities, a broader application range, and more accurate measurements, earning it

more extensive applications in various fields. Zhao et al. [4] conducted an in-depth study
of split-disk testing methods to evaluate the mechanical properties of filament-wound
composites. In their study, they used 3D-DIC to measure hoop strain on the outer surface
of ring specimens. Huang et al. [5] explored the fracture behavior of sharp V-notched
specimens using a combination of 3D-DIC and caustics. Wei et al. [6] proposed a multi-
camera 3D-DIC approach to monitor the deformation of a cassette structure subjected to
substantial vibrations from a shaking table. This methodology enabled the measurement of
full-field, high-resolution displacement fields for the cassette structure under significant
shaking conditions, marking a significant milestone in the field. However, in practical
applications, traditional 3D-DIC also has some obvious limitations [1,7], as follows: (1) at
least two synchronously triggered cameras are required for measurement, which greatly
increases the hardware cost; (2) precise synchronization of cameras is often complex and
challenging in scenarios demanding high-speed or high-precision measurements; (3) the
unavoidable variations in intensity and non-linear geometric distortions between image
pairs may pose difficulty in accurately matching the two images in stereo, hence reducing
the measurement accuracy. Therefore, some researchers have applied the principles of
virtual stereo vision to 3D-DIC [8–10].
Based on the principles of optics such as reflection and refraction, virtual stereo vision,
utilizing a single camera coupled with additional optical components, can capture images
of an object from at least two different views in a single shot [11–14]. Recently, virtual
stereo-vision techniques have been successfully combined with 3D-DIC. For instance, Gen-
ovese et al. [15] established a bi-prism-based virtual stereo-vision system for 3D-DIC and
verified its accuracy for measurement of 3D shapes and deformation. Pankow et al. [16]
combined a four-mirror adapter-assisted virtual stereo-vision system with DIC and mea-
sured the out-of-plane deformations of an aluminum plate under a shock-wave impact.
Mohammed et al. [17] established a miniature portable virtual stereo-vision system using a
triangular-prism mirror and two mirrors for 3D-DIC. Compared to the traditional 3D-DIC
methods, these approaches present at least the following advantages [8,18,19]: (1) complete
elimination of camera-synchronization errors; (2) reduction of the hardware cost by the cost
of at least one camera; (3) a typically smaller overall volume of the measurement system,
with a compact structural design. However, it is undeniable that these methods also lead to
the issue of loss of image resolution. In a measurement system with n virtual cameras, each
virtual camera experiences an average loss of (n − 1)/n in image resolution, and the loss of
image resolution can impact the subsequent measurement accuracy. Image super-resolution
reconstruction techniques are a potential solution to this problem. Image super-resolution
reconstruction is a classic research subject in the field of computer vision; it is used with
the goal of transforming low-resolution images into high-resolution versions and filling in
missing information [20–22]. A wealth of literature confirms that image super-resolution
reconstruction techniques based on various approaches can enhance image quality to a
certain degree [23–28]. The primary purpose of applying image super-resolution recon-
struction to images captured by the virtual stereo-vision system is to compensate for the
precision decrease caused by loss of image resolution.
3D-DIC based on the principles of virtual stereo vision has effectively reduced hard-
ware costs, making it more economical compared to traditional 3D-DIC. However, for
dynamic measurement scenarios with high-precision demands, high-precision high-speed
cameras and finely designed optical components are needed, and the hardware cost remains
considerable. In addition, the precision of DIC is influenced by various factors such as
system structural parameters, environmental lighting conditions, surface speckle settings,
pixel-matching algorithms, etc. [7,29–32]. Therefore, the development of high-fidelity simu-
lations of 3D-DIC is of significant value, as they can assist researchers in device selection,
system design, algorithm testing, precision assessment, etc., at a low cost. However, exist-
ing research on DIC simulation remains limited and is mostly carried out carried out by
generating random speckle images [33,34]. For example, Estrada and Franck [35] proposed
Sensors 2024, 24, 4031 3 of 26

a DIC random speckle image generator based on MATLAB. This generator first produces a
reference image with random speckles and then deforms this reference image according to
different displacement-field functions to attain a deformed image. It then uses the refer-
ence and deformed images to implement 2D-DIC. Wang et al.’s [36] research went even
further, integrating the principles of stereo vision and developing a stereo speckle image
generator. The stereo speckle images produced by this generator can be used for 3D-DIC
simulation. There are also similar studies that mainly differ in the use of various speckle
patterns [37–40]. These methods consider overly ideal situations, staying at the level of
mathematical derivation and basic image processing without considering the potential
noise impact caused by system structure design, environmental lighting conditions, sample
optical attributes, and camera imaging, among other factors.
The core purpose of this research is to propose a completely simulation-based ex-
perimental method. It integrates optical simulation, mechanical simulation, a 3D-DIC
algorithm, and image super-resolution reconstruction model and considers many factors
that exist in real experiments, which greatly enhances the fidelity of the simulation. Not
only does this approach offer a new perspective for existing researchers in this field, but
it also enables researchers who are without access to costly 3D-DIC hardware facilities to
participate in related research. The main contributions of this research are as follows:
1. The research innovatively integrates optical and mechanical simulations, establishing
a full 3D digital image correlation (3D-DIC) simulation method. This integration offers
more comprehensive and accurate analysis.
2. The research combines 3D-DIC, virtual stereo vision, and image super-resolution
reconstruction technology. This combination brings together the advantages of both
technologies and expands the application scenarios.
3. The application of image super-resolution reconstruction in virtual stereo vision is
in its infancy, and this research explores the feasibility of this approach, aiming to
compensate for the reduction in accuracy caused by loss of image resolution.
4. The novel method amalgamates multiple software tools for different parts of the simu-
lation. It utilizes ANSYS SPEOS 2024R1 [41] and ZEMAX 2024R1 [42,43] for optical
simulation and uses ANSYS MECHANICAL 2024R1 [44] for mechanical simulation,
with 3D-DIC carried out via MULTIDIC v1.1.0 [45,46].
The rest of this paper is organized as follows. Section 2 provides an overview of related
theories, Section 3 introduces the setup of the simulation experiment, Section 4 elaborates
on the experimental results, and Section 5 gives the conclusions.

2. Methods
2.1. 3D Digital Image Correlation
2.1.1. Correlation Function
The sample surface is usually filled with speckle patterns, which serve as carriers of
deformation information and deform with the surface. As shown in Figure 1, a fraction of
the initial speckle pattern is designated as the reference subset, while the corresponding
section of the speckle pattern subsequent to deformation is identified as the target subset.
The key to digital-image correlation is to establish the matching relationship between these
two subsets. Zero-mean normalized cross-correlation [47,48] (ZNCC) is a widely used
method for evaluating the similarity between a reference subset and a target subset. The
formulation of ZNCC is as follows [49]:

∑ ∑ F(x, y) − F G(x′ , y′ ) − G
  
CZNCC = q q 2 (1)
∑∑ ( ∑∑ ( ′ ′
 
F x, y ) − F ] 2 × G x , y ) − G

where F(x, y) is the gray-scale function of the reference subset, G(x′ , y′ ) is the gray-scale
function of the target subset; and F and G, respectively, represent the average gray-scale
 an optimization method based on the Gaussian window function, which combines tradi-
tional ZNCC with the Gaussian window function, aiming for better robustness. Firstly, a
two-dimensional Gaussian window function is introduced, as follows:
Sensors 2024, 24, x FOR PEER REVIEW 4 of 28
[ ( , )]
Sensors 2024, 24, 4031 G (x, y; D ) = e 4 of(2)
26

where D(x, y) = (x − x ) + (y − y ) represents the distance from any location (x, y)

where F(x, y) is the gray-scale function of the reference subset, G(x , y ) is the gray-scale
in the subset to the center ofand
the subset (x , y ). DTheislarger
directly
therelated ,to
thethe sizesimilar
of the
values
functionofof
thethe
reference subset
target subset; F‾ and
and the G‾, subset.
target respectively, represent Cthe
ZNCCaverage more
gray-scale
effective subset, as shown in Figure 2.
the reference
values subset and
of the reference the target
subset and thesubset
target subset. The larger the C
are. , the more similar
the reference subset and the target subset are.
Although ZNCC has been widely used in DIC, it also has some problems in practical
use, among which the more prominent is the influence of the choice of subset size on the
final result. Traditional ZNCC is sensitive to the subset size, and improper choice of subset
size can easily lead to a decrease in measurement accuracy. Therefore, this article adopts
an optimization method based on the Gaussian window function, which combines tradi-
tional ZNCC with the Gaussian window function, aiming for better robustness. Firstly, a
two-dimensional Gaussian window function is introduced, as follows:
[ ( , )]
(2)
G (x, y; D ) = e
where D(x, y) = (x − x ) + (y − y ) represents the distance from any location (x, y)
in the subset to the center of the subset (x , y ). D is directly related to the size of the
effective subset, as shown in Figure 2.
Figure 1.
Figure 1. Schematic
Schematic diagram
diagram of
of reference
reference subset
subset and
and target
target subset.
subset.

Although ZNCC has been widely used in DIC, it also has some problems in practical
use, among which the more prominent is the influence of the choice of subset size on
the final result. Traditional ZNCC is sensitive to the subset size, and improper choice of
subset size can easily lead to a decrease in measurement accuracy. Therefore, this article
adopts an optimization method based on the Gaussian window function, which combines
traditional ZNCC with the Gaussian window function, aiming for better robustness. Firstly,
a two-dimensional Gaussian window function is introduced, as follows:
[D(x,y)]2
−
2D2
Gw (x, y; D0 ) = e 0 (2)
q
2
where D(x, y) = (x − x0 )2 + y − y0 represents the distance from any location (x, y)


in the subset to the center of the subset x0 , y0 . D0 is directly related to the size of the
Figure 1. Schematic diagram of reference subset and target subset.
effective subset, as shown in Figure 2.
Figure 2. The influence of D value on subset.

The method used to calculate the optimization method based on the Gaussian win-
dow function is as follows [32]:
∑∑[F(x, y) − F‾]G (x, y; D )[G(x , y ) − G‾]G (x , y ; D )
C = (3)
∑∑G (x, y; D )[F(x, y) − F‾] × ∑∑G (x , y ; D )[G(x , y ) − G‾]

Figure 2.
Figure 2. The D0 value
The influence of D value on
on subset.
subset.

The method
The method used
usedto
tocalculate
calculatethetheoptimization
optimization method
method basedbased onon thethe Gaussian
Gaussian window
win-
function is as follows [32]:
dow function is as follows [32]:
′ ′ )‾−
∑ F(x,y)y)−−F‾]G
∑∑[F(x,
∑ F G(x,
w (y;
x, D )[G(x − G]GG (x
0 ) G,(yx ), y , y(x; ′D, y)′ ; D0 )
   
y; D Gw
C
CGZNCC ==q (3)
 (3)
y) − F‾] × G‾′]) − G 2
q
∑ Gw(x,
∑∑∑G (x,y;y;DD)[F(x, ∑∑G ∑ ∑(xGw , y(x; ′D, y)[G(x
′; D ) , G y ()x− ′, y
 
2
0 ) F(x, y) − F] × 0
Sensors 2024, 24, x FOR PEER REVIEW 5 of 28

Sensors 2024, 24, 4031 5 of 26

It can be seen that C is based on ZNCC, and a two-dimensional Gaussian-win-
dow-function term is added.

2.1.2. It can be
Shape seen that CGZNCC is based on ZNCC, and a two-dimensional Gaussian-
Function
window-function term is added.
When the shape changes significantly, the correlation between the reference subset
and the
2.1.2. targetFunction
Shape subset will degrade and the shape function can map the coordinate set of
the reference subset
When the shape before deformation
changes to thethe
significantly, coordinate setbetween
correlation of the target subset after
the reference de-
subset
formation with the purpose of restoring the high correlation between the reference
and the target subset will degrade and the shape function can map the coordinate set subset
and thereference
of the target subset.
subsetWhen considering
before deformation thetoexpansion deformation,
the coordinate set of theshear deformation,
target subset after
and rigid body rotation of the target subset, this mapping relationship can
deformation with the purpose of restoring the high correlation between the reference be expressed
subset
as follows
and [49,50]:
the target subset. When considering the expansion deformation, shear deformation,
and rigid body rotation of the target subset, this mapping relationship can be expressed as
⎧x = x + u + ∂u dx + ∂u dy
follows [49,50]: ⎪ ∂x ∂y
 (4)
 ⎨x′ = x + u0 + ∂v ∂u0 ∂v
∂u
+ ∂y0 dy
y = y + v + ∂x dx
dx + dy
⎪ ∂x0 ∂y (4)
 ⎩y′ = y + v0 + ∂v ∂v 0
∂x dx + ∂y dy
where (x, y) represent the coordinates of any point in the reference subset and (x ,′ y )′
where (x,the
represent y) coordinates
represent the in coordinates of anytarget
the corresponding pointsubset;
in the reference subset and
(u , v ) represent x ,y )
the (rigid-
represent the coordinates in the corresponding target subset; (u0 , v0 ) represent the rigid-
body displacement in the x and y directions, respectively; and ∂u0, ∂u0, ∂v0, and ∂v0
body displacement in the x and y directions, respectively; and ∂x , ∂y , ∂x , and ∂y
symbolize the displacement
symbolize the displacementgradients
gradientsofofthethe subset.
subset. TheThe mapping
mapping relationship
relationship of theofabove
the
above formula is also called the first-order shape function. Figure 3 shows the schematic
formula is also called the first-order shape function. Figure 3 shows the schematic diagram
diagram of the geometric
of the geometric model ofmodel of the first-order
the first-order shape function.
shape function.

Figure
Figure3.3.Geometric
Geometricdiagram
diagramof
offirst-order
first-ordershape
shapefunction.
function.

2.2.Virtual
2.2. VirtualStereo
StereoVision
Vision
Virtualstereo
Virtual stereovision
visionisis typically
typically composed
composed of aofcamera
a cameraandand structurally
structurally designed
designed op-
optical components [51,52]. By utilizing the principles of light reflection and refraction,
tical components [51,52]. By utilizing the principles of light reflection and refraction, an an
image collected at a given time contains images of the same test object from different
image collected at a given time contains images of the same test object from different an- angles.
The advantages
gles. The advantagesof this
of design include
this design complete
include elimination
complete of theofsynchronization
elimination error
the synchronization
present in traditional stereo vision, which makes it highly suitable for the measurement
error present in traditional stereo vision, which makes it highly suitable for the measure-
of dynamic objects; a measurement system that is more compact and conducive to the
ment of dynamic objects; a measurement system that is more compact and conducive to
development of portable devices; saving at least the hardware cost of a camera and a
the development of portable devices; saving at least the hardware cost of a camera and a
synchronization-triggering device. This section mainly introduces the basic principles of
two types of virtual stereo vision.
Sensors 2024, 24, x FOR PEER REVIEW 6 of 28

synchronization-triggering device. This section mainly introduces the basic principles of

Sensors 2024, 24, 4031 6 of 26
two types of virtual stereo vision.

2.2.1. Virtual Stereo Vision based on a Planar Mirror

2.2.1. Virtual Stereo Vision Based on a Planar Mirror
Figure 4’s geometric optics model describes a virtual stereo-vision system composed
Figure 4’s geometric optics model describes a virtual stereo-vision system composed
of a camera and four planar mirrors. Here, the four planar mirrors use optical reflections
of a camera and four planar mirrors. Here, the four planar mirrors use optical reflections to
toguide
guide light rays from the object to the same camera from two different angles. Specifi-
light rays from the object to the same camera from two different angles. Specifically,
cally, the
the camera camera generates
generates virtualvirtual
cameras C1L andCC1R and
cameras C the
through through the reflections
reflections of planar
of planar mirrors
mirrors M and M . C and C create virtual cameras C and C
M1L and M1R . C1L and C1R create virtual cameras C2L and C2R through the reflectionsthrough the re-
flections
of planarofmirrors
planarMmirrors M and M . Eventually, C and C can capture images
2L and M2R . Eventually, C2L and C2R can capture images from two
from two perspectives
perspectives through a through a single
single shot shotstereo
to achieve to achieve stereo vision.
vision.

Figure 4. Geometric optical model of virtual stereo vision based on four planar mirrors.
Figure 4. Geometric optical model of virtual stereo vision based on four planar mirrors.
To ensure the appropriate overlap of the fields of view of the left and right virtual
To ensure
cameras, the appropriate
it is necessary overlap
to calculate of the the
and design fields of viewconstruction
geometric of the left and right
of the virtual
system.
cameras, it is necessary to calculate and design the geometric construction
Considering the system to be symmetrical, we assume the intersection point O of the planar of the system.
Considering
mirrors M1L theandsystem
M1R as tothebeorigin.
symmetrical,
The fieldwe assume
of view thereal
of the intersection
camera ispoint O ofinthe
the x-axis thepla-
nar
horizontal M andand
mirrorsdirection M theasz-axis
the origin. The field
in the depth of view
direction, of then
and the real camera positions
the relative is the x-axis
inofthe
thehorizontal
camera and direction
the four and themirrors
planar z-axis in
inthe
the depth
systemdirection, and then
can be uniquely the relative
determined bypo-
four structural
sitions parameters:
of the camera and the thefour
angles α and
planar β at which
mirrors in thethe planarcan
system mirrors are placed,
be uniquely deter-
the distance
mined by four d structural
between the real camera’s
parameters: theapex
anglesandαtheand β atand
origin, whichthethe
coordinate L of the are
planar mirrors
intersection of the line on which the planar mirror lies with the x-axis.
placed, the distance d between the real camera’s apex and the origin, and the coordinate The placement
of theΦ,
Langles the baselineofdistance
intersection the line B,onand the fields
which of view
the planar σz and
mirror thez-axis
in the
liesσxwith andThe
x-axis. x-axis
place-
directions of the virtual cameras C and C can be obtained as follows
ment angles Φ, the baseline distance B, and the fields of view σ and σ in the z-axis
2L 2R [8]:

= 90◦ +C2α −and

and x-axis directions of the virtual ϕcameras 2β
C can be obtained as follows(5)
[8]:
ϕ = 90∘ + 2α − 2β2 (5)
B = 2dsin 2(β − α) + 4Lsin β (6)
( B = 2d sin 2(β − α) + 4L
sin β (6)
Hmin = B2 tan ϕ − d + d′
σz = (7)
B (ϕ + θ ) − d + d′
Hmax = B2 tan


H = tan ϕ − (d + d )
σ = 2 (7)
B
H = tan(ϕ + θ) − (d + d )
2
 Sensors2024,
Sensors 2024,24,
24,4031
x FOR PEER REVIEW 7 of 28
7 of 26

B B B
(H + d + d ) cot
⎧
β−2 B tan
h ϕ⩽H⩽ − (d + d
)i
′ 2 B cot ϕ + cot(ϕ
B + θ) ′
σ =  H + d + d cot β − 2 2 tan ϕ ⩽ H ⩽ cot ϕ+cot(ϕ+θ) − d + d (8)
B
σx =⎨ − (H + d + d ) cot(ϕ B (8)
+ θ) H − (d + d′
)
h i
B ′
B
⎩2 2 − H + d + d cot(ϕ + θ) Hcot > ϕcot+ϕcot(ϕ +
+cot(ϕ+θ)θ)− d + d
where H and H represent the coordinate range of the effective field of view in the
where Hmin and Hmax represent the coordinate range of the effective field of view in the
z-axis direction, H is the coordinate describing the effective field of view in the z-axis di-
z-axis direction, H is the coordinate describing the effective field of view in the z-axis
rection, and d is the distance between the virtual camera C (C ) and the real camera
direction, and d′ is the distance between the virtual camera C2L (C2R ) and the real camera
in the z-axis direction.
in the z-axis direction.
Intheory,
In theory,thethe model
model of of
thethe aforementioned
aforementioned virtual
virtual stereo
stereo vision
vision can
can be be determined
fully fully deter-
mined
by by the internal
the internal parameters
parameters of the and
of the camera camera and the
the four four structural
structural parametersparameters α, L.
α, β, d, and β,
d, and L. However, in reality, it is difficult to ensure the precise placement
However, in reality, it is difficult to ensure the precise placement of each component in the of each com-
ponent so
system, in the
the system, so the above
above geometric geometric
derivation derivation
is mainly used is
formainly used for the
the preliminary preliminary
determination
of the system’s structural parameters. To ensure measurement accuracy, it is stillaccuracy,
determination of the system’s structural parameters. To ensure measurement necessaryit
is still necessary to calibrate the system subsequently. The calibration of
to calibrate the system subsequently. The calibration of the virtual stereo-vision systemthe virtual stereo-
visionincorporate
must system must theincorporate
following two the considerations:
following two considerations:
(1) As
(1) As shown
shown in in Figure
Figure5, 5,first
firstdivide
divideallallcollected
collectedcalibration
calibrationimages
imagesintointotwotwoparts,
parts,use
use
pure black
pure black pixels
pixels toto fill
fill in
in the
the lost
lost part
part of
of the
the image,
image, andand finally
finally treat
treat the
the obtained
obtained
images as
images as images
images collected
collected by by the
the general
general stereo-vision
stereo-vision system
system and
and complete
complete single-
single-
camera calibration and stereo calibration;
camera calibration and stereo calibration;
(2) The
(2) The internal
internal parameters
parameters of of the
the two
two virtual
virtual cameras
cameras should
should theoretically
theoretically be be the
the same,
same,
but the
but the implementation
implementation of of the
the step
step above
above will
will most
most likely
likely result
result in
in inconsistent
inconsistent cali-
cali-
bration results
bration results for
for the
the internal
internal parameters
parameters of of the
the two
two virtual
virtual cameras.
cameras. In In view
view of
of this,
this,
aa single-camera
single-camera calibration
calibration can can bebe performed
performed withwithall
all image
image pairs,
pairs,and
and thethe result
resultcan
can
serve as
serve as the
the common
common internal
internal parameters
parameters of ofthe
thetwo
twovirtual
virtualcameras.
cameras.

Figure5.5.Schematic
Figure Schematicdiagram
diagramof
ofimage
imagepreprocessing
preprocessingfor
fora avirtual
virtualstereo-vision
stereo-visionsystem.
system.

2.2.2.
2.2.2. Virtual
VirtualStereo
StereoVision
VisionBased
Basedon onaaPrism
Prism
By
Byplacing
placingaaprism
prism in
in front
front of
of the
the camera
camera lens,
lens, due
due toto the presence of
the presence refraction effects,
of refraction effects,
the
thecamera
cameracan canalso
alsocapture
captureimages
imagesfrom
fromdifferent
differentperspectives
perspectivesininaasingle
singleshot
shot[18].
[18].
Figure
Figure66isisaa schematic
schematicdiagram
diagramof of the
the geometric
geometricoptical
opticalmodel
modelof of virtual
virtualstereo
stereovision
vision
based
based on a bi-prism. For easy differentiation, the red line represents the light path
on a bi-prism. For easy differentiation, the red line represents the light path of
of the
the
left
left virtual camera, the blue line represents the light path of the right virtual camera,and
virtual camera, the blue line represents the light path of the right virtual camera, and
the
the thick
thick line
linerepresents
representsthe thebi-prism.
bi-prism. Considering
Considering the the system
system symmetry,
symmetry, the thegeometric
geometric
relationship
relationship of the model can be determined by the following three structural parameters:
of the model can be determined by the following three structural parameters:
the
the base
base angle
angle ββ ofofthe
thebi-prism,
bi-prism,the the height
height h,h,and andthe
the distance
distance DD between
betweenthe thecamera’s
camera’s
optical center and the vertex of the bi-prism. If the actual camera’s
optical center and the vertex of the bi-prism. If the actual camera’s field-of-view field-of-view angle
angleθ isθ
known,
is known, the effective field-of-view range of the system can be determined
the effective field-of-view range of the system can be determined by the angles by the angles α′
and ′
φ between the light ray and
α and φ between the light ray the
andnormal
the normalto the tobase of the
the base ofbi-prism. According
the bi-prism. to the
According to
 be described by the following formula [53]:
sin α = ksin α
sin φ = ksin φ
(9)
Sensors 2024, 24, 4031
sin(θ + β) = ksin(α + β)
8 of 26
sin β = ksin(β − φ)
where k is the coefficient of the bi-prism material. Combining the above two equations
canof
law solve α andthe
refraction, φ ,refraction
as followsbehaviors
[18]: of the bi-prism’s base and oblique edge can be
described by the following formula [53]: 1
α = arcsin ′ ksin arcsin k sin(θ + β) − β
sin α = k sinα (10)
 1
φ = arcsin
sin φ′ ksin β − arcsin sin β

= k sinφ

k (9)
 sin(θ + β) = k sin(α + β)
) field-of-view width PS and QS at

Furthermore, through geometric

sin β =derivation,
k sin(β − φthe
the position g from the bottom edge of the bi-prism can be calculated by the following
where k is[18,54]:
formula the coefficient of the bi-prism material. Combining the above two equations can
solve α′ and φ′ , as follows [18]:
PS = m + gtan α
⎧
QS = htan
n φh+ gtanhφ
 i io
 α′ = arcsin k sin arcsin k1 sin(θ + β) − β
 (11)
⎨m = Dsin θ(cos β − sin β tan α)
+ htan α (10)
 φ′ =⎩arcsin k sin sin(90 θ − β)1 sin β
β −−arcsin
n h  io

k

Figure6.6.Geometric
Figure Geometricoptical
opticalmodel
modelofofvirtual
virtualstereo
stereovision
visionbased
basedon
onaabi-prism.
bi-prism.

Furthermore, throughthe
In addition, changing geometric
geometric derivation,
shape of the theprism,
field-of-view width
as by using PS and QS
a triangular at
prism
the position g fromprism,
or multi-pyramid the bottom edge ofthe
can increase thenumber
bi-prism of can be calculated
virtual by the7 following
cameras. Figure shows the
formula
schematic[18,54]:
of virtual stereo vision based on a triangular prism. With the derivation of more
′

complex geometric optics, 
 corresponding
PS = m + gtanα system-design schemes can be given, which is a
method not discussed further here. However, it should be noted that although the virtual

QS = htanφ + gtanφ ′
(11)
stereo-vision system based  on a triangular
Dsinθ ( cos β prism
− sin βtan loses
α ) at least 2/3 of the field of view and
image resolution, it also hasma = + htanα


distinctsinadvantage:
(90−θ−β) each camera pair used in virtual stereo
vision can correct
In addition, each other’s
changing posture through
the geometric shape ofalgorithms
the prism, as [55].
byIn other
using words, since
a triangular any
prism
two virtual cameras in it are sufficient to perform the task of three-dimensional
or multi-pyramid prism, can increase the number of virtual cameras. Figure 7 shows the measure-
ment, theofredundant
schematic information
virtual stereo can be
vision based on used to establish
a triangular prism.anWithoverdetermined
the derivationsystem
of moreof
equations and thus realize more robust measurements.
complex geometric optics, corresponding system-design schemes can be given, which is a
method not discussed further here. However, it should be noted that although the virtual
stereo-vision system based on a triangular prism loses at least 2/3 of the field of view
and image resolution, it also has a distinct advantage: each camera pair used in virtual
stereo vision can correct each other’s posture through algorithms [55]. In other words,
since any two virtual cameras in it are sufficient to perform the task of three-dimensional
measurement, the redundant information can be used to establish an overdetermined
system of equations and thus realize more robust measurements.
Sensors 2024,
Sensors 24,24,
2024, x FOR
4031 PEER REVIEW 9 26
9 of of 28

Figure
Figure7.7.Virtual
Virtualstereo
stereo vision diagrambased
vision diagram basedon
ona atriangular
triangular prism
prism [56].
[56].

2.3.Image
2.3. ImageSuper-Resolution
Super-Resolution Reconstruction
Reconstruction
Eventhough
Even though virtual
virtual stereo
stereo vision
vision can
cancompletely
completelyavoid avoidthe
theproblem
problem of of
camera
camerasyn-
syn-
chronization that
chronization that exists
exists in
in traditional
traditionalstereo
stereovision,
vision,ititisisundeniable
undeniable that it also
that leads
it also to the
leads to the
problemof
problem ofloss
loss of
of image
image resolution.
resolution.For
Forvirtual
virtualstereo
stereovision with
vision withn virtual cameras,
n virtual each
cameras, each
virtual camera loses n − 1/n image resolution on average. The loss of image resolution will
virtual camera loses n − 1/n image resolution on average. The loss of image resolution will
affect the accuracy of subsequent measurements. The application of image super-resolution
affect the accuracy of subsequent measurements. The application of image super-resolu-
reconstruction technology to the images captured by virtual stereo vision aims to improve
tion reconstruction technology to the images captured by virtual stereo vision aims to im-
the accuracy of system measurement.
prove the accuracy of system measurement.
2.3.1. Network Architecture
2.3.1. Network
The imageArchitecture
super-resolution network used in this research was optimized based on
The imagesuper-resolution
the enhanced super-resolution networkadversarial
generative used in this research
networks [25]was optimized
(ESRGAN). Thebased
three on
main
the improvements
enhanced made to the
super-resolution network are
generative as follows:
adversarial networks [25] (ESRGAN). The three
main
1. improvements made to the
The use of an Adaptive network
Weighted are as
Basic follows:
Block with noise inputs allows for more
1. effective
The use ofresidual learning
an Adaptive of local noise.
Weighted This helps
Basic Block with the network
noise inputsbetter
allows understand
for more ef-
and mitigate the impact of noise in the image, which enhances the clarity
fective residual learning of local noise. This helps the network better understand and qualityand
of the final
mitigate theoutput.
impact of noise in the image, which enhances the clarity and quality of
2. The adoption of a multi-scale reconstruction enables the full utilization of both low-
the final output.
frequency and high-frequency residuals. This means that the network can capture
2. The adoption of a multi-scale reconstruction enables the full utilization of both low-
both the larger structures and the finer details in the image, leading to a higher-
frequency
resolution and high-frequency residuals. This means that the network can capture
output.
both the larger structures
3. The fine-tuning of the loss and the finer
function detailsthe
balances in the image, leading
perceptual quality to
anda higher-reso-
synthetic
lution output.
artifacts of the super-resolution network. This ensures that the output image not only
3. The
hasfine-tuning of the but
a high resolution, lossalso
function balances
appears naturalthe
andperceptual
realistic. quality and synthetic ar-
tifacts of the super-resolution network. This ensures that the output image
The improved network was named the multi-scale multi-adaptive-weights not only
residual
has a high resolution, but also appears natural and realistic.
super-resolution generative adversarial network (MMRSRGAN). The overall framework of
MMRSRGAN is shown
The improved in Figure
network was 8.named the multi-scale multi-adaptive-weights residual
The MMRSRGAN
super-resolution generator
generative starts from
adversarial a single
network convolutionalThe
(MMRSRGAN). layer (3 × 3framework
overall kernels,
of64MMRSRGAN
feature maps,isstride
shown 1) in
forFigure
initial 8.
feature extraction, which can be formulated as fol-
lows [26]:
x0 = f 0 ( ILR ) (12)
where f 0 is the initial feature extraction function of the low-resolution image ILR and x0 is
the output feature map of the initial convolution layer.
2024, 24, x FOR PEER
Sensors REVIEW
2024, 24, 4031 10 of 28 10 of 26

Figure 8. MMRSRGAN
Figure 8.architecture.
MMRSRGAN architecture.
Sensors 2024, 24, 4031 11 of 26

In the MMRSRGAN, a basic residual unit for non-linear feature mapping called multi-
adaptive-weights residual in residual dense block (MRRDB) is adopted. This basic residual
unit is based on the residual dense block (RDB) structure and represents an improvement
on the residual in residual dense block (RRDB) basic residual unit previously used in
ESRGAN. The ESRGAN uses a fixed residual scaling of 0.2 in each DB, while this research,
for each DB, uses 11 independent weights instead. These weights can be adaptively learned
after the algorithm is given an initial value, thus more effectively transmitting information
and gradients. Based on the above improvements, for the n-th MRRDB unit, the input xn
and the output xn+1 can be expressed as follows:

xn+1 = f MRRDB ( xn ) = λnb ( xn6 ) + λna xn (13)

where λna and λnb are the two independent weights of the n-th MRRDB unit and xn6 can be
solved by the following equation:
 n n n

 xn1 = λr 0 f RRDB ( xn ) + λ x 0 xn + λn0 Gn
n n n

xn2 = λr 1 f RRDB ( xn1 ) + λ x 1 xn1 + λn1 Gn




 x = λ n2 f n2 n2

r RRDB ( xn2 ) + λ x xn2 + λn Gn

n3
n n n (14)


 xn4 = λr 3 f RRDB ( xn3 ) + λ x 3 xn3 + λn3 Gn
 n n n



 xn5 = λr 4 f RRDB ( xn4 ) + λ x 4 xn4 + λn4 Gn
 n5 n5 n
xn6 = λr f RRDB ( xn5 ) + λ x xn5 + λn5 Gn


n n n
where λr k , λ x k , and λnk are three independent sets of weights for the MRRDB unit and Gn
is the added Gaussian noise input.
To better utilize the features from the non-linear mapping module, the AWMSR
module [57] has been referenced. The output xn+1 serves as the input to the AWMSR unit,
and the super-resolution image ISR can be represented as follows:
!
n
ISR = f rec ∑ αi f AW
i
MSR ( xn+1 ) (15)
i =0

where f rec represents the final convolution layer, f AW i

MSR represents the i-th scale branch in
the AWMSR unit, and αi is the adaptive weight of the i-th scale branch.
For the discriminator, we use the relativistic discriminator, which estimates the proba-
bility from an average perspective, assuming that the given real data are more realistic than
the fake data, rather than simply predicting the real or fake data. This kind of relativistic
discriminator can be expressed as follows: [25,58]:




 DRa Ir , I f = σ C ( Ir ) − E I f C I f → 1 (more real than f ake)


 (16)
Ra I f , Ir = σ C I f − E Ir (C ( Ir )) → 0 (less real than f ake)
 D

where E I f represents the average operation of all fake data in a mini-batch sample and E Ir
represents the average operation of all real data.

2.3.2. Loss Functions

Loss functions play crucial roles in deep learning-based image super-resolution. Clas-
sic super-resolution models mostly aim to minimize the peak signal-to-noise ratio (PSNR)
between the super-resolution images and the ground-truth high-resolution image to achieve
convergence. Given the ill-posed nature of the image super-resolution problem, it is usually
difficult to match the textural details of super-resolution images with their counterparts
in real high-resolution images in a “pixel-to-pixel” manner. As a result, the PSNR-driven
models tend to make the image smoother. In developing the SRGAN [27], the authors
replaced the PSNR-driven pixel-space loss with a loss in the feature space defined by feature
Sensors 2024, 24, 4031 12 of 26

maps, denoted as φ(i, j), which represents the j-th convolution (after activation) before the
i-th maxpooling layer within the pre-trained VGG19. The authors used φ(2, 2) and φ(5, 4)
in the following experiments. Building upon SRGAN, ESRGAN uses VGG features before
the activation layers to obtain denser feature representations (before activation) while
keeping the consistency of the super-resolution image brightness. Moreover, ESRGAN
keeps the l1 norm-based content-loss term (weighted by a factor of 0.01) to balance the
perceptual-driven solutions. Due to the very small weight of the content loss, ESRGAN
proposed the “network interpolation” method, which is a weighted average of the two
networks trained with perceptual loss and l1 loss to balance the perceptual-driven and
l1 MSE represent the content loss based on the l norm
PSNR-driven solutions. Let lSR and lSR 1
and mean squared error (MSE), respectively. They can be calculated using the following
equations [25,27]:

1 W H

l1
 lSR = W H x∑ ∑ | IHR ( x, y) − ISR ( x, y)|



=1 y =1
(17)
1 W H
MSE = 2
∑ ∑ [ IHR ( x, y) − ISR ( x, y)]

 lSR


W H xws=1 y=1

where W and H represent the width and height of the IHR image, respectively. The VGG-
VGG/φ(i,j)
based perceptual loss lSR can be expressed as follows: [59]:

VGG W
i,j H
i,j h i2
1
∑ ∑
φ(i,j)
lSR = φ(i, j) IHR ( x, y) − φ(i, j) ISR ( x, y) (18)
Wi,j Hi,j x=1 y=1

where Wi,j and Hi,j represent the dimensions of the feature map φ(i, j) within the VGG
Ra , can be expressed as follows:
network. The adversarial loss for the generator, denoted as lSR
h 
i h 
i
Ra
lSR = − E Ir log 1 − DRa Ir , I f − E I f log DRa I f , Ir (19)

Considering the specific application scenario of the image super-resolution in this

research, we made slight adjustments to the total loss function in ESRGAN. First, we rebal-
anced the lower-level and higher-level perceptual losses derived from the VGG network
to act collectively as the perceptual-loss component. Then, we replaced the l1 norm-based
content loss with the MSE-based content loss, thereby reducing the generation of “halluci-
nations” or artifacts.
The total loss function in ESRGAN, denoted as lSR ESRGAN , can be expressed as fol-

lows: [25]:
VGG
ESRGAN φ(5,4) Ra l1
lSR = lSR + λlSR + ηlSR (20)
MMRSRGAN ,
After fine-tuning, the total generator loss function in this research, denoted as lSR
can be expressed as follows:
VGG VGG
MMRSRGAN φ(2,2) φ(5,4) Ra MSE
lSR = αlSR + (1 − α)lSR + λlSR + ηlSR (21)

where α, λ and η are hyperparameters used to balance various loss terms.

2.3.3. Evaluating Indicator

The application of image super-resolution reconstruction technology to the images
collected by the virtual stereo-vision system aims to improve the accuracy of the system
measurement. To evaluate the performance of image super-resolution networks, we pro-
posed an evaluation metric based on visual-measurement accuracy. The accuracy of visual
measurements can be determined by the 3D reconstruction error [60], which is the distance
Sensors 2024, 24, 4031 13 of 26

between the reconstructed 3D coordinate points and the actual coordinate points, with the
average value taken as a statistical measure.
Let us assume that the average 3D reconstruction error before the application of image
super-resolution reconstruction is Ep1 and that the average 3D re-projection error after
image super-resolution reconstruction is Ep2 . The relative value Epr between Ep1 and Ep2 is
used as a quantitative indicator for assessing the performance of the image super-resolution
model and is calculated as follows:
Ep1 − Ep2
Epr = × 100% (22)
Ep1

where Epr < 1. The closer Epr is to 1, the greater the precision of the visual-measurement
results and thus the better the performance of the image super-resolution network. In an
ideal scenario, if Epr = 1, it means that there is no error between the true value and the
measurement value after image super-resolution reconstruction. When Epr = 0, it indicates
that image super-resolution reconstruction has no effect on precision. When Epr < 0, it
indicates that a negative effect has been produced.

3. Experimental Setup
All experiments in this research were simulation experiments, and the main software
tools involved were ANSYS SPEOS 2024R1 [41] (SPEOS), ANSYS ZEMAX OPTIC STUDIO
2024R1 [42,43] (ZEMAX), ANSYS MECHANICAL 2024R1 [44] (MECHANICAL), and MUL-
TIDIC v1.1.0 [45,46]. The specific experimental process is as follows (the main parameter
settings can be found in Appendix A):
1. Simulation Preparation Works
As shown in Figure 9, a tensile test of a sample with one fixed end was simulated in
MECHANICAL, and the coordinate information and deformation information of all nodes
were exported. This information was not only used for geometric structure import in SPEOS,
but was also considered to represent true values, which were subsequently compared with
the measurement values obtained from optical simulation. Then, the camera lens, planar
Sensors 2024, 24, x FOR PEER REVIEW 14 of 28
mirrors, and prisms were designed in ZEMAX and the designed parameters were imported
into the simulation camera in SPEOS.

(a) (b)
Figure 9.
Figure 9. Schematic
Schematic diagram
diagram of the sample
of the sample tensile
tensile simulation
simulation experiment
experiment (a)
(a) and
and simulated
simulated lens
lens
design (b).
design (b).

2. ItBuild the be
should simulation
noted thatmodel
since this part of the content is not the focus of this research,
a simplified
In SPEOS,mechanical
the indoorsimulation and lens design
optical environment were usedand
was simulated in the example.
necessary However,
optical com-
considering that MECHANICAL can simulate very complex mechanical behaviors
ponents such as simulation cameras, planar mirrors, and prisms were created. We built and
ZEMAX can design complex lenses and optical components, the use of a simplified
four simulation-based measurement systems based on the principle of stereo vision: Basic ex-
ample does not affect the innovation, completeness, and generalizability of
Stereo System (BSS), composed of two simulation cameras; Mirror-Based Virtual Stereo the overall
simulation method.
System (MVSS), composed of a simulation camera and four planar mirrors; Biprism-Based
Virtual Stereo System (BVSS), composed of a simulation camera and bi-prism; and Virtual
Stereo System Based on Quadrangular Pyramid Prism (QVSS), composed of a simulation
camera and a quadrangular pyramid prism. In addition, the morphology of each time step
before and after the deformation of the sample were imported into SPEOS and a random
speckle texture was set for its surface. Examples of each measurement system and its col-
 (a) (b)
Sensors 2024, 24, 4031 Figure 9. Schematic diagram of the sample tensile simulation experiment (a) and simulated lens
14 of 26
design (b).

2.
2. Build the
Build the simulation
simulation model
model
In SPEOS,
In SPEOS, thethe indoor
indoor optical
optical environment
environment was was simulated
simulated and
and necessary
necessary optical
optical com-
com-
ponents such as simulation cameras, planar mirrors, and prisms were
ponents such as simulation cameras, planar mirrors, and prisms were created. We built created. We built
four
four simulation-based
simulation-based measurement
measurement systemssystems
based based
on theon the principle
principle of vision:
of stereo stereo vision: Basic
Basic Stereo
Stereo System
System (BSS), composed
(BSS), composed of two simulation
of two simulation cameras; cameras; Mirror-Based
Mirror-Based VirtualSystem
Virtual Stereo Stereo
System (MVSS),
(MVSS), composed composed of a simulation
of a simulation camera camera
and fourand fourmirrors;
planar planar mirrors; Biprism-Based
Biprism-Based Virtual
Virtual Stereo System (BVSS), composed of a simulation camera and
Stereo System (BVSS), composed of a simulation camera and bi-prism; and Virtual bi-prism; and Virtual
Stereo
Stereo System
System Based on Based on Quadrangular
Quadrangular Pyramid Pyramid Prism (QVSS),
Prism (QVSS), composed
composed of a simulation
of a simulation camera
camera
and and a quadrangular
a quadrangular pyramidpyramid
prism. Inprism. In addition,
addition, the morphology
the morphology of each of each
time time
step step
before
before and after the deformation of the sample were imported into SPEOS
and after the deformation of the sample were imported into SPEOS and a random speckle and a random
speckle was
texture texture
set was setsurface.
for its for its surface.
ExamplesExamples
of eachofmeasurement
each measurement
system system
and its and its col-
collected
lected images
images are shown
are shown in Figure
in Figure 10. 10.

(a) (b) (c) (d)

Figure 10. Schematic
Schematicdiagram
diagramofofthe
themeasurement-systems
measurement-systemssimulation model.
simulation (a)(a)
model. BSS (b)(b)
BSS MVSS (c)
MVSS
BVSS (d) QVSS.
(c) BVSS (d) QVSS.

3. Calibration simulation experiment

All measurement systems
systems were
were directed
directed to to simultaneously
simultaneouslycapture
captureimages
imagesof ofaacali-
cal-
bration board
ibration boardwith
witha acheckerboard
checkerboard pattern,
pattern,continuously
continuously changing
changing thethe
position of the
position of cal-
the
ibration board
calibration until
board each
until system
each hadhad
system gathered
gathered10001000
images of it. of
images Toit.
reduce the uncertainty
To reduce the uncer-
tainty
in the in the calibration
calibration experiment,
experiment, for each
for each stereoscopic
stereoscopic vision
vision systemsystem calibration,
calibration, 40 40
setssets
of
of images
images were
were randomly
randomly selected.OfOfthese,
selected. these,2020sets
setswere
wereused
usedfor
forsingle-camera
single-camera and stereo
calibration of the system and the remaining 20 sets were used to to verify
verify the
the calibration
calibration
accuracy.
accuracy. The method for verifying
method for verifying accuracy involved extracting all corner points in the
10 sets of image pairs. Based on the principles of stereoscopic vision, distances between all
adjacent corner points can be determined. After the average value has been obtained and
the true value of the corner-point distance has been subtracted, the result, represented as
δd , was used as a single verification result that reflected the precision of the measurement
system. The above extraction, calibration, and verification steps were repeated 1000 times.
4. 3D-DIC Simulation Experiment
All measurement systems were directed to simultaneously capture images of a sample
with a random speckle pattern, recording the entire process of tensile deformation of the
sample in the optical environment according to the set time steps. The surface morphology
of the sample and its deformation process were then reconstructed using MULTIDIC, and
compared with the true values from step 1. In addition, considering the issue of camera
synchronization time difference in the stereo-vision systems used in reality, for BSS, this
was simulated by setting a different exposure time for the simulation cameras. As for
the remaining three measurement systems, being virtual stereo, there were no issues of
differences in camera-synchronization-time.
 phology of the sample and its deformation process were then reconstructed using MUL-
TIDIC, and compared with the true values from step 1. In addition, considering the issue
of camera synchronization time difference in the stereo-vision systems used in reality, for
BSS, this was simulated by setting a different exposure time for the simulation cameras.
Sensors 2024, 24, 4031 As for the remaining three measurement systems, being virtual stereo, there were 15 noofis-
26
sues of differences in camera-synchronization-time.
5. Image Super-resolution Reconstruction Experiment
5. Image Super-resolution Reconstruction Experiment
We collected all images gathered by the measurement systems from Step 4 (above).
We collected
Multiple networks,all images gathered
including by the measurement
the MMRSRGAN systems
proposed in this fromwere
research, Stepseparately
4 (above).
Multiple networks, including the MMRSRGAN proposed
trained and tested on the set composed of images of the sample.in this research, were separately
trained and tested on the set composed of images of the sample.
4. Results
4. Results
Figures 11 shows the results of the precision verification for each measurement sys-
Figure 11 shows the results of the precision verification for each measurement system.
tem. Specifically, Figure 11a presents the average reprojection error e from single-camera
Specifically, Figure 11a presents the average reprojection error e from single-camera cali-
calibration, while Figure 11b shows the difference δ between the measured and true val-
bration, while Figure 11b shows the difference δd between the measured and true values
ues of the corner-point distances. From the perspective of the average reprojection error,
of the corner-point distances. From the perspective of the average reprojection error, BSS
BSS yielded the best precision and stability for single-camera calibration, followed by
yielded the best precision and stability for single-camera calibration, followed by MVSS,
MVSS, then BVSS, with QVSS being the least precise. On the other hand, from the per-
then BVSS, with QVSS being the least precise. On the other hand, from the perspective of
spective of δ , the order of measurement precision is QVSS > BSS > MVSS > BVSS.
δd , the order of measurement precision is QVSS > BSS > MVSS > BVSS.

(a) (b)
Figure
Figure 11.
11. Accuracy-verification
Accuracy-verification results
results for
for four
four measurement
measurement systems.
systems. (a) e; (b)
(a) e; (b) δδd . .

As can be seen, the two evaluation methods yielded quite different different results. During
camera calibration,
calibration, to
to obtain
obtainmore
moreprecise
precisecalibration
calibrationinformation,
information,it it
is is
generally
generally required to
required
capture
to capturethethe
calibration board
calibration from
board different
from angles
different andand
angles distances andand
distances to cover the the
to cover camera’s
cam-
field of
era’s view
field as much
of view as possible.
as much As BSS
as possible. Asconsists of twoofcameras,
BSS consists it is theit system
two cameras, in which
is the system in
it is easiest to meet the above requirements, while in the remaining measurement
which it is easiest to meet the above requirements, while in the remaining measurement systems,
the virtual cameras “occupy” 1/2 or 1/4 of the field of view and also lose a corresponding
proportion of image resolution; hence, BSS has the best single-camera calibration effect,
and QVSS the worst. Furthermore, prisms can change the light path, introducing more
image distortion; this effect makes the single-camera calibration effect of MVSS, which is
based on plane mirrors, better than that of BVSS and QVSS, which are based on prisms.
However, from the perspective of δd , the measurement precision and stability of QVSS are
comparable to those of BSS. This is likely because, although QVSS is subject to the most loss
in field of view and image resolution, it has leveraged the advantage of multi-viewpoint
stereo vision in the measurement process, i.e., the four cameras can correct each other. From
the calibration results of 1000 iterations for each measurement system, the δd at the median
was selected as the final calibration result for each measurement system.
Figure 12 represents the average relative error between the result parameters obtained
through calibration by each measurement system and the actual values. It can be seen
that the precision ranking of the internal parameter calibration of the four measurement
systems is BSS, MVSS, BVSS, and QVSS, which is generally consistent with the reprojection-
error results. It is worth noting that the internal parameter-calibration error of QVSS is
significantly higher than those of the other three measurement systems, but its external
 ment system.
Figure 12 represents the average relative error between the result parameters ob-
tained through calibration by each measurement system and the actual values. It can be
seen that the precision ranking of the internal parameter calibration of the four measure-
Sensors 2024, 24, 4031 ment systems is BSS, MVSS, BVSS, and QVSS, which is generally consistent with the
16 of 26
reprojection-error results. It is worth noting that the internal parameter-calibration error
of QVSS is significantly higher than those of the other three measurement systems, but its
external
parameterparameter calibration
calibration error is error
loweristhan
lowerthat
than
ofthat
BVSSof BVSS
and isand is close
close to those
to those of BSS
of BSS and
and MVSS. This can once again be attributed to the advantages of multi-view stereo
MVSS. This can once again be attributed to the advantages of multi-view stereo vision. vision.

Sensors 2024, 24, x FOR PEER REVIEW 17 of 28

of the test sample. Figure 15b, on the other hand, is a log-log plot of the node average
reconstruction error of BSS in relation to the set camera-synchronization error. It can be
seen that when the synchronization error is within 10 time steps, the error in the re-
constructed deformation amount can still be controlled within 0.005 mm. However, when
the synchronization error exceeds 10 , there is an uncontrolled increase in the amount
of deformation error. In contrast, virtual stereo-vision systems can completely eliminate
the impact of camera-synchronization error on measurement results during the measure-
Figure 12. Bar chart of relative error between camera calibration results and true values.
Figure
ment 12. Bar chart
process. As of relative
shown byerror
the between
three dashedcameralines
calibration results
in Figure 15b,andthetrue values.
measurement accu-
racies of MVSS, BVSS, and QVSS lie between the BSS synchronization
Figure 13 shows the 3D reconstruction cloud map of the sample surface, whichand errors of 10 is
10 Figure
time 13 shows the 3Ditreconstruction cloud
themap of the sample surface, which is
obtained bysteps.
usingTherefore,
images collected is clear
by that
variouswhen camera-synchronization
measurement systems based onerror is diffi-
the 3D-DIC
obtained
principle
by using
cult to control
and then
images
or the cost of
fitting
collected
theachieving
by itvarious
reconstructed is too measurement
high, systems
a measurement
three-dimensional
based
system
scatter
on
based
points
the
on3D-
through vir-
a
DIC principle
tual stereo
surface. The and
vision then
colorcan fitting
notbars
scale the reconstructed
onlyreflect
significantly three-dimensional
reducefrom
the distance the system
the points scatter
constructionpoints through
cost but
on the surface to also
the
aprojection
surface. high
maintain The color scale making
precision,
surface. bars reflect
it anthe distance
excellent from the points on the surface to the
solution.
projection surface.
Figure 14 shows the box plot of reconstruction errors for each node of the sample. In
combination with Figure 13, it reveals that BSS has the best reconstruction effect and the
smoothest fitted surface, that BVSS has the worst effect, and that MVSS and QVSS have
similar effects.
Figure 15a displays the deformation process of the test sample reconstructed by BSS
using the 3D-DIC principle (four time steps were uniformly selected without considering
camera-synchronization error), with the color scale indicating the degree of deformation

Figure13.
Figure 13.3D-reconstruction
3D-reconstructioncloud
cloudmap
mapof
ofthe
thesample
samplesurface.
surface.
Sensors 2024, 24, 4031 17 of 26

Figure 14 shows the box plot of reconstruction errors for each node of the sample. In
Sensors 2024, 24, x FOR PEER REVIEW 18 of 28
combination with Figure 13, it reveals that BSS has the best reconstruction effect and the
smoothest fitted surface, that BVSS has the worst effect, and that MVSS and QVSS have
similar effects.

Figure 14.
Figure Sample node-reconstruction
14. Sample node-reconstruction error.
error.

Figure 15a displays the deformation process of the test sample reconstructed by BSS
using the 3D-DIC principle (four time steps were uniformly selected without considering
camera-synchronization error), with the color scale indicating the degree of deformation of the
test sample. Figure 15b, on the other hand, is a log-log plot of the node average reconstruction
error of BSS in relation to the set camera-synchronization error. It can be seen that when the
synchronization error is within 10−3 time steps, the error in the reconstructed deformation
amount can still be controlled within 0.005 mm. However, when the synchronization error
exceeds 10−3 , there is an uncontrolled increase in the amount of deformation error. In contrast,
virtual stereo-vision systems can completely eliminate the impact of camera-synchronization
error on measurement results during the measurement process. As shown by the three dashed
lines in Figure 15b, the measurement accuracies of MVSS, BVSS, and QVSS lie between the
BSS synchronization errors of 10−7 and 10−6 time steps. Therefore, it is clear that when the
camera-synchronization error is difficult to control or the cost of achieving it is too high, a
measurement system based on virtual stereo vision can not only significantly reduce the
system construction cost but also maintain high precision, making it an excellent solution.
Figure 16 shows the scatter-diagram representation of the 3D-reprojection error after
application of the MMRSRGAN model proposed in this study to images captured by QVSS.
As the 3D-reprojection error is less than 0.002 mm, which is very small relative to the overall
size of the sample, it is magnified 1000 times in the three-dimensional scatter diagram for
(a) (b)
easier observation. It can be seen that after the MMRSRGAN has been applied to QVSS, not
only is15.
Figure the density of the
Reconstruction (a)scatter points
and error increased,
evaluation (b) ofbut the deformation
sample three-dimensional reprojection
process by BSS.
error is also significantly reduced.
Figure
Figure 16 shows the
17 displays thescatter-diagram representation
Epr metrics calculated of the 3D-reprojection
after applying the MMRSRGAN error after
model to
application of thesystems.
four measurement MMRSRGAN As shown model
in theproposed
figure, thein this study to
MMRSRGAN images
model captured
performs worstbyat
QVSS. As the
improving the 3D-reprojection
accuracy of the BSS error is less
system, than
even 0.002 mm,
becoming negative ×8 scale.
whichatisa very smallOnerelative to
possible
the overall
reason sizecould
for this of thebesample, it is magnified
that during the training 1000 times the
process, in the three-dimensional
model lacks images of scatter
higher
diagram
resolutionfor easier
than thoseobservation.
collected byItBSS
canandbe seen that
cannot after
learn the
the MMRSRGAN
pattern has been
of degradation fromapplied
higher
resolutions.
to QVSS, notTherefore, its density
only is the accuracyofimprovement is not obvious
the scatter points increased,at × the×
2 and
but 4 scales, and at ×8
three-dimensional
scale, an excessive
reprojection error amount of noise has been
is also significantly magnified, resulting in a negative Epr.
reduced.
Sensors 2024, 24, 4031 18 of 26
Figure 14. Sample node-reconstruction error.

Sensors 2024, 24, x FOR PEER REVIEW 19 of 28

(a)
Sensors 2024, 24, x FOR PEER REVIEW (b) 19 of 28

Figure
Figure15.
15.Reconstruction
Reconstruction(a)
(a)and
anderror
errorevaluation
evaluation(b)
(b)ofofsample
sampledeformation
deformationprocess
processby
byBSS.
BSS.

Figure 16 shows the scatter-diagram representation of the 3D-reprojection error after

application of the MMRSRGAN model proposed in this study to images captured by
QVSS. As the 3D-reprojection error is less than 0.002 mm, which is very small relative to
the overall size of the sample, it is magnified 1000 times in the three-dimensional scatter
(a) diagram for easier observation. (b) It can be seen that after the MMRSRGAN
(c) has been applied
to QVSS, not only is the density of the scatter points increased, but the three-dimensional
Figure 16. The effect of the application of the MMRSRGAN model in QVSS. (a) ×2 scale; (b) ×4
reprojection error is also significantly reduced.
scale; (c) ×8 scale.

Figure 17 displays the E metrics calculated after applying the MMRSRGAN model
to four measurement systems. As shown in the figure, the MMRSRGAN model performs
worst at improving the accuracy of the BSS system, even becoming negative at a ×8 scale.
One possible reason for this could be that during the training process, the model lacks
images of higher resolution than those collected by BSS and cannot learn the pattern of
(a) (b)
degradation from higher resolutions. (c)
Therefore, its accuracy improvement is not obvious
atFigure
×2 and
Figure 16.×4
Thescales,
effect and
of theat ×8 scale,
application ofan excessive
the MMRSRGANamount of
model noise
in QVSS.has
(a) been
×2
16. The effect of the application of the MMRSRGAN model in QVSS. (a) ×2 scale; magnified,
scale;
(b)(b)
××4
4 scale;
resulting
scale; (c) in
×8
(c) ×8 scale. a negative
scale. E .

Figure 17 displays the E metrics calculated after applying the MMRSRGAN model
to four measurement systems. As shown in the figure, the MMRSRGAN model performs
worst at improving the accuracy of the BSS system, even becoming negative at a ×8 scale.
One possible reason for this could be that during the training process, the model lacks
images of higher resolution than those collected by BSS and cannot learn the pattern of
degradation from higher resolutions. Therefore, its accuracy improvement is not obvious
at ×2 and ×4 scales, and at ×8 scale, an excessive amount of noise has been magnified,
resulting in a negative E .

17.EEpr index
Figure17.
Figure index of
of four
four measurement
measurement systems
systems after
after the
the application
application of
of the
the MMRSRGAN
MMRSRGAN model.
model.

Note the E indices of the systems other than QVSS are all greater at the ×4 scale
than at the ×8 scale. A possible reason could be that in QVSS, the effective resolution of
Sensors 2024, 24, 4031 19 of 26

Note the Epr indices of the systems other than QVSS are all greater at the ×4 scale
than at the ×8 scale. A possible reason could be that in QVSS, the effective resolution of
each virtual camera accounts for only 1/4 of the Epr value on average, with that of BSS
being four times greater than those of QVSS and MVSS and that of BVSS being twice that
of QVSS. Consequently, during the training process, QVSS, compared to other systems, is
more likely to learn the degradation pattern of the images captured by its own system.
The above results reveal that the MMRSRGAN model has a better accuracy-improving
effect on virtual stereo-vision systems, indirectly indicating that image super-resolution
reconstruction technology is particularly beneficial for countering the resolution loss asso-
ciated with virtual stereo vision.

5. Discussion and Conclusions

The most remarkable contribution and value of this research lie in the innovative pro-
posal of a complete set of simulation-based experimental methods. These methods integrate
optical simulation, mechanical simulation, 3D-DIC algorithms, and image super-resolution
reconstruction models. This approach not only provides a novel research pathway and
means for existing researchers in the field, but also opens up the possibility for those
researchers who lack the support of costly 3D-DIC hardware facilities to participate in
related research. The remaining primary conclusions are as follows:
The experiments with 3D-DIC were simulated completely and with high quality by
combining multiple software tools. Specifically, the stretch behavior of the specimen was
simulated using MECHANICAL and its dynamic geometric deformation information
was imported into the indoor optical environment in SPEOS. ZEMAX was then used to
design camera lenses, plane mirrors, and prisms, which were imported into SPEO. Four
measurement systems based on stereo vision and virtual stereo vision were designed: BSS,
MVSS, BVSS, and QVSS. Within the optical-simulation environment, each system completed
measurement-system calibration and then recorded the stretching process of the specimen.
The acquired calibration images were then used for calibration calculation and analysis,
and the specimen’s images were used for 3D-DIC computation and analysis. In addition,
the image super-resolution reconstruction networks were applied to the specimen’s images.
The experimental results demonstrate the following points: BSS has the largest field
of view, the highest effective resolution, the highest single-camera calibration accuracy, and
if synchronization errors are not considered, it has the highest deformation-measurement
accuracy. However, when the rate of errors triggering camera synchronization increases, its
measurement accuracy drops sharply and the ability of image super-resolution reconstruction
to effect accuracy improvement is relatively poor. When the synchronization error of BSS is
within 10−3 time steps, the reconstruction error is within 0.005 mm and the accuracy of the
virtual stereo-vision system is between the BSS’s synchronization error of 10−7 and 10−6 time
steps. In addition, after image super-resolution reconstruction technology has been applied,
the reconstruction error will be reduced to within 0.002 mm. QVSS benefits from multi-
camera stereo vision, with the highest stereo calibration accuracy and good improvement
in its accuracy from the image super-resolution reconstruction, especially when the super-
resolution scale reaches ×8. However, QVSS has the smallest field of view and the lowest
effective resolution. Neither MVSS nor BVSS perform best in various simulation-evaluation
methods, but they show fewer camera-synchronization errors compared to BSS and image
super-resolution reconstruction improves their accuracy. Compared to QVSS, MVSS and BVSS
offer a larger field of view, higher effective resolution, and a less complex structure design.
Due to the limitations of camera hardware, the author is unable to directly compare the
application of 3D-DIC technology between real-world and simulation scenarios. However,
with sufficient hardware support, the simulation method proposed in this article can be
deeply compared and analyzed with actual experiments. The initial goal of future research
could be to establish certain connections between simulation scenarios and the real world,
and even more ambitious goals could involve generating digital twins in high-precision
experimental-measurement environments.
 Sensors 2024, 24, 4031 20 of 26

Author Contributions: Conceptualization, H.C.; Methodology, H.C.; Validation, H.C.; Formal

Sensors 2024, 24, x FOR PEER REVIEWanalysis, H.C. and H.L.; Investigation, H.C.; Data curation, H.C. and H.L.; Writing—original
21 ofdraft,
28
H.C.; Writing—review & editing, H.C.; Visualization, H.L.; Supervision, G.L.; Project administration,
H.C., G.L. and Z.W.; Funding acquisition, G.L. All authors have read and agreed to the published
version of the manuscript.
Funding: This research was funded by National Natural Science Foundation of China grant number
Funding: This research was funded by National Natural Science Foundation of China grant num-
51979244.
ber 51979244.
Institutional Review Board Statement: Not applicable.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Data are contained within the article.
Data Availability Statement: Data are contained within the article.
Conflicts of Interest: The authors declare no conflict of interest.
Conflicts of Interest: The authors declare no conflict of interest.
Appendix A
Appendix A
1. ANSYS MECHANICAL
1. ANSYS MECHANICAL

(a) (b)
Figure
FigureA1.
A1.Schematic
Schematicdiagram
diagramofofthree-dimensional
three-dimensionaldimensions
dimensionsofoftensile
tensilespecimen
specimen(a)
(a)and
andloading
loading
curve (b).
curve (b).

Table
TableA1.
A1.Main
Mainparameters
parametersfor
forsimulation
simulationofoftensile
tensilespecimens.
specimens.

Main
MainParameter
Parameter Value Value
Volume
Volume
4.789 × 10−5 m³
4.789 × 10−5 m3
Mass
Mass 0.13266 0.13266
kg kg
Nodes
Nodes 69,831 69,831
Elements
Elements 14,197 14,197
Moment of Inertia Ip1 1.0619 × 10−4 kg·m−4 2
Moment of Inertia Ip1 1.0619 × 10 kg·m²
Moment of Inertia Ip2 1.6147 × 10−5 kg·m2
Moment
Moment ofofInertia
InertiaIp2
Ip3 1.0619 ×1.6147 × ·10
10−4 kg m2 kg·m²
−5

Moment
Compressive YieldIp3
of Inertia Strength 1.0619
280,000,000 Pa × 10−4 kg·m²
Compressive
Tensile YieldYield Strength
Strength 280,000,000
280,000,000 Pa Pa
Tensile
Tensile Ultimate
Yield Strength
Strength 310,000,000 Pa
280,000,000 Pa
Young’s Modulus 71,000,000,000 Pa
Tensile Ultimate Strength 310,000,000 Pa
Poisson’s Ratio 0.33
Young's Modulus
Bulk Modulus 71,000,000,000
69,608,000,000 Pa Pa
Poisson's Ratio
Shear Modulus 0.33 Pa
26,692,000,000
Bulk Modulus 69,608,000,000 Pa
Shear Modulus 26,692,000,000 Pa
Sensors 2024, 24, 4031 21 of 26

2. ANSYS ZEMAX OPTICSTUDIO

Table A2. Lens-design parameter.

Surface Semi- Mech

Number Radius Thickness Material
Type Diameter Semi-Dia
0 Standard Infinity Infinity 0.000 0.000
1 Standard Infinity 25.400 0.000 0.000
2 Standard −234.599 6.325 SK4 34.036 34.036
3 Standard 32.756 2.337 24.892 34.036
4 Standard 35.103 13.741 SF1 25.146 25.146
5 Standard −138.133 5.436 SK4 23.033 25.146
6 Standard 22.672 10.490 16.257 25.146
7 Standard −46.805 3.531 SF1 16.099 16.289
8 Standard −38.717 5.258 SK4 16.210 16.289
9 Standard −55.461 28.854 16.289 16.289
10 Standard Infinity 7.595 10.790 10.790
11 Standard −168.262 5.740 SK4 14.064 15.284
12 Standard −50.142 0.533 15.284 15.284
13 Standard 651.985 8.636 SK4 16.230 18.335
14 Standard −26.373 3.708 SF1 16.662 18.335
15 Standard −106.728 19.253 18.335 18.335
16 Standard 209.596 5.258 SF1 23.476 24.220
17 Standard 61.450 0.533 24.220 24.220
18 Standard 62.276 11.227 SK4 24.583 24.883
19 Standard −87.028 1.803 24.883 24.883
20 Standard 55.115 8.052 SK4 24.996 24.996
21 Standard 855.096 39.624 24.558 24.996
22 Image Infinity 13.540 13.540

3. ANSYS SPEOS
Table A3. Main parameters for simulation cameras, light sources and optical elements.

Camera
Resolution (pixel) 5120 × 4320
Sensor size (mm) 9.6 × 8.1
Focal length (mm) 18.6
Temporal Dark Noise (e-) 7.65
Saturation Capacity (e-) 6365
Dynamic Range (dB) 75.85
Gain Range (dB) [0, 24]
Gamma correction 2.2
transmittance (%) 95
RGB sensitive see Figure A2
Luminaire 1
Measured flux (lm) 7124.6
Light output ratio luminaire (%) 56.69
Downward flux fraction (%) 47.66
Spectrum type white LED
UTE C71 -121 photometric 0.27 J + 0.30 T
Luminaire 2
Measured flux (lm) 6180.9
Light output ratio luminaire (%) 52.19
Downward flux fraction (%) 40.98
Spectrum type daylight fluorescent
UTE C71 -121 photometric 0.27 J + 0.30 T
 Light output ratio luminaire (%) 52.19
Sensors 2024, 24, 4031 Downward flux fraction (%) 40.98 22 of 26
Spectrum type daylight fluorescent
UTE C71 -121 photometric 0.27 J + 0.30 T
Table A3. Cont. Optical element 1 (mirror)
Surface type
Optical element 1 (mirror)
mirror
Reflectance (%) 90
Surface type mirror
Reflectance (%)Optical element 2 (bi-prism) 90
Volume type optic
Optical element 2 (bi-prism)
Surface type optical polished
Volume type optic
Refractive index 1.5163
Surface type optical polished
Constringence (Abbe number) 1.5163
Refractive index 63.96
Optical
Constringence (Abbe element 3 (quadrangular
number) pyramid prism)
63.96
Optical elementVolume type pyramid prism)
3 (quadrangular optic
Volume type Surface type optic optical polished
Surface type Refractive index optical polished 1.5168
Refractive index
Constringence (Abbe number) 1.5168 64.06
Constringence (Abbe number) 64.06

(a)

(b)

(c)

Figure A2. Simulated camera spectral curve. (a) Red sensitivity curve; (b) green sensitivity curve;
(c) blue sensitivity curve.
 Sensors
Sensors 2024, 24,2024,
x FOR24,PEER
x FOR PEER REVIEW
REVIEW 24 of 28
24 of 28

Sensors 2024, 24, 4031 23 of 26

Figure Figure A2. Simulated
A2. Simulated cameracamera spectral
spectral curve. curve.
(a) Red(a)sensitivity
Red sensitivity
curve; curve; (b) green
(b) green sensitivity
sensitivity curve;curve;
(c) blue(c) blue sensitivity
sensitivity curve. curve.

(a) (a) (b) (b)

Figure
Figure Figure A3. Surface
A3. Surface
A3. opticaloptical
Surface optical parameters
parameters of
of simulated
of simulated
parameters calibration
calibration
simulated plates plates
calibration (a) and(a)
plates and
andtensile
tensile
(a) specimens
specimens
tensile (b). (b).
specimens (b).

(a) (a) (b) (b)

(c) (d)
(c) (d)
Figure A4. Schematic diagram of structural of BSS, MVSS, BVSS, and QVSS. See Table A4 for struc-
Figure Figure
A4. Schematic diagram diagram
A4. Schematic of structural of BSS, MVSS,
ofBVSS;
structural BVSS,
of BSS, and QVSS. See Table A4 for struc-
tural-parameter values. (a) BSS; (b) (c) MVSS; (d) MVSS,
QVSS. BVSS, and QVSS. See Table A4 for
tural-parameter values. (a) BSS; (b) BVSS; (c) MVSS; (d) QVSS.
structural-parameter values. (a) BSS; (b) BVSS; (c) MVSS; (d) QVSS.
Table A4. Structural parameters for BSS, MVSS, BVSS, and QVSS.
Table A4. Structural
Table parameters
A4. Structural for BSS,for
parameters MVSS, BVSS, and
BSS, MVSS, QVSS.
BVSS, and QVSS.
BSS
BSS BSS
B (mm) 55
B (mm)B (mm) 55 55
Objective distance (mm) 100
Objective distance
Objective (mm)(mm)
distance 100 100
MVSS
MVSS MVSS
α (°) 45.5
α (°) α (◦ ) 45.5 45.5
β (°) 50.2
β (°) β (◦ ) 50.2 50.2
d (mm) 15.5
d (mm)d (mm) 15.5 15.5
L (mm) 15.4
 Sensors 2024, 24, x FOR PEER REVIEW 25 of 28
Sensors 2024, 24, 4031 24 of 26

L (mm) 15.4
Table A4. Cont.
𝑙 (mm) 20.5
𝑙 l1 (mm)
(mm) 20.5 35.2
l2 (mm) distance (mm)
Objective 35.2 150
Objective distance (mm) 150
BVSS
DBVSS
(mm) 10
βD(°)(mm) 10 22.5
(◦ ) 22.5
hβ(mm) 8.5
h (mm) 8.5
Objective
Objective distance (mm)
distance (mm) 160 160
QVSS
QVSS
D (mm) 10
D (mm) 10
b (mm) 25.6
b (mm) 25.6
c c(mm)
(mm) 20.2 20.2
Objective
Objective distance (mm)
distance (mm) 200 200

4.
4. MULTIDIC
MULTIDIC

(a) (b)

(c) (d)
Figure A5. DIC parameter settings. (a) BSS; (b) BVSS; (c) MVSS; (d) QVSS.
Figure A5. DIC parameter settings. (a) BSS; (b) BVSS; (c) MVSS; (d) QVSS.

References
References
1.
1. Pan,
Pan, B.B.Digital
DigitalImage
Image Correlation
Correlation forfor
Surface Deformation
Surface DeformationMeasurement:
Measurement: Historical Developments,
Historical Recent
Developments, Advances
Recent and Future
Advances and
FutureMeas.
Goals. Goals.Sci. Technol.
Meas. 2018, 29,
Sci. Technol. 082001.
2018, [CrossRef]
29, 082001. https://doi.org/10.1088/1361-6501/aac55b.
2.
2. Chouhan,
Chouhan, K.; K.; Chavda,
Chavda,J.T. J.T.AA Review
Review on on Digital
Digital Image
Image Correlation
Correlation in Experimental
in Experimental Geotechnics.
Geotechnics. Indian Geotech.
Indian Geotech. J. 639–
J. 2024, 54, 2024,
54,
656.639–656. [CrossRef]
https://doi.org/10.1007/s40098-023-00783-8.
3.
3. Yoon,
Yoon, S.;
S.; Jung,
Jung, H.-J.;
H.-J.; Knowles,
Knowles, J.C.; Lee, H.-H. Digital Image Image Correlation
Correlation in in Dental
DentalMaterials
Materialsand
andRelated
RelatedResearch:
Research:AAReview.
Review.
Dent.
Dent. Mater. 2021, 37,
Mater. 2021, 37, 758–771.
758–771. [CrossRef] [PubMed]
https://doi.org/10.1016/j.dental.2021.02.024.
4.
4. Zhao,
Zhao,Y.;Y.;Druzhinin,
Druzhinin, P.; P.;
Ivens, J.; Vandepitte,
Ivens, D.; Lomov,
J.; Vandepitte, S.V. Split-Disk
D.; Lomov, Test withTest
S.V. Split-Disk 3D Digital
with 3DImage Correlation
Digital Image Strain Measurement
Correlation Strain
for Filament Woundfor
Measurement Composites.
FilamentCompos. Wound
Struct. 2021, 263, 113686. [CrossRef]
Composites. Compos. Struct. 2021, 263, 113686.
5. https://doi.org/10.1016/j.compstruct.2021.113686.
Huang, C.; Song, Y.; Yang, L.; Fan, P.; Wang, Q.; Liu, H. Study on Fracture Behavior for Sharp V-Notched Specimens Combined
5. Huang,
with C.; Song,
3D-DIC and Y.; Yang, L.;
Caustics. Fan,
Eng. P.; Wang,
Fract. Mech. Q.; Liu,
2022, H. 108190.
260, Study on Fracture Behavior for Sharp V-Notched Specimens Combined
[CrossRef]
6. with K.;
Wei, 3D-DIC
Yuan,and Caustics.
F.; Shao, X.; Eng.
Chen, Fract. Mech.
Z.; Wu, G.;2022,
He, 260, 108190. https://doi.org/10.1016/j.engfracmech.2021.108190.
X. High-Speed Multi-Camera 3D DIC Measurement of the Deformation of
6. Wei, K.; Structure
Cassette Yuan, F.; with
Shao,Large
X.; Chen,
ShakingZ.; Wu, G.;Mech.
Table. He, X. High-Speed
Syst. Multi-Camera
Signal Process. 2022, 177,3D DIC Measurement
109273. [CrossRef] of the Deformation of
7. Cassette T.J.;
Beberniss, Structure
Ehrhardt, with Large
D.A. High-Speed Shaking
3D Digital Table. Mech.
Image Correlation Syst. Signal
Vibration Process. Recent
Measurement: 2022,Advancements
177, 109273.
and
https://doi.org/10.1016/j.ymssp.2022.109273.
Noted Limitations. Mech. Syst. Signal Process. 2017, 86, 35–48. [CrossRef]
8. Wang, R.; Li, X.; Zhang, Y. Analysis and Optimization of the Stereo-System with a Four-Mirror Adapter. J. Eur. Opt. Soc. Rapid
Publ. 2008, 3, 08033. [CrossRef]
Sensors 2024, 24, 4031 25 of 26

9. Jailin, C. Full Field Modal Measurement with a Single Standard Camera. Opt. Lasers Eng. 2018, 107, 265–272. [CrossRef]
10. Barone, S.; Neri, P.; Paoli, A.; Razionale, A.V. 3D Vibration Measurements by a Virtual-Stereo-Camera System Based on a Single
Low Frame Rate Camera. Procedia Struct. Integr. 2018, 12, 122–129. [CrossRef]
11. Chi, Y.; Yu, L.; Pan, B. Low-Cost, Portable, Robust and High-Resolution Single-Camera Stereo-DIC System and Its Application in
High-Temperature Deformation Measurements. Opt. Lasers Eng. 2018, 104, 141–148. [CrossRef]
12. Durand-Texte, T.; Simonetto, E.; Durand, S.; Melon, M.; Moulet, M.-H. Vibration Measurement Using a Pseudo-Stereo System,
Target Tracking and Vision Methods. Mech. Syst. Signal Process. 2019, 118, 30–40. [CrossRef]
13. Mouaddib, E.M.; Sagawa, R.; Echigo, T.; Yagi, Y. Stereovision with a Single Camera and Multiple Mirrors. In Proceedings of the
2005 IEEE International Conference on Robotics and Automation, Barcelona, Spain, 18–22 April 2005; IEEE: Piscataway, NJ, USA,
2005; pp. 800–805.
14. Zhang, K.; Qu, Z.; Zhong, X.; Chen, Q.; Zhang, X. Design of Binocular Stereo Vision Optical System Based on a Single Lens and a
Single Sensor. Appl. Opt. 2022, 61, 6690. [CrossRef] [PubMed]
15. Genovese, K.; Casaletto, L.; Rayas, J.A.; Flores, V.; Martinez, A. Stereo-Digital Image Correlation (DIC) Measurements with a
Single Camera Using a Biprism. Opt. Lasers Eng. 2013, 51, 278–285. [CrossRef]
16. Pankow, M.; Justusson, B.; Waas, A.M. Three-Dimensional Digital Image Correlation Technique Using Single High-Speed Camera
for Measuring Large Out-of-Plane Displacements at High Framing Rates. Appl. Opt. 2010, 49, 3418. [CrossRef] [PubMed]
17. Mohammed, M.E.; Shao, X.; He, X. Portable Device for the Local Three-Dimensional Deformation Measurement Using a Single
Camera. Sci. China Technol. Sci. 2018, 61, 51–60. [CrossRef]
18. Cui, X.; Zheng, J. Biprism-Based Monocular Stereovision System Parameter Optimization. J. Electron. Imag. 2018, 27, 033020.
[CrossRef]
19. Shao, X.; Qu, J.; Chen, W. Single-Camera Three-Dimensional Digital Image Correlation with Enhanced Accuracy Based on
Four-View Imaging. Materials 2023, 16, 2726. [CrossRef] [PubMed]
20. Bashir, S.M.A.; Wang, Y.; Khan, M.; Niu, Y. A Comprehensive Review of Deep Learning-Based Single Image Super-Resolution.
PeerJ Comput. Sci. 2021, 7, e621. [CrossRef]
21. Wang, X.; Yi, J.; Guo, J.; Song, Y.; Lyu, J.; Xu, J.; Yan, W.; Zhao, J.; Cai, Q.; Min, H. A Review of Image Super-Resolution Approaches
Based on Deep Learning and Applications in Remote Sensing. Remote Sens. 2022, 14, 5423. [CrossRef]
22. Ye, S.; Zhao, S.; Hu, Y.; Xie, C. Single-Image Super-Resolution Challenges: A Brief Review. Electronics 2023, 12, 2975. [CrossRef]
23. Angarano, S.; Salvetti, F.; Martini, M.; Chiaberge, M. Generative Adversarial Super-Resolution at the Edge with Knowledge
Distillation. Eng. Appl. Artif. Intell. 2023, 123, 106407. [CrossRef]
24. Dong, C.; Loy, C.C.; He, K.; Tang, X. Image Super-Resolution Using Deep Convolutional Networks. IEEE Trans. Pattern Anal.
Mach. Intell. 2016, 38, 295–307. [CrossRef] [PubMed]
25. Wang, X.; Yu, K.; Wu, S.; Gu, J.; Liu, Y.; Dong, C.; Qiao, Y.; Loy, C.C. ESRGAN: Enhanced Super-Resolution Generative Adversarial
Networks. In Computer Vision—ECCV 2018 Workshops; Leal-Taixé, L., Roth, S., Eds.; Lecture Notes in Computer Science; Springer
International Publishing: Cham, Switzerland, 2019; Volume 11133, pp. 63–79, ISBN 978-3-030-11020-8.
26. Li, K.; Yang, S.; Dong, R.; Wang, X.; Huang, J. Survey of Single Image Super-resolution Reconstruction. IET Image Process 2020,
14, 2273–2290. [CrossRef]
27. Ledig, C.; Theis, L.; Huszar, F.; Caballero, J.; Cunningham, A.; Acosta, A.; Aitken, A.; Tejani, A.; Totz, J.; Wang, Z.; et al. Photo-
Realistic Single Image Super-Resolution Using a Generative Adversarial Network. In Proceedings of the 2017 IEEE Conference
on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; IEEE: Piscataway, NJ, USA, 2017;
pp. 105–114.
28. Hore, A.; Ziou, D. Image Quality Metrics: PSNR vs. SSIM. In Proceedings of the 2010 20th International Conference on Pattern
Recognition, Istanbul, Turkey, 23–26 August 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 2366–2369.
29. He, X.; Zhou, R.; Liu, Z.; Yang, S.; Chen, K.; Li, L. Review of Research Progress and Development Trend of Digital Image
Correlation. Multidiscip. Model. Mater. Struct. 2024, 20, 81–114. [CrossRef]
30. Pan, B.; Yu, L.; Zhang, Q. Review of Single-Camera Stereo-Digital Image Correlation Techniques for Full-Field 3D Shape and
Deformation Measurement. Sci. China Technol. Sci. 2018, 61, 2–20. [CrossRef]
31. Yang, R.; Li, Y.; Zeng, D.; Guo, P. Deep DIC: Deep Learning-Based Digital Image Correlation for End-to-End Displacement and
Strain Measurement. J. Mater. Process. Technol. 2022, 302, 117474. [CrossRef]
32. Yuan, Y.; Huang, J.; Peng, X.; Xiong, C.; Fang, J.; Yuan, F. Accurate Displacement Measurement via a Self-Adaptive Digital Image
Correlation Method Based on a Weighted ZNSSD Criterion. Opt. Lasers Eng. 2014, 52, 75–85. [CrossRef]
33. Gu, G. A Comparative Study of Random Speckle Pattern Simulation Models in Digital Image Correlation. Optik 2015,
126, 3713–3716. [CrossRef]
34. Lavatelli, A.; Balcaen, R.; Zappa, E.; Debruyne, D. Closed-Loop Optimization of DIC Speckle Patterns Based on Simulated
Experiments. IEEE Trans. Instrum. Meas. 2019, 68, 4376–4386. [CrossRef]
35. Estrada, J.B.; Franck, C. Intuitive Interface for the Quantitative Evaluation of Speckle Patterns for Use in Digital Image and
Volume Correlation Techniques. J. Appl. Mech. 2015, 82, 095001. [CrossRef]
36. Wang, G.; Zhang, L.; Yao, X. StrainNet-3D: Real-Time and Robust 3-Dimensional Speckle Image Correlation Using Deep Learning.
Opt. Lasers Eng. 2022, 158, 107184. [CrossRef]
Sensors 2024, 24, 4031 26 of 26

37. Song, J.; Liu, Z.; Luo, C. Studies on the Influence of Speckle Density on the Accuracy of Digital Image Correlation Method Based
on Numerical Simulation. J. Phys. Conf. Ser. 2021, 1971, 012072. [CrossRef]
38. Thai, T.Q.; Ruesch, J.; Gradl, P.R.; Truscott, T.T.; Berke, R.B. Speckle Pattern Inversion in High Temperature DIC Measurement.
Exp. Tech. 2022, 46, 239–247. [CrossRef]
39. Crammond, G.; Boyd, S.W.; Dulieu-Barton, J.M. Speckle Pattern Quality Assessment for Digital Image Correlation. Opt. Lasers
Eng. 2013, 51, 1368–1378. [CrossRef]
40. Dai, M.; Wei, K.; Gao, B.; Zhou, B.; Shao, X. Experimental and Simulation Investigation of Stereo-DIC via a Deep Learning
Algorithm Based on Initial Speckle Positioning Technology. Appl. Opt. 2024, 63, 1895. [CrossRef] [PubMed]
41. Ansys Speos | Optical System Design Software. Available online: https://www.ansys.com/products/optics/ansys-speos
(accessed on 13 April 2024).
42. ZEMAX Tutorials & Applications—Knowledgebase. Available online: https://support.zemax.com/hc/en-us/categories/150000
0770122 (accessed on 5 April 2024).
43. Ansys Zemax OpticStudio | Optical Design and Analysis Software. Available online: https://www.ansys.com/products/optics/
ansys-zemax-opticstudio (accessed on 13 April 2024).
44. Ansys Mechanical | Structural FEA Analysis Software. Available online: https://www.ansys.com/products/structures/ansys-
mechanical (accessed on 13 April 2024).
45. Solav, D.; Moerman, K.M.; Jaeger, A.M.; Genovese, K.; Herr, H.M. MultiDIC: An Open-Source Toolbox for Multi-View 3D Digital
Image Correlation. IEEE Access 2018, 6, 30520–30535. [CrossRef]
46. MultiDIC/MultiDIC: Matlab 3D Digital Image Correlation Toolbox. Available online: https://github.com/MultiDIC/MultiDIC?
tab=readme-ov-file (accessed on 13 April 2024).
47. Dong, Y.L.; Pan, B. A Review of Speckle Pattern Fabrication and Assessment for Digital Image Correlation. Exp. Mech. 2017,
57, 1161–1181. [CrossRef]
48. Su, Y.; Zhang, Q.; Xu, X.; Gao, Z. Quality Assessment of Speckle Patterns for DIC by Consideration of Both Systematic Errors and
Random Errors. Opt. Lasers Eng. 2016, 86, 132–142. [CrossRef]
49. Orteu, J.-J. 3-D Computer Vision in Experimental Mechanics. Opt. Lasers Eng. 2009, 47, 282–291. [CrossRef]
50. Schreier, H.; Orteu, J.-J.; Sutton, M.A. Image Correlation for Shape, Motion and Deformation Measurements: Basic Concepts, Theory and
Applications; Springer: Boston, MA, USA, 2009; ISBN 978-0-387-78746-6.
51. Aggarwal, R.; Vohra, A.; Namboodiri, A.M. Panoramic Stereo Videos with a Single Camera. In Proceedings of the 2016 IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; IEEE: Piscataway, NJ,
USA, 2016; pp. 3755–3763.
52. Liu, X.; Li, A. Multiview Three-Dimensional Imaging Using a Risley-Prism-Based Spatially Adaptive Virtual Camera Field. Appl.
Opt. 2022, 61, 3619–3629. [CrossRef] [PubMed]
53. Wu, L.F.; Zhu, J.G.; Xie, H.M.; Zhang, Q. An Accurate Method for Shape Retrieval and Displacement Measurement Using
Bi-Prism-Based Single Lens 3D Digital Image Correlation. Exp. Mech. 2016, 56, 1611–1624. [CrossRef]
54. Lim, K.B.; Kee, W.L.; Wang, D. Virtual Camera Calibration and Stereo Correspondence of Single-Lens Bi-Prism Stereovision
System Using Geometrical Approach. Signal Process. Image Commun. 2013, 28, 1059–1071. [CrossRef]
55. Xiao, Y.; Lim, K.B. A Prism-Based Single-Lens Stereovision System: From Trinocular to Multi-Ocular. Image Vis. Comput. 2007,
25, 1725–1736. [CrossRef]
56. Chen, H.; Liu, G.; Wang, Z. A Stereovision-Based Efficient Measurement Approach for Surface Flatness of Concrete Members.
Structures 2024, 63, 106374. [CrossRef]
57. Wang, C.; Li, Z.; Shi, J. Lightweight Image Super-Resolution with Adaptive Weighted Learning Network. arXiv 2019,
arXiv:1904.02358.
58. Rakotonirina, N.C.; Rasoanaivo, A. ESRGAN+: Further Improving Enhanced Super-Resolution Generative Adversarial Network.
In Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP),
Barcelona, Spain, 4–8 May 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 3637–3641.
59. Kim, J.; Lee, J.K.; Lee, K.M. Accurate Image Super-Resolution Using Very Deep Convolutional Networks. In Proceedings of
the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; IEEE:
Piscataway, NJ, USA, 2016; pp. 1646–1654.
60. Xing, H.Z.; Zhang, Q.B.; Braithwaite, C.H.; Pan, B.; Zhao, J. High-Speed Photography and Digital Optical Measurement
Techniques for Geomaterials: Fundamentals and Applications. Rock Mech. Rock Eng. 2017, 50, 1611–1659. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.