lOMoARcPSD|36625914

CV UNIT-4 - unit 4

Computer Vision (Anna University)

Scan to open on Studocu

Studocu is not sponsored or endorsed by any college or university

Downloaded by VAISHNAVI S ([email protected])
 lOMoARcPSD|36625914

UNIT IV
3D RECONSTRUCTION
Shape from X - Active range finding - Surface representations -
Point-based representationsVolumetric representations - Model-based
reconstruction - Recovering texture maps and albedosos.

1. Shape from X:

"Shape from X" refers to a category of computer vision and computer graphics
techniques that aim to recover the three-dimensional (3D) shape or structure of objects
or scenes from different types of information or cues, represented by the variable "X".
The "X" can stand for various sources or modalities that provide information about the
scene. Some common examples include:

​ Shape from Shading (SfS): This technique involves recovering 3D shape

information from variations in brightness and shading in 2D images. It assumes
that the shading patterns in an image are influenced by the underlying 3D
geometry.
​
​ Shape from Stereo (SfS): This method utilizes the disparity or parallax
information between two or more images of a scene taken from different

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

viewpoints. By triangulating corresponding points, the 3D structure of the scene

can be reconstructed.
​
​ Shape from Motion (SfM): SfM aims to recover the 3D structure of a scene by
analyzing the motion of objects or the camera itself. This is often achieved by
tracking features across multiple frames of a video sequence.
​
​ Shape from Texture (SfT): SfT relies on the variations in texture patterns across
surfaces to infer their 3D structure. The assumption is that different surface
orientations result in distinct texture variations.
​
​ Shape from Focus (SfF): In SfF, the depth information is inferred from the
variation in image sharpness or focus. By analyzing the focus information at
different depths, the 3D shape can be estimated.
​ Shape from Defocus (SfD): Similar to SfF, SfD leverages the effects of defocusing
in images to estimate depth information. Objects at different distances from the
camera will exhibit different degrees of blur.
​
​ Shape from Light (SfL): This technique involves using information about the
lighting conditions in a scene to infer 3D shape. The interaction between light and
surfaces provides cues about the geometry.

These approaches demonstrate the diversity of methods used in computer vision to

recover 3D shape information from different types of visual cues. The choice of the
specific "X" (shading, stereo, motion, etc.) depends on the available data and the
characteristics of the scene being reconstructed.

2. Active range finding:

Active range finding is a technique used in computer vision and remote sensing to
determine the distance to objects in a scene actively. Unlike passive methods that rely
on existing ambient illumination, active range finding involves emitting a signal or probe
towards the target and measuring the time it takes for the signal to return. This process
is often based on the principles of time-of-flight or phase-shift measurement. The goal
is to obtain accurate depth or distance information about the surfaces in the scene.

Here are a few common methods of active range finding:

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

​ Laser Range Finding: This method involves emitting laser beams towards the
target and measuring the time it takes for the laser pulses to travel to the object
and back. By knowing the speed of light, the distance to the object can be
calculated.
​
​ Structured Light: In structured light range finding, a known light pattern, often a
grid or a set of stripes, is projected onto the scene. Cameras capture the
deformed pattern on surfaces, and the distortion helps calculate depth
information based on the known geometry of the projected pattern.
​
​ Time-of-Flight (ToF) Cameras: ToF cameras emit modulated light signals (often
infrared) and measure the time it takes for the light to travel to the object and
return. The phase shift of the modulated signal is used to determine the distance
to the object.
​
​ Ultrasound Range Finding: Ultrasound waves are emitted, and the time it takes
for the waves to bounce back to a sensor is measured. This method is commonly
used in environments where optical methods may be less effective, such as in
low-light conditions.

Active range finding has various applications, including robotics, 3D scanning,

autonomous vehicles, augmented reality, and industrial inspection. The ability to actively
measure distances is valuable in scenarios where ambient lighting conditions may vary
or when accurate depth information is essential for understanding the environment.

3. Surface representations:

Surface representations in computer vision refer to the ways in which the geometry or
shape of surfaces in a three-dimensional (3D) scene is represented. These
representations are crucial for tasks such as 3D reconstruction, computer graphics, and
virtual reality. Different methods exist for representing surfaces, and the choice often
depends on the application's requirements and the characteristics of the data. Here are
some common surface representations:

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

​ Polygonal Meshes:
● Description: Meshes are composed of vertices, edges, and faces that
define the surface geometry. Triangular and quadrilateral meshes are
most common.
● Application: Widely used in computer graphics, gaming, and 3D modeling.
​ Point Clouds:
● Description: A set of 3D points in space, each representing a sample on
the surface of an object.
● Application: Generated by 3D scanners, LiDAR, or depth sensors; used in
applications like autonomous vehicles, robotics, and environmental
mapping.
​ Implicit Surfaces:
● Description: Represent surfaces as the zero level set of a scalar function.
Points inside the surface have negative values, points outside have
positive values, and points on the surface have values close to zero.
● Application: Used in physics-based simulations, medical imaging, and
shape modeling.
​ NURBS (Non-Uniform Rational B-Splines):
● Description: Mathematical representations using control points and basis
functions to define smooth surfaces.
● Application: Commonly used in computer-aided design (CAD), automotive
design, and industrial design.
​ Voxel Grids:
● Description: 3D grids where each voxel (volumetric pixel) represents a
small volume in space, and the surface is defined by the boundary
between occupied and unoccupied voxels.
● Application: Used in medical imaging, volumetric data analysis, and
computational fluid dynamics.
​ Level Set Methods:
● Description: Represent surfaces as the zero level set of a
higher-dimensional function. The evolution of this function over time
captures the motion of the surface.
● Application: Used in image segmentation, shape optimization, and fluid
dynamics simulations.
​ Octrees:
● Description: Hierarchical tree structures that recursively divide space into
octants. Each leaf node contains information about the geometry within
that region.

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

● Application: Used in real-time rendering, collision detection, and efficient

storage of 3D data.

The choice of surface representation depends on factors such as the nature of the
scene, the desired level of detail, computational efficiency, and the specific
requirements of the application.

4. Point-based representations:
Point-based representations in computer vision and computer graphics refer to
methods that represent surfaces or objects using a set of individual points in
three-dimensional (3D) space. Instead of explicitly defining the connectivity between
points as in polygonal meshes, point-based representations focus on the spatial
distribution of points to describe the surface geometry. Here are some common
point-based representations:

​ Point Clouds:
● Description: A collection of 3D points in space, each representing a sample
on the surface of an object or a scene.
● Application: Point clouds are generated by 3D scanners, LiDAR, depth
sensors, or photogrammetry. They find applications in robotics,
autonomous vehicles, environmental mapping, and 3D modeling.
​ Dense Point Clouds:
● Description: Similar to point clouds but with a high density of points,
providing more detailed surface information.

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

● Application: Used in applications requiring detailed 3D reconstructions,

such as cultural heritage preservation, archaeological studies, and
industrial inspections.
​ Sparse Point Sets:
● Description: Representations where only a subset of points is used to
describe the surface, resulting in a sparser dataset compared to a dense
point cloud.
● Application: Sparse point sets are useful in scenarios where
computational efficiency is crucial, such as real-time applications and
large-scale environments.
​ Point Splats:
● Description: Represent each point as a disc or a splat in 3D space. The
size and orientation of the splats can convey additional information.
● Application: Commonly used in point-based rendering and visualization to
represent dense point clouds efficiently.
​ Point Features:
● Description: Represent surfaces using distinctive points or key points,
each associated with local features such as normals, color, or texture
information.
● Application: Widely used in feature-based registration, object recognition,
and 3D reconstruction.
​ Point Set Surfaces:
● Description: Represent surfaces as a set of unorganized points without
connectivity information. Surface properties can be interpolated from
neighboring points.
● Application: Used in surface reconstruction from point clouds and
point-based rendering.
​ Radial Basis Function (RBF) Representations:
● Description: Use radial basis functions to interpolate surface properties
between points. These functions define a smooth surface that passes
through the given points.
● Application: Commonly used in shape modeling, surface reconstruction,
and computer-aided design.

Point-based representations are particularly useful when dealing with unstructured or

irregularly sampled data. They provide flexibility in representing surfaces with varying
levels of detail and are well-suited for capturing complex and intricate structures.

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

5. Volumetric representations:

Volumetric representations in computer vision and computer graphics are methods

used to describe and model three-dimensional (3D) space in a volumetric manner.
Unlike surface representations, which focus on defining the surface geometry explicitly,
volumetric representations capture information about the entire volume, including the
interior of objects. Here are some common volumetric representations:

​ Voxel Grids:
● Description: A regular grid of small volume elements, called voxels, where
each voxel represents a small unit of 3D space.
● Application: Used in medical imaging, computer-aided design (CAD),
computational fluid dynamics, and robotics. Voxel grids are effective for
representing both the exterior and interior of objects.
​ Octrees:
● Description: A hierarchical data structure that recursively divides 3D space
into octants. Each leaf node in the octree contains information about the
occupied or unoccupied status of the corresponding volume.
● Application: Octrees are employed for efficient storage and representation
of volumetric data, particularly in real-time rendering, collision detection,
and adaptive resolution.
​ Signed Distance Fields (SDF):
● Description: Represent the distance from each point in space to the
nearest surface of an object, with positive values inside the object and
negative values outside.
● Application: Used in shape modeling, surface reconstruction, and
physics-based simulations. SDFs provide a compact representation of
geometry and are often used in conjunction with implicit surfaces.
​ 3D Texture Maps:

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

● Description: Extend the concept of 2D texture mapping to 3D space,

associating color or other properties with voxels in a volumetric grid.
● Application: Employed in computer graphics, simulations, and
visualization to represent complex volumetric details such as smoke,
clouds, or other phenomena.
​ Point Clouds with Occupancy Information:
● Description: Combine the idea of point clouds with additional information
about the occupancy of each point in space.
● Application: Useful in scenarios where capturing both the surface and
interior details of objects is necessary, such as in robotics and 3D
reconstruction.
​ Tensor Fields:
● Description: Represent the local structure of a volumetric region using
tensors. Tensor fields capture directional information, making them
suitable for anisotropic materials and shapes.
● Application: Commonly used in materials science, biomechanics, and
simulations where capturing anisotropic properties is important.
​ Shell Maps:
● Description: Represent the surfaces of objects as a collection of shells or
layers, each encapsulating the object's geometry.
● Application: Used in computer graphics and simulation to efficiently
represent complex objects and enable dynamic level-of-detail rendering.

Volumetric representations are valuable in various applications where a comprehensive

understanding of the 3D space is required, and they offer flexibility in capturing both
surface and interior details of objects. The choice of representation often depends on
the specific requirements of the task at hand and the characteristics of the data being
modeled.

6. Model-based reconstruction:
Model-based reconstruction in computer vision refers to a category of techniques that
involve creating a 3D model of a scene or object based on predefined models or
templates. These methods leverage prior knowledge about the geometry, appearance,
or structure of the objects being reconstructed. Model-based reconstruction is often
used in scenarios where a known model can be fitted to the observed data, providing a
structured and systematic approach to understanding the scene. Here are some key
aspects and applications of model-based reconstruction:

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

​ Prior Model Representation:

● Description: In model-based reconstruction, a mathematical
representation or a geometric model of the object or scene is assumed or
known in advance.
● Application: Commonly used in computer-aided design (CAD), medical
imaging, and industrial inspection, where known shapes or structures can
be explicitly represented.
​ Model Fitting:
● Description: The reconstruction process involves adjusting the parameters
of the model to best fit the observed data, typically obtained from images
or sensor measurements.
● Application: Used in applications such as object recognition, pose
estimation, and 3D reconstruction by aligning the model with the observed
features.
​ Geometric Constraints:
● Description: Constraints on the geometry of the scene, such as the
relationships between different components or the expected shape
characteristics, are incorporated into the reconstruction process.
● Application: Applied in robotics, augmented reality, and computer vision
tasks where geometric relationships play a crucial role.
​ Deformable Models:
● Description: Models that can adapt and deform to fit the observed data,
allowing for more flexible and realistic representations.
● Application: Commonly used in medical imaging for organ segmentation
and shape analysis, as well as in computer graphics for character
animation.
​ Stereo Vision with Model Constraints:
● Description: Stereo vision techniques that incorporate known models to
improve depth estimation and 3D reconstruction.

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

● Application: Used in stereo matching algorithms and 3D reconstruction

pipelines to enhance accuracy by considering geometric priors.
​ Parametric Surfaces:
● Description: Representing surfaces using parametric equations or
functions, allowing for efficient adjustment of parameters during the
reconstruction process.
● Application: Applied in computer graphics, virtual reality, and industrial
design where surfaces can be described mathematically.
​ Multi-View Reconstruction with Known Models:
● Description: Leveraging multiple views or images of a scene to reconstruct
a 3D model while incorporating information from known models.
● Application: Common in photogrammetry and structure-from-motion
applications where multiple perspectives contribute to accurate 3D
reconstruction.

Model-based reconstruction is valuable when there is prior knowledge about the objects
or scenes being reconstructed, as it allows for more efficient and accurate
reconstruction compared to purely data-driven approaches. This approach is
particularly useful in fields where a well-defined understanding of the underlying
geometry is available.

7. Recovering texture maps and albedos:

Recovering texture maps and albedos in computer vision and computer graphics
involves estimating the surface appearance, color, and reflectance properties of objects
in a scene. These processes are integral to creating realistic and detailed 3D models for
applications like virtual reality, computer games, and simulations. Here's a brief
overview of these concepts:

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

​ Texture Maps:
● Description: Texture mapping involves applying a 2D image, known as a
texture map, onto a 3D model's surface to simulate surface details,
patterns, or color variations.
● Recovery Process: Texture maps can be recovered through various
methods, including image-based techniques, photogrammetry, or using
specialized 3D scanners. These methods capture color information
associated with the surface geometry.
● Application: Used in computer graphics, gaming, and virtual reality to
enhance the visual appearance of 3D models by adding realistic surface
details.
​ Albedo:
● Description: Albedo represents the intrinsic color or reflectance of a
surface, independent of lighting conditions. It is a measure of how much
light a surface reflects.
● Recovery Process: Albedo can be estimated by decoupling surface
reflectance from lighting effects. Photometric stereo, shape-from-shading,
or using multi-view images are common methods to recover albedo
information.
● Application: Albedo information is crucial in computer vision applications,
such as material recognition, object tracking, and realistic rendering in
computer graphics.

Recovering Texture Maps and Albedos often involves the following techniques:

​ Photometric Stereo:

Downloaded by VAISHNAVI S ([email protected])

 lOMoARcPSD|36625914

● Description: A technique that uses multiple images of an object

illuminated from different directions to recover surface normals and,
subsequently, albedo information.
● Application: Used in scenarios where detailed surface properties are
needed, such as facial recognition, material analysis, and industrial
inspection.
​ Shape-from-Shading:
● Description: Inferring the shape of a surface based on variations in
brightness or shading in images. By decoupling shading from geometry,
albedo information can be estimated.
● Application: Applied in computer vision for shape recovery, as well as in
computer graphics to enhance the realism of rendered images.
​ Multi-View Stereo (MVS):
● Description: In the context of 3D reconstruction, MVS involves capturing
images of a scene from multiple viewpoints and recovering both geometry
and texture information.
● Application: Commonly used in 3D modeling, virtual reality, and cultural
heritage preservation to create detailed and textured 3D models.
​ Reflectance Transformation Imaging (RTI):
● Description: A technique that captures a series of images with controlled
lighting conditions to reveal surface details, including albedo variations.
● Application: Widely used in cultural heritage preservation and art
restoration for capturing fine details on surfaces.

Recovering texture maps and albedos is crucial for creating visually appealing and
realistic 3D models. These techniques bridge the gap between the geometry of the
objects and their appearance, contributing to the overall fidelity of virtual or augmented
environments.

Downloaded by VAISHNAVI S ([email protected])