IPCV Unit – 04

➢ Facet Model recognition

Facet model recognition in image processing focuses on identifying and interpreting geometric
shapes, edges, and structures within images, particularly line drawings or projections of three-
dimensional objects. This process is essential for various applications like computer vision,
robotics, and graphic rendering, where accurate recognition and understanding of object
boundaries and shapes are crucial.

Its uses ranges from edge detection, background normalization, shape and surface topography, to
image segmentation procedures involving detection of corners, curves, valleys and ridges. The
facet model principle is based on minimization of the error between the image thought of as a
piecewise continuous gray level intensity surface and the observed data form the physical scene.

The general forms of facet model include piecewise constant, piecewise linear, piecewise
quadratic, and piecewise cubic. In the constant model, each region in the image has a constant
gray level. In the sloped models, each region has a gray level surface that is a sloped plane.

• Labeling lines
In the context of facet model recognition, labeling lines involves assigning identifiers to
different lines within a line drawing. This step helps distinguish between various edges and
contours, enabling the system to understand the fundamental structure of the shapes
depicted.
Accurate line labeling is crucial as it lays the groundwork for subsequent steps in recognizing
and interpreting the geometric facets within an image.
Labeling lines creates the foundation for subsequent steps in the recognition process, enabling
the system to differentiate between various edges and boundaries. It helps in identifying the
structural components of objects, such as edges, corners, and surfaces. Proper labeling is
essential for tasks like shape recognition, classification, and 3D reconstruction.

Labeling a line involves number of different symbols for the annotation of edges in an image.
Some of the symbols are described below:
+ : Indicates a convex edge where two surfaces meet forming an outward curve.
- : Indicates a concave edge where two surfaces meet forming an inward curve.
> : Represents an edge where the left side is closer to the viewer than the right side.
< : Represents an edge where the right side is closer to the viewer than the left side.

Process of Labeling Lines:

a. Line Detection:
- Edge Detection: Detect edges within the image using algorithms like the Canny edge
detector or Sobel filter.
 - Line Segmentation: Segment the detected edges into individual line segments.
b. Determination of Edge Type and Orientation:
- Convex vs. Concave: Analyze the geometry to determine if the edge is convex (+) or
concave (-).
- Depth Perception: Determine the orientation relative to the viewer to assign > or <.

c. Contextual Interpretation:
- Connecting Surfaces: Understand how the lines connect different surfaces of the object.
- 3D Structure: Infer the three-dimensional structure from the two-dimensional
representation.
Some common line labeling techniques include:
1. Connected Component Labeling: Connected Component Labeling is a technique used to
label connected regions in an image. It can be applied to label lines that are connected.
2. Line segment clustering: Line segment clustering is a technique used to group similar line
segments together. It can help in labeling lines that are fragmented or broken.
3. Line merging and splitting: Line merging and splitting techniques are used to merge or split
lines based on certain criteria. These techniques can help in labeling lines that are
overlapping or intersecting.

• Understanding line drawings

Line drawings are a simplified form of visual representation that use lines to denote the edges
and boundaries of objects. They are often used in technical fields, such as engineering,
architecture, and computer graphics, to convey the structure and layout of objects without the
use of shading or color. In the context of facet model recognition, line drawings are critical as
they serve as the primary input from which geometric shapes and structures are identified and
interpreted.

Once lines are labeled, their meaning and structure need to be interpreted. Some common line
drawing interpretation techniques include:
1. Line orientation and length estimation: Line orientation and length estimation techniques
are used to estimate the orientation and length of lines in an image. This information can
be used to interpret the meaning of the lines.
2. Line junction detection: Line junction detection techniques are used to detect junctions or
intersections between lines. This information can be used to interpret the structure of the
lines and identify objects.
3. Line continuity analysis: Line continuity analysis techniques are used to analyze the
continuity of lines. This can help in identifying breaks or gaps in lines and interpreting the
structure of the drawing.
Line drawings were a natural early target for computer vision for the following reasons:
 1. They are related closely to surface features of polyhedral scenes.
2. They may be represented exactly, completely eliminating the noise and incomplete visual
processing that may have affected the line drawing extraction.
3. They present an interpretation problem that is significant but seems approachable.
Challenges in Interpreting Line Drawings
1. Ambiguity: Line drawings can sometimes be ambiguous, with multiple possible
interpretations of the shapes.
2. Occlusion: Hidden lines and occluded parts of objects can complicate recognition.
3. Complexity: Highly detailed drawings with many lines can be difficult to segment and label
accurately.

• Classification of shapes by labeling of edges

Facet model recognition is a powerful method for recognizing and classifying three-
dimensional (3D) shapes by analyzing their geometric features, specifically their faces, edges,
and vertices. This technique relies on accurately labeling the edges of the shapes to understand
their structural properties. Each edge in a 3D shape can be labeled as a boundary edge, an
interior edge, or a crease edge.
1. Boundary Edge: A boundary edge is part of the outer boundary of the shape and is not
shared with any other face or edge. In a cube, the edges on the outermost boundary are
boundary edges.
2. Interior Edge: An interior edge is shared by two faces within the shape. In a rectangular
prism, the edges connecting the top and bottom faces to the side faces are interior edges.
3. Crease Edge: A crease edge is shared by two or more faces and forms a prominent feature
of the shape, such as a corner or ridge. In a pyramid, the edges connecting the base to the
apex are crease edges.
By labeling the edges in this way, the facet model recognition algorithm can identify the
different types of edges in a shape and use this information to classify the shape.
For example, if a shape has only boundary edges and interior edges, it may be classified as a
prism. If it has crease edges, it may be classified as a polyhedron with a more complex shape,
such as a pyramid or a truncated cone.
Overall, the classification of shapes by labeling of edges in facet model recognition is a useful
tool for analyzing and categorizing three-dimensional shapes based on their geometric
features.
• Recognition of shapes
Recognition of shapes is a critical component of facet model recognition, a process that
involves identifying and classifying three-dimensional (3D) shapes by analyzing their geometric
features such as faces, edges, and vertices. This process is essential in various fields like
computer vision, robotics, and CAD (Computer-Aided Design) systems.

Steps in Shape Recognition

1. Preprocessing:
- Edge Detection: Identify all the edges in the image or model using edge detection
algorithms like Canny or Sobel.
- Line Segmentation: Segment the detected edges into individual line segments.
- Noise Reduction: Filter out noise and irrelevant details to focus on significant geometric
features.

2. Edge Labeling:
- Assign Labels: Label each edge as a boundary edge, interior edge, or crease edge.
- Boundary Edge: Part of the outer boundary of the shape.
- Interior Edge: Shared by two faces within the shape.
- Crease Edge: Shared by two or more faces, forming features like corners or ridges.

3. Feature Extraction:
- Vertices and Junctions: Identify and label vertices where edges meet.
- Face Detection: Detect and classify the faces formed by connected edges.
- Angle and Curvature Analysis: Analyze angles and curvatures to understand the
geometric relationships between edges and faces.

4. Shape Classification:
- Pattern Matching: Match the extracted features and labeled edges against a database of
known shapes.
- Geometric Properties: Use properties like symmetry, parallelism, and perpendicularity to
classify shapes.
- Template Matching: Compare the detected shape with predefined templates or
prototypes of shapes.

5. Verification and Refinement:

- Consistency Check: Verify the consistency of the recognized shape with the input data.
- Refinement Algorithms: Apply refinement techniques to correct any misclassifications or
inaccuracies.

Techniques and Algorithms

1. Template Matching:
Uses predefined templates of shapes to compare and recognize shapes in the input data.
Effective for recognizing standard geometric shapes like rectangles, circles, and triangles.
 2. Geometric Invariants:
Uses properties that remain unchanged under transformations like rotation, scaling, and
translation. It is helpful in recognizing shapes regardless of their orientation and size.

3. Machine Learning and Neural Networks:

Train models using labeled datasets to recognize and classify shapes based on learned
patterns. Deep learning techniques, such as convolutional neural networks (CNNs), are
particularly effective for complex shape recognition tasks.

4. Graph-Based Methods:
Represent shapes as graphs where vertices correspond to features (e.g., corners) and edges
correspond to connections between features. Graph matching algorithms can then be used to
recognize and classify shapes.

• Consisting labeling problem

The consistent labeling problem is a critical challenge in facet model recognition. It involves
ensuring that the labels assigned to the edges, vertices, and faces of a three-dimensional shape
are coherent and logically consistent across the entire model. This consistency is crucial for
accurate shape recognition and classification.

Consistent Labeling: In the context of facet model recognition, consistent labeling refers to the
assignment of labels to the geometric features (edges, vertices, faces) such that the
relationships and properties of these features align correctly with the actual structure of the
shape.

The Problem: The consistent labeling problem arises when there are conflicts or ambiguities in
the assigned labels, leading to an incorrect or ambiguous interpretation of the shape. This can
happen due to errors in edge detection, noise in the data, or complex geometric
configurations.

Steps to Address the Consistent Labeling Problem

1. Initial Labeling:
Assign initial labels to edges, vertices, and faces based on their detected properties.
Use predefined criteria to label edges as boundary edges, interior edges, or crease edges.

2. Geometric Analysis:
Local Consistency: Check the consistency of labels within local regions of the shape.
Global Consistency: Ensure that the local consistency holds across the entire shape.
 3. Consistency Constraints:
Define constraints that the labels must satisfy to be consistent. These constraints can be
based on geometric properties, such as angles, edge connectivity, and face adjacency.

4. Constraint Propagation:
Use constraint propagation algorithms to enforce the consistency constraints. This involves
iteratively adjusting labels to resolve conflicts and ensure coherence.
Example Algorithms: Backtracking, arc consistency, and path consistency algorithms can be
used for this purpose.

5. Error Correction:
Identify and correct labeling errors. This can involve re-evaluating the properties of edges,
vertices, and faces. Use additional information or contextual clues to refine the labels.

6. Verification:
Verify the final labeling by checking against known properties and characteristics of the
shape. Ensure that the final labels accurately represent the geometric structure and
relationships of the shape.

Techniques to Solve the Consistent Labeling Problem

1. Constraint Satisfaction Problems (CSP): Treat the labeling problem as a CSP where variables
(edges, vertices, faces) need to satisfy certain constraints (geometric relationships).
2. Backtracking Algorithms: Use backtracking to explore possible label assignments and
backtrack when a conflict is detected. It is effective for smaller or less complex shapes
where the search space is manageable.
3. Arc Consistency: Ensure that for every pair of connected features, the labels assigned to
them are consistent. Propagate constraints through the network of features to maintain
consistency.
4. Graph-Based Methods: Represent the shape as a graph with nodes (features) and edges
(connections). Use graph algorithms to check and enforce consistency of labels.

• Backtracking Algorithm
The backtracking algorithm is a systematic method for solving constraint satisfaction problems,
including the consistent labeling problem in facet model recognition. In this context, the
backtracking algorithm iteratively explores possible label assignments for the edges, vertices,
and faces of a three-dimensional shape, backtracking when conflicts are encountered to find a
consistent labeling solution. Here's how the backtracking algorithm can be applied to solve the
consistent labeling problem:

1. Initialization:
 Start with an initial labeling assignment for the edges, vertices, and faces based on detected
properties. Define constraints and consistency checks that the labels must satisfy.

2. Variable Selection:
Choose a variable to assign a label. Variables can be edges, vertices, or faces. Select the
variable based on a heuristic, such as choosing the variable with the fewest remaining
values or the most constrained variable.

3. Value Assignment:

Choose a label (value) to assign to the selected variable. Select the label based on the
constraints and consistency checks defined for the problem.

4. Consistency Check:
Perform consistency checks to ensure that the label assignment maintains coherence and
consistency with the constraints. Check for conflicts or violations of constraints. If a conflict
is detected, backtrack.

5. Backtracking:
If a conflict is encountered or if all possible label assignments have been exhausted for a
variable, backtrack to the previous variable and try a different label. Undo the previous
label assignment and choose a different label for the variable.

6. Termination Conditions:
Repeat steps 2-5 until a consistent labeling solution is found or until all possible label
assignments have been explored. Terminate the algorithm when a consistent labeling
solution is found or when all possibilities have been exhausted.

7. Solution Output:
Output the consistent labeling solution found, if any, or indicate that no solution exists.

• Perspective projective geometry

Perspective projective geometry is a branch of geometry that deals with the projection of
three-dimensional objects onto a two-dimensional plane, while preserving the spatial
relationships and perspective effects inherent in the original scene. It is fundamental in
computer graphics, computer vision, architectural drawing, and various other fields where the
representation of 3D objects in 2D space is required.

1. Projection Plane: Perspective projection involves the transformation of points in three-

dimensional space onto a two-dimensional plane known as the projection plane. This
 transformation simulates how objects appear when
viewed from a particular viewpoint in space.
2. Vanishing point: a vanishing point is a point where
projection line intersects view plane.
3. Center of Projection – It is a point where lines or
projection that are not parallel to projection plane appear
to meet.
4. View Plane or Projection Plane – The view plane is
determined by : View reference point R0(x0, y0, z0) and
View plane normal.
5. Location of an Object – It is specified by a point P that is
located in world coordinates at (x, y, z) location. The objective of perspective projection is
to determine the image point P’ whose coordinates are (x’, y’, z’)

Types of Perspective projection:

1. One Point Perspective Projection – One point perspective projection occurs when any of
principal axes intersects with projection plane or we can say when projection plane is
perpendicular to principal axis.
2. Two Point Perspective Projection – Two point perspective projection occurs when
projection plane intersects two of principal axis.
3. Three Point Perspective Projection – Three point perspective projection occurs when all
three axis intersects with projection plane. There is no principal axis which is parallel to
projection plane.

• Inverse perspective projection

Inverse perspective projection, also known as inverse perspective mapping (IPM), is the
process of transforming points from a two-dimensional image back into three-dimensional
space, particularly in the context of computer vision and robotics. While perspective projection
converts 3D points to 2D coordinates, inverse perspective projection reverses this process,
allowing for the reconstruction of depth information from a single or multiple images.

Recovering Depth Information: By analyzing the apparent size and position of objects in the
image, depth information can be inferred.

Vanishing Point Detection: Similar to perspective projection, inverse perspective projection

relies on detecting vanishing points in the image. These vanishing points correspond to the
projection of parallel lines in 3D space and provide crucial cues for estimating depth.

Depth Estimation: Once vanishing points are detected, the depth of objects can be estimated
based on their apparent size and position relative to the vanishing points. Objects closer to the
 camera will appear larger and closer to the horizon line, while those farther away will appear
smaller and further from the horizon.

• Photogrammetric from 2D to 3D
Photogrammetry is the science and technology of obtaining reliable information about physical
objects and the environment through the process of analyzing and interpreting photographs
and other imaging techniques. It involves extracting three-dimensional (3D) information from
two-dimensional (2D) images, allowing for the reconstruction of the shape, size, and spatial
relationships of objects in the real world.

The process of transforming 2D images into 3D models involves several key steps:
1. Image Acquisition:
Photogrammetry begins with the acquisition of 2D images using cameras, drones, or other
imaging devices.
2. Camera Calibration:
Before proceeding with photogrammetric analysis, the intrinsic and extrinsic parameters of
the cameras used to capture the images must be calibrated. This involves determining
factors such as focal length, lens distortion, and the position and orientation of the camera
relative to the scene.
3. Feature Extraction:
In this step, distinctive features or points are identified within the images. These features
could include corners, edges, or other recognizable patterns that can be reliably matched
across multiple images.
4. Image Matching:
Once features are extracted from the images, they are matched between overlapping
image pairs to establish correspondences. This process involves finding corresponding
points in different images that represent the same physical point in the scene.
5. Triangulation:
Using the correspondences identified in the previous step, the 3D coordinates of the
matched points are computed through triangulation.
6. Point Cloud Generation:
The triangulated 3D coordinates of the matched points are aggregated to form a dense
point cloud representation of the scene.
7. Surface Reconstruction:
From the point cloud, a continuous surface model of the scene can be generated using
surface reconstruction algorithms.
8. Texture Mapping:
Finally, texture information from the original images is applied to the surface mesh to
create a photorealistic 3D model. This involves projecting the 2D image textures onto the
 corresponding locations of the 3D surface, ensuring that the model accurately reflects the
appearance of the real-world objects.

➢ Image Matching
Image matching is a fundamental process in computer vision and image processing that involves
finding correspondences or similarities between different images or image features. It plays a
crucial role in various applications such as object recognition, image registration, stereo vision,
and motion tracking. Image matching techniques aim to identify relationships between images or
image elements, allowing for tasks like image retrieval, object detection, and 3D reconstruction.

• Intensity matching of 1D signals

- Definition: Intensity matching of 1D signals involves comparing the intensity profiles along
one-dimensional signals, such as image rows or columns.
- Process: This technique typically involves aligning or comparing the intensity profiles of
two 1D signals by measuring their similarity or dissimilarity. Methods such as correlation,
cross-correlation, or mutual information are commonly used for this purpose.
- Applications: Intensity matching of 1D signals is used in tasks like template matching,
where a template image is compared to different regions of a target image to locate
instances of the template.

• Matching of 2D image
- Definition: Matching of 2D images involves comparing entire images to find
correspondences or similarities between them.
- Process: This process typically involves feature extraction, where distinctive features are
detected and described in both images, followed by feature matching, where
correspondences between features in the two images are established. Common feature
descriptors include SIFT, SURF, ORB, and AKAZE. Matching algorithms like nearest neighbor
matching, RANSAC, and random sample consensus are often used to find robust
correspondences.
- Applications: Matching of 2D images is used in tasks such as image stitching, panorama
creation, object recognition, and image registration.

• Hierarchical image matching

- Definition: Hierarchical image matching involves performing image matching at multiple
levels of abstraction or resolution.
- Process: In this approach, images are first matched at coarse resolutions, and then the
matching process is refined at finer resolutions using progressively more detailed
information. This hierarchical strategy helps to speed up the matching process and
improve robustness to variations in scale, rotation, and viewpoint.
 - Applications: Hierarchical image matching is commonly used in tasks such as multi-scale
image registration, where images captured at different resolutions or scales need to be
aligned.

➢ Object Models and Matching

• 2D Representation
• Global vs Local Features
Local Features
Local features, also known as local descriptors, are distinct, informative characteristics of an
image or video frame that are used in computer vision and image processing. They can be
used to represent an image’s content and perform tasks such as object recognition, image
retrieval, and tracking.

Local features are typically derived from small patches or regions of an image and are
designed to be invariant to changes in illumination, viewpoint, and other factors that can
affect the image’s appearance. Common local feature extraction techniques include Scale-
Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), Oriented FAST, and
Rotated BRIEF (ORB).

Once local features have been extracted, they can be used as the basis for a variety of
computer vision tasks. For example, they can be matched between different images to
perform image registration or object recognition, or they can be used to track the movement
of objects in a video sequence.

In summary, local features are a crucial component of computer vision and image processing
systems, and they play a vital role in many applications, such as robotics, surveillance, and
augmented reality.

Global features
Global features, also known as global descriptors, are high-level, holistic characteristics of an
image or video frame that are used in computer vision and image processing. Unlike local
features, which describe distinct, informative regions of an image, global features provide a
summary of the entire image or video.

Global features are typically derived by aggregating local features in some way, such as by
computing statistics over the local feature descriptors or by constructing a histogram of the
local feature orientations. Global features can be used to represent an image’s content and
perform tasks such as image classification, scene recognition, and video segmentation.
One advantage of global features is that they are more invariant to variations in illumination,
viewpoint, and other factors that can affect the appearance of the image. This makes them
well-suited for high-level tasks that require a summary representation of the image content.

In summary, global features are a useful complement to local features in computer vision and
image processing systems, and they play a vital role in many applications, such as image
retrieval, image classification, and scene recognition.