Motion Tracking

Presenters:
Hana AL Barghouthy
Hiba Mansour

Supervised By:
Dr. Anas
1
Toma
 Outline

1-Introduction for Motion

Tracking “ 5-Methods of Motion Tracking

6-Feature Extraction in Motion

2-Importance of Motion
Tracking
Tracking
7-Recent Developments in
3-Optical Flow and Motion
Tracking Tracking Algorithms

4-Overview of How Motion 2

8-Real-world applications
 Outline

“
Introduction to Motion Tracking

3
 Introduction
Definition of Motion Tracking:

Motion tracking is the process of monitoring and recording

the movement of objects or people within a given space
over time. It involves the use of technology to analyze and
capture the dynamic positions, orientations, and
trajectories of these entities, enabling a detailed
understanding of their motion patterns.

4
 Outline

“
Importance of Motion Tracking

5
 Importance of Motion Tracking
1. Entertainment Industry
2. Surveillance and Security
3. Robotics and Autonomous Systems
4. Sports Analysis
5. Human-Computer Interaction
6. Healthcare

6
Motion Field

7
 Motion Field
𝑑𝑟 𝑖 𝑑𝑟 0
𝑣𝑖=
Image Point Velocity: 𝑣0=
Scene Point Velocity:
(Motion field) 𝑑𝑡 𝑑𝑡

𝑖 𝑟 𝑟0
=
Perspective projection:
𝑓 𝑟0 . 𝑧

𝑑𝑟 𝑖 ( 𝑣 ¿¿ 0 . 𝑧 ) 𝑟 0
Image Point 𝑣𝑖 =
Velocity: 𝑓 (𝑟 0 . 𝑧 ) 𝑣0 −
=¿ ¿
(Motion field) 𝑑𝑡 2
(𝑟 ¿¿ 0 . 𝑧 ) ¿

( 𝑟 0 × 𝑣 0 ) × 𝑧❑
¿ 𝑓 2
(𝑟 ¿ ¿ 0 . 𝑧 ) ¿ 8
 Outline

“
Optical Flow and Motion Tracking

9
 Optical Flow and Motion Tracking
▷ Optical Flow and Motion Tracking:
Optical flow and motion tracking are interconnected, with optical flow serving as a
fundamental tool within the motion tracking process.

▷ How Optical Flow Contributes to Tracking:

Contribution: Optical flow contributes by estimating the apparent motion of pixels or
keypoints between consecutive frames.

Role: This information enhances the tracking process by providing a detailed

understanding of how objects move, aiding in the identification and monitoring of
specific features or entities within a visual sequence.

10
Optical Flow Motion Field
Motion of brightness patterns in the
image

Ideally, Optical Flow = Motion

Field 11
 Optical Flow Motion Field

Spinning Sphere Stationary Sphere

Stationary Light Source Moving Light Source

Motion Field exists No Motion Field exists

But no Optical Flow But there is Optical Flow 12
 Optical Flow Motion Field

Barber Pole Motion Field Optical Flow

Illusion
13
 Outline

“
Optical Flow Constraint Equation

14
 Optical Flow Constraint equation

𝑡 𝑡+𝛿 𝑡
(𝑥 , 𝑦 ) ( 𝑥+ 𝛿 𝑥 , 𝑦 + 𝛿 𝑦 )
𝜕𝑥 𝛿 𝑦
(𝛿 𝑥 , 𝛿 𝑦 )
Displacement: ( 𝑢 , 𝑣 )=(
Optical Flow: , )
𝜕 𝑡 𝛿𝑡
15
 Optical Flow Constraint equation
▷ Assumption 1:
Brightness of image point remains constant over timeBy solving these two equations
and divided by :

𝐼 ( 𝑥 +𝛿 𝑥 , 𝑦 +𝛿 𝑦 , 𝑡+𝛿𝑡 )= 𝐼(𝑥 , 𝑦 ,𝑡)

▷ Assumption 2:
Displacement (, and time step are small Constraint Equation

Tayler Series 𝐼 𝑥 𝑢 + 𝐼 𝑦 𝑣 + 𝐼 𝑡 =0
++ + optical flow

+++

16
 Optical Flow Constraint equation
 For any point on the image, its optical
flow lies on the constraint line

 Optical flow can be split into two

components

But we cannot determine

17
Aperture Problem

Actual
Motion

18
Outline

“
Lucas Kanade Solution

19
 Lucas Kanade Solution
Assumption: For each pixel, assume Motion field and hence Optical flow is
constant within a small neighborhood

20
 Lucas Kanade Solution
𝐼 𝑥 𝑢 + 𝐼 𝑦 𝑣 + 𝐼 𝑡 =0

21
 Lucas Kanade Solution
Conditions:

𝑇
(𝐴 𝐴 ) be
Must
invertible
• Must be well conditioned

Both are
small
Smooth
region
Bad case 22
 Lucas Kanade Solution
Conditions:

𝑇
(𝐴 𝐴 ) be
Must
invertible
• Must be well conditioned

𝜆 1 ≫ > 𝜆2
Edges region
Bad case
23
 Lucas Kanade Solution
Conditions:

𝑇
(𝐴 𝐴 ) be
Must
invertible
• Must be well conditioned

Both
large
Textured
region
Good case 24
Outline

“
Large Motion

25
 Coarse To Fine Estimation algorithm

+ 𝛿 𝑡

Resolutio Resolutio Resolutio

Resolutio
n n n
n

26
Coarse To Fine Estimation algorithm

27
 Outline

“
Overview of How Motion Tracking Works

28
 Overview of How Motion Tracking Works
• Motion tracking involves monitoring and recording the movement of objects or people
in a video sequence over time.
• The process typically includes the identification of key features, their tracking from
frame to frame, and the computation of their trajectories.

29
 Outline

“
Methods of Motion Tracking

30
 Block Matching Techniques
▷ Block Matching:
Block matching is a technique used in motion tracking that involves dividing the image
into blocks and finding the best match between corresponding blocks in consecutive
frames.

• Pros:
• Simple and computationally efficient.
• Suitable for scenarios with consistent motion and predictable patterns.
• Provides a basic yet effective approach for motion estimation.
• Cons:
• Limited accuracy in the presence of complex motion or
occlusions.
• May lead to artifacts in regions with sudden changes.
• Sensitivity to noise in the image data.

31
 Methods of Motion Tracking
▷ Correlation Tracking:
• Definition: Correlation tracking involves matching patterns or features between
consecutive frames based on similarity metrics.
• Key Characteristics: It relies on comparing pixel values or image regions to establish
correspondences, making it suitable for tracking objects with distinctive textures.

32
 Methods of Motion Tracking
▷ Feature-Based Tracking:
• Definition: Feature-based tracking identifies and tracks specific features or keypoints
in the image, such as corners or edges.
• Key Characteristics: It focuses on tracking distinctive points, providing robustness to
changes in scale, rotation, and illumination.

33
 Methods of Motion Tracking
▷ Hybrid Methods:
• Definition: Hybrid methods combine different tracking approaches to leverage their
respective strengths.
• Key Characteristics: By integrating correlation tracking and feature-based methods,
hybrid approaches aim to enhance overall tracking performance, improving accuracy
and adaptability in various scenarios.

34
 Outline

“
Feature Extraction in Motion Tracking

35
 Feature Extraction in Motion Tracking

▷ Importance of Identifying and Tracking Features:

• Importance:

Identifying and tracking features is crucial for robust and accurate motion tracking.

• Reasons:
• Features serve as distinctive points, ensuring tracking resilience to changes in
scale, rotation, and illumination.
• They enable precise localization and monitoring of specific objects or entities in a
scene.

36
 Key Methods for Feature Extraction:

▷ Corner Detection Algorithms:

Identify corner points, e.g., Harris corner detector.

▷ Edge-based Methods:
Focus on tracking edges in the image.

▷ Blob Detection:
Identify regions of interest based on intensity variations.

37
 Key Methods for Feature Extraction:

▷ Scale-Invariant Feature Transform (SIFT):

Extracts distinctive keypoints invariant to scaling and rotation.

38
 Common Challenges in Motion Tracking
▷ Illumination Changes:
1. Challenge: Variations in lighting conditions can affect the appearance of objects,
making it challenging to maintain accurate tracking.
2. Impact: Sudden changes in illumination may lead to misinterpretation of object
features and hinder the tracking process.

▷ Occlusions:
1. Challenge: When objects partially or fully block each other in the scene, it
becomes difficult to track them continuously.
2. Impact: Occlusions can result in temporary loss of tracking, requiring robust
algorithms to handle such interruptions.
.
39
 Common Challenges in Motion Tracking
▷ Complex Scenes:

1. Challenge: Scenes with intricate patterns, multiple moving objects, or fast motion
can introduce complexities.
2. Impact: The complexity of the scene can pose challenges for algorithms to
accurately distinguish and track individual objects, affecting overall tracking
performance.

40
 Outline

“
Recent Developments in Tracking Algorithms

41
 Recent Developments in Tracking Algorithms
• Advancements: Recent years have seen significant improvements in tracking
algorithms.
• Examples:
• Deep learning-based trackers leverage convolutional neural networks (CNNs) for
enhanced object recognition and tracking.
• Online learning algorithms adapt to changing tracking conditions, improving real-
time performance.

42
Integration with Artificial Intelligence and Machine
Learning
• Integration: Motion tracking is increasingly integrated with artificial intelligence (AI)
and machine learning (ML) techniques.
• Benefits:
• ML algorithms improve tracking accuracy by learning from diverse datasets.
• AI-based trackers adapt to varying tracking scenarios, enhancing overall
robustness and efficiency.

43
Outline

“
Real-world applications

44
 Real-world applications

▷ Surveillance and Security:

• Application: Motion tracking is extensively used in video surveillance systems.
• Role: Enables the monitoring and identification of moving objects for enhanced
security
• Benefits: Improves the accuracy of threat detection and minimizes false alarms.

45
 Real-world applications

▷ Robotics and Autonomous Systems:

• Application: Motion tracking plays a crucial role in the field of robotics.
• Role: Facilitates navigation, obstacle avoidance, and interaction with the
environment.
• Benefits: Enhances the autonomy and efficiency of robots, contributing to
applications in fields such as manufacturing, logistics, and exploration.

46
Motion Tracking in the Entertainment Industry Field

▷ Virtual Reality and Augmented Reality:

• Application: Motion tracking is essential for immersive experiences in virtual and
augmented reality.
• Role: Tracks the user's movements, enabling interaction with virtual objects and
environments.
• Benefits: Enhances the sense of presence and realism in VR/AR applications, from
gaming to training simulations.

47
 References

1. https://mpolinowski.github.io/docs/IoT-and-Machine-Learning/ML/2021-12-10--opencv-
optical-flow-tracking/2021-12-10/

2. http://vision.stanford.edu/teaching/cs231a_autumn1112/lecture/lecture13_optical_flo
w_cs231a.pdf

3. https://www.baeldung.com/cs/optical-flow-lucas-kanade-method

4. https://www.sciencedirect.com/science/article/abs/pii/S1047320314001941

5. https://www.cs.cornell.edu/sumit/projects/3D_Geometry/feature.html

6. https://www.researchgate.net/publication/221125432_Object_tracking_by_adaptive_fe
ature_extraction

7. https://www.sciencedirect.com/science/article/abs/pii/S0925231221007220
48
Thanks!
Any questions?