Click to edit Master title style

NAME ----- A R P I TA S H AW
COURSE ----- B. T E C H
BRANCH ----- CSE(AI&ML)
R E G . N O. ----- 23355010038
R O L L N O. ----- 35530823013
SEMESTER - - - - - 3 rd

TOPIC:- ORTHOGONALISATION OF VECTORS

1
 CONTENT
Click to edit Master title style
• Introduction
• Vectors & Vector Spaces
• Properties
• Subspaces
• Matrices & their role in vector spaces
• Inner product spaces
• Orthogonality in vector spaces
• Gram Schmidt process
• Orthogonal and Orthonormal basis
• Applications
• Challenges
• Conclusion
• References

2 2
Click to edit Master title
INTRODUCTION
style
• Orthogonalization is the process of converting a set
of vectors into a set of mutually orthogonal vectors. • Importance of Orthogonalization:
In simpler terms, it’s about finding a way to make • Simplifies Calculations: When vectors are
vectors perpendicular to each other, which orthogonal, operations like finding projections and
simplifies many mathematical operations. decompositions become more straightforward and
less error-prone.
• Improves Numerical Stability: In numerical
computations, orthogonal vectors reduce the chance
of rounding errors, making algorithms more stable.
• Applications in Multiple Fields:
Orthogonalization is used in various fields,
including data science, computer graphics, physics,
and engineering. For example, in signal processing,
orthogonalization helps in filtering out noise and
improving signal clarity.

3 3
 VECTORS
Click to edit Master & VECTOR SPACES
title style
Definition: A vector is defined to be a directed Vector Spaces
line segment. It has both direction and
magnitude. In n-dimensional space, a vector is • A vector space is a collection of vectors that can be
represented as an ordered list of n numbers. added together and multiplied by scalars (real or
For example, in 2D space, a vector can be complex numbers), satisfying certain axioms (like
written as v = [v1, v2]. closure, associativity, distributivity, etc.).

Examples
• The space of all polynomials
of degree less than or equal
to n.
• The space of continuous
functions.

4 4
 PROPERTIES
Click to edit Master title style
• Closure under addition: If u and v are in the • Existence of the Additive Inverse: For every
vector space, then u + v is also in the vector space. element x in V, there exists another element in V
that we can call -x such that x + (-x) = 0.
• Closure under scalar multiplication: If v is in the
vector space and c is a scalar, then cv is in the • Existence of the Multiplicative Identity: There
vector space. exists an element in F notated as 1 so that for all x
in V, 1x = x.

• Commutativity of Addition: For every element x • Associativity of Scalar Multiplication: For every
and y in V, x + y = y + x. element x in V, and each pair of elements a and b in
F, (ab)x = a(bx).
• Associativity of Addition: For every element x, y,
and z in V, (x + y) + z = x + (y + z). • Distribution of Elements to Scalars: For every
element a in F and every pair of elements x and y in
V, a(x + y) = ax + ay.
• Existence of the Additive Identity: There exists
an element in V which is denoted as 0 such that x +
0 = x, for all x in V. • Distribution of Scalars to Elements: For every
element x in V, and every pair of elements a and b
in F, (a + b)x = ax + bx.
5 5
 SUBSPACES
Click to edit Master title style
• A subspace is a subset of a vector space that is
itself a vector space under the same operations of
vector addition and scalar multiplication.
• Example: In 𝑅3 , the set of all vectors lying in a
plane through the origin is a subspace.

6 6
 Matrices and Their Role in Vector Spaces
Click to edit Master title style
Definition: A matrix is a rectangular array of
numbers arranged in rows and columns. Matrix Operations
Mathematically, a matrix 𝐴 with m rows and n
columns is represented as:
• Addition: Two matrices of the same dimension can
𝑎11 ⋯ 𝑎1𝑛 be added by adding corresponding elements:
• ⋮ ⋱ ⋮ • C=A+B, where cij = aij+ bij
𝑎𝑚1 ⋯ 𝑎𝑚𝑛
• Multiplication:
• Elements: Each element of a matrix is denoted by • Scalar Multiplication: Multiplying each element
aij, where i represents the row and j represents the of a matrix by a scalar.
column.
a11 = 5, a12 = 8, a13 = -9, a14 = 0 • Matrix Multiplication: The product of a m × n
5 8
• matrix A and a n × p matrix B results in a m × p
−9 0 2x2
matrix C, where:
• Cij = σ𝑛𝑘=1 𝑎𝑖𝑘𝑏𝑘𝑗
• Transpose: The transpose of a matrix 𝐴 is obtained
by flipping rows and columns, denoted by 𝐴𝑇 .
7 7
 Role
Click to of Matrices
edit in Vector
Master title styleSpaces:

• Representation of Linear Transformations: Matrices and Orthogonalization

Matrices are fundamental in representing linear
transformations between vector spaces. A matrix A • Matrices are crucial in the process of
can transform a vector v in one vector space to orthogonalization, especially in the Gram-Schmidt
another vector w: w=Av process where we use matrices to systematically
transform a set of vectors into an orthogonal or
• System of Linear Equations: A system of linear orthonormal set.
equations can be represented in matrix form as
𝐴𝑥=𝑏, where 𝐴 is the coefficient matrix, 𝑥 is the • The Gram-Schmidt Process itself can be viewed as
vector of variables, and 𝑏 is the vector of constants. a method of constructing an orthogonal matrix from
a given set of vectors.

Basis Transformation: Matrices can be used to

change the basis of a vector space, allowing vectors
to be represented in different coordinate systems.

8 8
 Click to edit Inner Product
Master Spaces
title style
Definition: An inner product is a mathematical
operation that takes two vectors and returns a scalar. Inner Product Spaces
It generalizes the concept of the dot product. • An inner product space is a vector space equipped
with an inner product. It allows the measurement of
• For vectors 𝑢 and 𝑣 in an 𝑛-dimensional space, the inner
angles and lengths (norms) within the space.
product is denoted as ⟨𝑢,𝑣⟩.
• Formula: ⟨𝑢,𝑣⟩. = u1v1 + u2v2 + … + unvn = σ𝒏𝒊=𝟏 𝒖𝒊𝒗𝒊 • Norm of a Vector: The norm (or length) of a vector
v in an inner product space is defined as:
• For vectors u = [1, 2, 3] and v = [4, 5, 6],
• || v || = < 𝒗, 𝒗 >
• ⟨𝑢,𝑣⟩ = 1x4+2x5+3x6 = 4+10+18 = 32

• Properties of Inner Product Spaces

• Commutativity: ⟨ u, v ⟩ = ⟨ v, u ⟩
• Linearity: ⟨cu + dw, v⟩=c⟨u, v⟩+d⟨w, v⟩, where 𝑐 and 𝑑 are
scalars.
• Positivity: ⟨u, u⟩ ≥ 0 and ⟨u, u⟩ = 0 if and only if 𝑢 = 0.
• Orthogonality: Vectors 𝑢 and 𝑣 are orthogonal if ⟨u, v⟩ = 0 9 9
 Orthogonality in Vector Spaces
Click to edit Master title style
Definition of Orthogonality: Orthogonality between
two vectors u and v means that they are perpendicular
• Geometric Interpretation of Orthogonality:
to each other. In mathematical terms, two vectors are
orthogonal if their inner product is zero: • 2D Example: In two dimensions, two vectors are
orthogonal if the angle between them is 90 ∘. For
⟨u, v⟩ = 0
instance, the vectors u = [3,0] and v = [0,4] are
• Orthogonal Vectors: In 𝑅2 , vectors u = [1,0] and v orthogonal, and their geometric representation would
= [0,1] are orthogonal because their dot product is show them forming a right angle.
zero: ⟨u, v⟩=1×0+0×1=0 • 3D Example: In three-dimensional space, vectors u =
• Orthogonality is a key concept in linear algebra, [1,0,0] and v = [0,1,0] are orthogonal, meaning they
especially when dealing with bases of vector spaces lie along the x-axis and y-axis, respectively, and are
and projections. perpendicular.

1010
Click to edit Master title style
• Orthogonal Sets of Vectors: A set of vectors {v1,
v2, …, vn} is orthogonal if every pair of distinct
vectors in the set is orthogonal:
Importance of Orthogonality:
Simplification of Computations: Orthogonal vectors
Orthonormal Set: If, in addition to being orthogonal,
simplify many calculations in linear algebra, such as
each vector in the set has a norm of 1, the set is called
projections, decompositions, and solving linear
orthonormal:
systems.
Projection of Vectors: The projection of a vector v
onto an orthogonal vector u simplifies to

1111
 The Gram-Schmidt Process
Click to edit Master title style
• 1. Introduction to the Gram-Schmidt Process:
The Gram-Schmidt process is a method used to
orthogonalize a set of vectors in an inner product
space. Starting with a linearly independent set of
vectors, the process generates an orthogonal (or
orthonormal) set of vectors that spans the same
subspace as the original set.
• This process is fundamental in linear algebra and is • 2. Purpose of the Gram-Schmidt Process:
used to simplify many problems involving vector • Orthogonalization: Converts a set of linearly
spaces, such as finding an orthogonal basis for a independent vectors into an orthogonal set.
subspace.
• Orthonormalization: Further normalizes the
orthogonal vectors to create an orthonormal set,
where each vector has a unit length.
• Basis Construction: Helps in constructing an
orthogonal or orthonormal basis for a vector space,
which simplifies many calculations, including
projections and solving systems of linear equations.
1212
Click to edit Master title style

Applications of the Gram-Schmidt Process:

• QR Decomposition: The Gram-Schmidt process is
used to decompose a matrix 𝐴 into the product of
an orthogonal matrix 𝑄 and an upper triangular
matrix 𝑅 (i.e., 𝐴=𝑄𝑅).
• Solving Linear Systems: Orthogonalization is used
to simplify the solution of linear systems by
converting the matrix into an easier form to work
with.

1313
 Orthogonal and Orthonormal Basis
Click to edit Master title style
• An orthogonal basis for a vector space is a set of
vectors that are all mutually orthogonal to each An orthonormal basis is a special case of an
orthogonal basis where each vector is not only
other. This means that for any two distinct vectors
orthogonal but also has a unit length (norm equals 1).
vi​ and vj​ in the basis, their inner product is zero: For any vector vi in the basis:
|| vi || = 1
The inner product of any two distinct vectors in an
• Example of an Orthogonal Basis: orthonormal basis is still zero, and the inner product
of a vector with itself is 1:
Consider the standard basis in 𝑅3

Example:

These vectors are mutually orthogonal because the

dot product of any pair is zero.
These vectors are orthogonal, and their norms are
equal to 1, making them orthonormal.
1414
 APPLICATIONS
Click to edit Master title style
• Noise Reduction: Orthogonalization helps in
filtering out noise from signals by isolating the • Solving Linear Systems: In numerical linear
most relevant components of the signal while algebra, orthogonalization is used in methods such
discarding components that are orthogonal to the as QR decomposition to solve systems of linear
signal of interest. equations. QR decomposition expresses a matrix as
the product of an orthogonal matrix and an upper
• Quantum Computing: In quantum computing, triangular matrix, simplifying the solution process
orthogonalization techniques are used in algorithms and improving numerical stability.
and error correction to maintain the coherence and
separability of quantum states, which are essential
for accurate computations.

1515
Click to edit CHALLENGES
Master title style
• Rounding Errors: In practical computations, • Poorly Conditioned Vectors: If the initial
especially with floating-point arithmetic, set of vectors is nearly linearly dependent
rounding errors can accumulate when performing or very close to each other in space,
orthogonalization processes, such as the Gram- orthogonalization processes like Gram-
Schmidt process. These errors can lead to vectors Schmidt can become unstable or produce
that are not perfectly orthogonal, reducing the less accurate results. The process might
accuracy of the results. amplify errors, leading to vectors that are
nearly orthogonal but not exactly, which
• Efficiency in Large Systems: can affect the stability and accuracy of
Orthogonalization, particularly in large matrices subsequent calculations.
or high-dimensional vector spaces, can be • Complex Vector Spaces: When dealing
computationally expensive. The Gram-Schmidt with complex vector spaces, the
process, for example, involves multiple dot orthogonalization process becomes more
products and vector subtractions, which can be complicated due to the need to consider
slow for very large datasets. complex conjugates in the inner product.
This adds layer of complexity to both the
theory and implementation of
orthogonalization methods.

1616
Click to edit Master title style
CONCLUSION
• We began by understanding the fundamental concept of orthogonalization, which is the process of
converting a set of vectors into mutually orthogonal vectors. This process is essential in linear algebra for
simplifying calculations and ensuring numerical stability.
• We explored the importance of inner product spaces, where the concept of orthogonality is defined, and
how the Gram-Schmidt process provides a systematic way to orthogonalize a set of vectors.
• We also examined the difference between orthogonal and orthonormal bases, understanding how they
provide a simplified and efficient framework for various mathematical operations.
• Orthogonalization reduces complexity in vector operations, making it easier to solve linear systems,
perform projections, and decompose matrices.
• In numerical methods, the stability provided by orthogonal vectors ensures more accurate and reliable
computations.
• In signal processing and quantum mechanics, the use of orthogonal and orthonormal bases is crucial for
efficient and precise calculations, leading to advancements in technology and science.
• Orthogonalization is more than just a mathematical tool—it is a powerful technique that underpins many
modern technologies and scientific discoveries. Understanding this concept not only enhances one’s grasp
of linear algebra but also opens doors to its wide-ranging applications in various disciplines.

1717
Click to edit Master title style
REFERENCES

• Advanced Higher Algebra, Chakravorty and Ghosh

• https://en.wikipedia.org/wiki/Orthogonalization

1818
Click to edit Master title style

Thank You

19