Linear Algebra & its Applications

Shreyus Goutham Kumar

Department of Electronics and Communication
Engineering

1
Linear Algebra & its Applications

Unit 1 : Lecture Notes

Shreyus Goutham Kumar

Department of Electronics and Communication Engineering
2
LINEAR ALGEBRA & ITS APPLICATIONS
UE20MA251

Matrices and Gaussian Elimination

Shreyus Goutham Kumar

Department of Electronics and Communication Engineering
3
LINEAR ALGEBRA - UE20MA251
PESU – EC Campus

Course Instructor

1. Mr. Shreyus Goutham Kumar

Assistant Professor
Dept of ECE
PESU – EC Campus
Email : [email protected]

4
LINEAR ALGEBRA - UE20MA251
Syllabus

• Unit 1: 12 hrs

• Matrices and Gaussian Elimination:

Introduction, The Geometry of Linear Equations, Gaussian Elimination, Singular Cases,
Elimination Matrices, Triangular Factors and Row Exchanges, Inverses and Transposes, Inverse
by Gauss -Jordan method
Self-Learning Component: Algebra of Matrices.
• Text-Book : Chapter 1

5
LINEAR ALGEBRA - UE20MA251
Text Book

6
LINEAR ALGEBRA - UE20MA251
Unit 1 : Lesson Plan

7
LINEAR ALGEBRA - UE20MA251
Applications

8
LINEAR ALGEBRA - UE20MA251
MATLAB

9
 LINEAR ALGEBRA - UE20MA251

10
*https://www.youtube.com/watch?v=CXF3z8xdIeo&list=PLFW6lRTa1g829epNHKEjb9KdHd2uUtW83&index=1
 LINEAR ALGEBRA - UE20MA251

11
*https://www.youtube.com/watch?v=CXF3z8xdIeo&list=PLFW6lRTa1g829epNHKEjb9KdHd2uUtW83&index=1
 LINEAR ALGEBRA - UE20MA251

12
*https://www.youtube.com/watch?v=CXF3z8xdIeo&list=PLFW6lRTa1g829epNHKEjb9KdHd2uUtW83&index=1
 LINEAR ALGEBRA - UE20MA251

13
*https://www.youtube.com/watch?v=CXF3z8xdIeo&list=PLFW6lRTa1g829epNHKEjb9KdHd2uUtW83&index=1
 LINEAR ALGEBRA - UE20MA251

14
*https://www.youtube.com/watch?v=CXF3z8xdIeo&list=PLFW6lRTa1g829epNHKEjb9KdHd2uUtW83&index=1
 LINEAR ALGEBRA - UE20MA251

15
*https://www.youtube.com/watch?v=CXF3z8xdIeo&list=PLFW6lRTa1g829epNHKEjb9KdHd2uUtW83&index=1
 LINEAR ALGEBRA - UE20MA251

16
*https://www.youtube.com/watch?v=CXF3z8xdIeo&list=PLFW6lRTa1g829epNHKEjb9KdHd2uUtW83&index=1
LINEAR ALGEBRA - UE20MA251

17
LINEAR ALGEBRA - UE20MA251

18
LINEAR ALGEBRA - UE20MA251

19
LINEAR ALGEBRA - UE20MA251

20
LINEAR ALGEBRA - UE20MA251

21
LINEAR ALGEBRA - UE20MA251

22
LINEAR ALGEBRA - UE20MA251

23
LINEAR ALGEBRA - UE20MA251

24
LINEAR ALGEBRA - UE20MA251

25
LINEAR ALGEBRA - UE20MA251

26
LINEAR ALGEBRA - UE20MA251

27
LINEAR ALGEBRA - UE20MA251

Good method is Gaussian Elimination

28
LINEAR ALGEBRA - UE20MA251

29
LINEAR ALGEBRA - UE20MA251

30
LINEAR ALGEBRA - UE20MA251

31
LINEAR ALGEBRA - UE20MA251

32
LINEAR ALGEBRA - UE20MA251

33
LINEAR ALGEBRA - UE20MA251

THE GEOMETRY OF LINEAR EQUATIONS

We can look at this system by rows or by columns

34
LINEAR ALGEBRA - UE20MA251

Row picture : This first approach concentrates on the separate equations

(the rows).

35
LINEAR ALGEBRA - UE20MA251

Column picture

The second approach looks at the columns of the linear system. The two
separate equations are really one vector equation:

The problem is to find the

combination of the column
vectors on the left side that
produces the vector on the
right side.

36
LINEAR ALGEBRA - UE20MA251

37
LINEAR ALGEBRA - UE20MA251

38
LINEAR ALGEBRA - UE20MA251

How does this row picture extend into n dimensions? Then equations will
contain n unknowns. The first equation still determines a "plane." It is no
longer a two-dimensional plane in 3-space; somehow it has "dimension" n - 1.

The point is that linear algebra can operate with any number of equations.
The first equation produces an (n - 1) -dimensional plane in n dimensions. The
second plane intersects it (we hope) in a smaller set of "dimension n - 2."
Assuming all goes well, every new plane (every new equation) reduces the
dimension by one. At the end, when all n planes are accounted for, the
intersection has dimension zero. It is a point, it lies on all the planes, and its
coordinates satisfy all n equations. It is the solution!

39
LINEAR ALGEBRA - UE20MA251

Column Vectors and Linear Combinations

40
LINEAR ALGEBRA - UE20MA251

• Those are three-dimensional column vectors. The vector b is identified with the
point whose coordinates are 5, -2, 9.

• Every point in three-dimensional space is matched to a vector, and vice versa. That
was the idea of Descartes, who turned geometry into algebra by working with the
coordinates of the point.

• We can write the vector in a column, or we can list its components as b = (5, -2, 9),
or we can represent it geometrically by an arrow from the origin.

• You can choose the arrow, or the point, or the three numbers.

• In six dimensions it is probably easiest to choose the six numbers.

41
LINEAR ALGEBRA - UE20MA251

42
LINEAR ALGEBRA - UE20MA251

In the figure is one of the central ideas of linear

algebra. It uses both the basic operations;
vectors are multiplied by numbers and then
added. The result is called a linear combination,
and this combination solves our equation:

43
LINEAR ALGEBRA - UE20MA251

• Our true goal is to look beyond two or three dimensions into n dimensions.

• With n equations in n unknowns, there are n planes in the row picture.

• There are n vectors in the column picture, plus a vector b on the right side. The equations
ask for a linear combination of the n columns that equals b.

• For certain equations that will be impossible.

• Paradoxically, the way to understand the good case is to study the bad one.

• Therefore we look at the geometry exactly when it breaks down, in the singular case.

44
LINEAR ALGEBRA - UE20MA251

The Singular Case

45
LINEAR ALGEBRA - UE20MA251

46
LINEAR ALGEBRA - UE20MA251

47
LINEAR ALGEBRA - UE20MA251

48
LINEAR ALGEBRA - UE20MA251

• The method starts by subtracting multiples of the first equation from the other
equations.

• The goal is to eliminate u from the last two equations. This requires that we

49
LINEAR ALGEBRA - UE20MA251

• The coefficient 2 is the first pivot.

• The pivot for the second stage of elimination is -8.

50
LINEAR ALGEBRA - UE20MA251

51
LINEAR ALGEBRA - UE20MA251

To repeat:

• Forward elimination produced the pivots 2, -8, 1.

• It subtracted multiples of each row from the rows beneath.

• It reached the "triangular" system (3), which is solved in reverse order:

Substitute each newly computed value into the equations that are
waiting.

52
LINEAR ALGEBRA - UE20MA251

53
LINEAR ALGEBRA - UE20MA251

By definition, pivots cannot be zero.

54
 LINEAR ALGEBRA - UE20MA251

The Breakdown of Elimination

Under what circumstances could the process break down? Something must go
wrong in the singular case, and something might go wrong in the nonsingular
case.

55
LINEAR ALGEBRA - UE20MA251

56
LINEAR ALGEBRA - UE20MA251

57
LINEAR ALGEBRA - UE20MA251

58
LINEAR ALGEBRA - UE20MA251

59
 LINEAR ALGEBRA - UE20MA251

So far….

60
LINEAR ALGEBRA - UE20MA251

61
LINEAR ALGEBRA - UE20MA251

62
LINEAR ALGEBRA - UE20MA251

63
LINEAR ALGEBRA - UE20MA251

64
LINEAR ALGEBRA - UE20MA251

65
LINEAR ALGEBRA - UE20MA251

66
LINEAR ALGEBRA - UE20MA251

ROW REDUCTION AND ECHELON FORMS

• Row reduction algorithm that will enable us to analyze any system of

linear equations.

• By using only, the first part of the algorithm, we will be able to answer
the fundamental existence and uniqueness questions

• The algorithm applies to any matrix, whether or not the matrix is viewed as an
augmented matrix for a linear system.

• In the definitions that follow, a non-zero row or column in a matrix means a

row or column that contains at least one nonzero entry; a leading entry of a
row refers to the leftmost nonzero entry (in a nonzero row).
67
LINEAR ALGEBRA - UE20MA251

68
LINEAR ALGEBRA - UE20MA251

69
LINEAR ALGEBRA - UE20MA251

70
LINEAR ALGEBRA - UE20MA251

71
LINEAR ALGEBRA - UE20MA251

72
LINEAR ALGEBRA - UE20MA251

73
LINEAR ALGEBRA - UE20MA251

74
LINEAR ALGEBRA - UE20MA251

75
LINEAR ALGEBRA - UE20MA251

76
LINEAR ALGEBRA - UE20MA251

77
LINEAR ALGEBRA - UE20MA251

78
LINEAR ALGEBRA - UE20MA251

79
LINEAR ALGEBRA - UE20MA251

80
LINEAR ALGEBRA - UE20MA251

81
LINEAR ALGEBRA - UE20MA251

82
LINEAR ALGEBRA - UE20MA251

83
LINEAR ALGEBRA - UE20MA251

84
LINEAR ALGEBRA - UE20MA251

85
LINEAR ALGEBRA - UE20MA251

86
LINEAR ALGEBRA - UE20MA251

87
LINEAR ALGEBRA - UE20MA251

88
LINEAR ALGEBRA - UE20MA251

89
LINEAR ALGEBRA - UE20MA251

90
LINEAR ALGEBRA - UE20MA251

91
LINEAR ALGEBRA - UE20MA251

92
LINEAR ALGEBRA - UE20MA251

93
LINEAR ALGEBRA - UE20MA251

94
LINEAR ALGEBRA - UE20MA251

95
LINEAR ALGEBRA - UE20MA251

96
LINEAR ALGEBRA - UE20MA251

MATRIX NOTATION AND MATRIX MULTIPLICATION

97
LINEAR ALGEBRA - UE20MA251

98
LINEAR ALGEBRA - UE20MA251

99
 LINEAR ALGEBRA - UE20MA251

Multiplication of a Matrix and a Vector

100
LINEAR ALGEBRA - UE20MA251

101
LINEAR ALGEBRA - UE20MA251

102
LINEAR ALGEBRA - UE20MA251

103
LINEAR ALGEBRA - UE20MA251

104
LINEAR ALGEBRA - UE20MA251

105
LINEAR ALGEBRA - UE20MA251

106
 LINEAR ALGEBRA - UE20MA251

The Matrix Form of One Elimination Step

107
LINEAR ALGEBRA - UE20MA251

108
LINEAR ALGEBRA - UE20MA251

109
LINEAR ALGEBRA - UE20MA251

110
LINEAR ALGEBRA - UE20MA251

111
LINEAR ALGEBRA - UE20MA251

112
LINEAR ALGEBRA - UE20MA251

113
LINEAR ALGEBRA - UE20MA251

114
LINEAR ALGEBRA - UE20MA251

115
LINEAR ALGEBRA - UE20MA251

116
LINEAR ALGEBRA - UE20MA251

117
LINEAR ALGEBRA - UE20MA251

118
LINEAR ALGEBRA - UE20MA251

119
LINEAR ALGEBRA - UE20MA251

120
LINEAR ALGEBRA - UE20MA251

121
LINEAR ALGEBRA - UE20MA251

122
LINEAR ALGEBRA - UE20MA251

123
LINEAR ALGEBRA - UE20MA251

124
LINEAR ALGEBRA - UE20MA251

125
LINEAR ALGEBRA - UE20MA251

126
LINEAR ALGEBRA - UE20MA251

Definition The nonsingular matrix A has a triangular factorization if it

can be expressed as the product of a lower-triangular matrix L
and an upper-triangular matrix U:

A = LU
LINEAR ALGEBRA - UE20MA251

Theorem : (Direct Factorization A = LU: No Row Interchanges).

Suppose that Gaussian elimination, without row interchanges, can be
performed successfully to solve the general linear system AX = B. Then
the matrix A can be factored as the product of a lower-triangular matrix L
and an upper-triangular matrix U:

A = LU
LINEAR ALGEBRA - UE20MA251

129
LINEAR ALGEBRA - UE20MA251

130
 LINEAR ALGEBRA - UE20MA251

One Linear System = Two Triangular Systems

131
LINEAR ALGEBRA - UE20MA251

132
LINEAR ALGEBRA - UE20MA251

133
LINEAR ALGEBRA - UE20MA251

134
LINEAR ALGEBRA - UE20MA251

135
LINEAR ALGEBRA - UE20MA251

136
LINEAR ALGEBRA - UE20MA251

137
LINEAR ALGEBRA - UE20MA251

138
LINEAR ALGEBRA - UE20MA251

139
LINEAR ALGEBRA - UE20MA251

140
LINEAR ALGEBRA - UE20MA251

141
LINEAR ALGEBRA - UE20MA251

142
LINEAR ALGEBRA - UE20MA251

143
LINEAR ALGEBRA - UE20MA251

144
LINEAR ALGEBRA - UE20MA251

145
LINEAR ALGEBRA - UE20MA251

146
LINEAR ALGEBRA - UE20MA251

147
LINEAR ALGEBRA - UE20MA251

148
LINEAR ALGEBRA - UE20MA251

149
LINEAR ALGEBRA - UE20MA251

150
LINEAR ALGEBRA - UE20MA251

151
LINEAR ALGEBRA - UE20MA251

152
LINEAR ALGEBRA - UE20MA251

153
LINEAR ALGEBRA - UE20MA251

154
LINEAR ALGEBRA - UE20MA251

155
LINEAR ALGEBRA - UE20MA251

156
LINEAR ALGEBRA - UE20MA251

157
LINEAR ALGEBRA - UE20MA251

158
LINEAR ALGEBRA - UE20MA251

159
LINEAR ALGEBRA - UE20MA251

160
LINEAR ALGEBRA - UE20MA251

161
LINEAR ALGEBRA - UE20MA251

162
LINEAR ALGEBRA - UE20MA251

163
LINEAR ALGEBRA - UE20MA251

164
LINEAR ALGEBRA - UE20MA251

165
LINEAR ALGEBRA - UE20MA251

166
LINEAR ALGEBRA - UE20MA251

167
LINEAR ALGEBRA - UE20MA251

168
LINEAR ALGEBRA - UE20MA251

169
LINEAR ALGEBRA - UE20MA251

170
LINEAR ALGEBRA - UE20MA251

171
LINEAR ALGEBRA - UE20MA251

172
LINEAR ALGEBRA - UE20MA251

173
LINEAR ALGEBRA - UE20MA251

174
LINEAR ALGEBRA - UE20MA251

175
LINEAR ALGEBRA - UE20MA251

176
LINEAR ALGEBRA - UE20MA251

177
THANK YOU

Shreyus Goutham Kumar

Department of Electronics and Communication
Engineering
[email protected]

178