Signal Space Representations

Dr. James S. Kang

1
 Vector Space Concepts
𝑣𝑣11 𝑣𝑣21
𝑣𝑣12 𝑣𝑣22
• 𝒗𝒗1 = … , 𝒗𝒗2 = …
𝑣𝑣1𝑛𝑛 𝑣𝑣2𝑛𝑛
• The dot product (inner product) of 𝒗𝒗1 and 𝒗𝒗2 is defined as 𝒗𝒗1𝑇𝑇 𝒗𝒗2 .
𝑣𝑣21
𝑣𝑣22
• 𝒗𝒗1 � 𝒗𝒗2 = 𝒗𝒗2 � 𝒗𝒗1 = 𝒗𝒗1 𝒗𝒗2 = 𝒗𝒗2 𝒗𝒗1 = 𝑣𝑣11 𝑣𝑣12 … 𝑣𝑣1𝑛𝑛 … =
𝑇𝑇 𝑇𝑇

𝑣𝑣2𝑛𝑛
𝑣𝑣11 𝑣𝑣21 + 𝑣𝑣12 𝑣𝑣22 + ⋯ + 𝑣𝑣1𝑛𝑛 𝑣𝑣2𝑛𝑛

2
 Orthogonal Vectors
• Two vectors𝑇𝑇𝒗𝒗1 and 𝒗𝒗2 are orthogonal if the inner product is zero, i.e.,
𝒗𝒗1 � 𝒗𝒗2 = 𝒗𝒗1 𝒗𝒗2 = 0.
• A set of m vectors, 𝒗𝒗𝑘𝑘 , 1 ≤ k ≤ m, are orthogonal if
• 𝒗𝒗𝑇𝑇𝑖𝑖 𝒗𝒗𝑗𝑗 = 0
• for all 1 ≤ i, j ≤ m and i ≠ j.
• Norm of a vector 𝒗𝒗 is defined as

• 𝒗𝒗 = 𝒗𝒗 � 𝒗𝒗 = 𝒗𝒗𝑇𝑇 𝒗𝒗 = ∑𝑛𝑛𝑖𝑖=1 𝑣𝑣𝑖𝑖2

3
 Orthonormal vectors
• A set of m vectors are orthonormal if they are orthogonal and normal.
𝑇𝑇 0, 𝑖𝑖 ≠ 𝑗𝑗
• 𝒗𝒗𝑖𝑖 � 𝒗𝒗𝑗𝑗 = 𝒗𝒗𝑖𝑖 𝒗𝒗𝑗𝑗 = �
1, 𝑖𝑖 = 𝑗𝑗
• The length of each vector is one.
• A set of vectors is said to be linearly independent if no one vector can
be represented as a linear combination of remaining vectors.

4
 Cauchy-Schwarz Inequality
• 𝒗𝒗1 � 𝒗𝒗2 ≤ 𝒗𝒗1 𝒗𝒗2
• Equality if 𝒗𝒗1 = 𝑎𝑎𝒗𝒗2 .
• Proof 𝑡𝑡 is a scalar.
• Let 𝑃𝑃 𝑡𝑡 = 𝑡𝑡𝒗𝒗2 − 𝒗𝒗1 2 = 𝑡𝑡𝒗𝒗2 − 𝒗𝒗1 � 𝑡𝑡𝒗𝒗2 − 𝒗𝒗1 ≥ 0
• 𝑃𝑃 𝑡𝑡 = 𝒗𝒗2 � 𝒗𝒗2 𝑡𝑡 2 − 2𝒗𝒗1 � 𝒗𝒗2 𝑡𝑡 + 𝒗𝒗1 � 𝒗𝒗1
• Let 𝑎𝑎 = 𝒗𝒗2 � 𝒗𝒗2 = 𝒗𝒗2 2 , 𝑏𝑏 = 2(𝒗𝒗1 � 𝒗𝒗2 ), 𝑐𝑐 = 𝒗𝒗1 � 𝒗𝒗1 = 𝒗𝒗1 2

• 𝑃𝑃 𝑡𝑡 = 𝑎𝑎𝑡𝑡 2 − 𝑏𝑏𝑏𝑏 + 𝑐𝑐 ≥ 0
𝑏𝑏 𝑏𝑏 2 𝑏𝑏 𝑏𝑏 2
• 𝑃𝑃 = 𝑎𝑎 − 𝑏𝑏 + 𝑐𝑐 = − + 𝑐𝑐 ≥ 0
2𝑎𝑎 2𝑎𝑎 2𝑎𝑎 4𝑎𝑎
5
 Cauchy-Schwarz Inequality
• 4𝑎𝑎𝑎𝑎 ≥ 𝑏𝑏 2
• 4 𝒗𝒗2 2 𝒗𝒗1 2 ≥ 4(𝒗𝒗1 � 𝒗𝒗2 )2
• 𝒗𝒗1 𝒗𝒗2 ≥ 𝒗𝒗1 � 𝒗𝒗2
• If 𝒗𝒗2 = 𝑐𝑐𝒗𝒗1 ,
• 𝒗𝒗2 = 𝑐𝑐 𝒗𝒗2
• 𝒗𝒗1 𝒗𝒗2 = 𝑐𝑐 𝒗𝒗1 𝒗𝒗1 = 𝑐𝑐 𝒗𝒗1 2
• 𝒗𝒗1 � 𝒗𝒗2 = 𝒗𝒗1 � 𝑐𝑐𝒗𝒗1 = 𝑐𝑐 𝒗𝒗1 2
• If 𝒗𝒗2 = 𝑐𝑐𝒗𝒗1 , equality holds. 𝒗𝒗1 𝒗𝒗2 = 𝒗𝒗1 � 𝒗𝒗2

6
 Triangular Inequality
• Triangular inequality
• 𝒗𝒗1 + 𝒗𝒗2 ≤ 𝒗𝒗1 + 𝒗𝒗2
• Equality if 𝒗𝒗2 = 𝑎𝑎𝒗𝒗1 . 𝑎𝑎 = positive real scalar.
• 𝒗𝒗1 + 𝒗𝒗2 2 = 𝒗𝒗1 + 𝒗𝒗2 � 𝒗𝒗1 + 𝒗𝒗2
• = 𝒗𝒗1 � 𝒗𝒗1 +2 𝒗𝒗1 � 𝒗𝒗2 + 𝒗𝒗2 � 𝒗𝒗2 = 𝒗𝒗1 2 + 2 𝒗𝒗1 � 𝒗𝒗2 + 𝒗𝒗2 2

• 𝒗𝒗1 � 𝒗𝒗2 ≤ 𝒗𝒗1 � 𝒗𝒗2 ≤ 𝒗𝒗1 𝒗𝒗2

• 𝒗𝒗1 + 𝒗𝒗2 ≤ 𝒗𝒗1 + 𝒗𝒗2

7
 Triangular Inequality
• If 𝒗𝒗2 = 𝑎𝑎𝒗𝒗1 ,
• 𝒗𝒗1 + 𝒗𝒗2 = 𝒗𝒗1 + 𝑎𝑎𝒗𝒗1 = (1 + 𝑎𝑎) 𝒗𝒗1 ||v1 + v2||
• 𝒗𝒗1 + 𝒗𝒗2 = 𝒗𝒗1 + 𝑎𝑎 𝒗𝒗1 = (1 + 𝑎𝑎) 𝒗𝒗1 ||v2||

• Thus, ||v1||
• 𝒗𝒗1 + 𝒗𝒗2 = 𝒗𝒗1 + 𝒗𝒗2

||v1 + v2||

||v2||
||v1||

8
 Gram-Schmidt Orthogonalization for Vectors
• Gram-Schmidt orthogonalization is a procedure which takes a set of
nonorthogonal set of linearly independent vectors and constructs a
set of orthogonal vectors.
• A set of n-dimensional vectors 𝒗𝒗𝑖𝑖 , 1 ≤ 𝑖𝑖 ≤ 𝑚𝑚 is given.
• 1. Select a vector 𝒗𝒗1 arbitrarily. Normalize its length.
𝒗𝒗1
• 𝒖𝒖1 =
𝒗𝒗1
• 2. Select a vector 𝒗𝒗2 . Subtract the projection of 𝒗𝒗2 𝒗𝒗2
onto 𝒖𝒖1 . 𝒖𝒖′2

• 𝒖𝒖′2 = 𝒗𝒗2 − 𝒗𝒗2 � 𝒖𝒖1 𝒖𝒖1 𝒗𝒗2 � 𝒖𝒖1 𝒖𝒖1

9
 Gram-Schmidt Orthogonalization for Vectors
• Normalize 𝒖𝒖′2 to unit length.
𝒖𝒖′2
• 𝒖𝒖2 =
𝒖𝒖′2
• 3. Select a vector 𝒗𝒗2 . Subtract the projection of 𝒗𝒗2 onto 𝒖𝒖1 and 𝒖𝒖2 .
• 𝒖𝒖′3 = 𝒗𝒗3 − 𝒗𝒗3 � 𝒖𝒖1 𝒖𝒖1 − 𝒗𝒗3 � 𝒖𝒖2 𝒖𝒖2
• Normalize 𝒖𝒖′3 to unit length.
𝒖𝒖′3
• 𝒖𝒖3 =
𝒖𝒖′3
• Construct n1 orthonormal vectors. n1 ≤ n. If m < n, n1 ≤ m.

10
 Frobenius Norm
• The Frobenius norm, sometimes also called the Euclidean norm, is a
matrix norm of am mxn matrix A defined as the square root of the
sum of the absolute squares of its elements,
2
• 𝐴𝐴 𝐹𝐹 = ∑𝑚𝑚 ∑ 𝑛𝑛
𝑖𝑖=1 𝑗𝑗=1 𝑎𝑎𝑖𝑖𝑖𝑖
• The Frobenius norm is equal to the square root of the matrix trace of
𝐴𝐴𝐴𝐴𝐻𝐻 , where 𝐴𝐴𝐻𝐻 is the conjugate transpose (Hermitian) of A.
• 𝐴𝐴 𝐹𝐹 = 𝑇𝑇𝑇𝑇 𝐴𝐴𝐴𝐴𝐻𝐻
• https://mathworld.wolfram.com/FrobeniusNorm.html

11
 EXAMPLE
• The three vectors v1, v2, and v3 are given by
1 1 −2
• 𝒗𝒗1 = 2 , 𝒗𝒗2 = −1 , 𝒗𝒗3 = 1
3 2 4
• (a) Find the four vectors u1, u2, and u3 that are orthonormal using Gram-
Schmidt orthogonalization procedure for vectors.
• (b) Find a synthesis matrix S = [u1, u2, u3] and find the analysis matrix A =
S H.
2
• (c) Find the coefficients b when a vector 𝒗𝒗4 = −5 is represented as a
3
linear combination of u1, u2, and u3.

12
 EXAMPLE
• Reconstruct 𝒗𝒗4 from the coefficients b and find the error between the
original 𝒗𝒗4 and the reconstructed 𝒗𝒗4 .
• (d) A 3×3 image given by X=[4 5 6;5 7 6;4 6 5]; is transformed by the
orthonormal basis functions. Find Y and Z defined by
Y = A*X %Column transform
Z = A*Y’ %Row transform
Reconstruct X from the coefficients Z and find the Frobenius norm of error
matrix between the original X and the reconstructed X.
Ya=S*Z
Xa=S*Ya'

13
 EXAMPLE
• (e) Apply threshold of 2 to Z and take inverse transform.
Z1=Z.*(abs(Z)>=2)
Y1=S*Z1
X1=S*Y1’
Find the Frobenius norm of error matrix.

14
clear all; b=A*s4
s1=[1 2 3].'; s4b=S*b
s2=[1 -1 2].'; Es4=s4-s4b
s3=[-2 1 4].'; X=[4 5 6;5 7 6;4 6 5]
s4=[2 -5 3].'; Y=A*X
u1=s1/norm(s1) Z=A*Y'
up2=s2-(s2'*u1)*u1 Ya=S*Z
Xa=S*Ya'
u2=up2/norm(up2)
Ex=X-Xa
up3=s3-(s3'*u1)*u1-(s3'*u2)*u2
FnormEx=norm(Ex,'fro')
u3=up3/norm(up3)
Z1=Z.*(abs(Z)>=2)
S=[u1,u2,u3] Y1=S*Z1
E=norm(eye(3)-S'*S,'fro’) X1=S*Y1'
Ea=norm(eye(3)-S'*S) E1=X-X1
A=S' FnormE1=norm(E1,'fro')

15
 Signal Space Concepts
• Continuous time signals instead of vectors. Continuous time signals
are defined on the interval [a, b].
• Inner product
𝑏𝑏
• 𝑥𝑥1 𝑡𝑡 , 𝑥𝑥2 (𝑡𝑡) = ∫𝑎𝑎 𝑥𝑥1 (𝑡𝑡)𝑥𝑥2∗ (𝑡𝑡) 𝑑𝑑𝑑𝑑
• Norm
𝑏𝑏
• 𝑥𝑥(𝑡𝑡) = ∫𝑎𝑎 𝑥𝑥(𝑡𝑡) 2 𝑑𝑑𝑑𝑑

16
 Cauchy-Schwarz Inequality
2
• ∫ 𝑓𝑓 𝑥𝑥 𝑔𝑔∗ 𝑥𝑥 𝑑𝑑𝑑𝑑 ≤ ∫ 𝑓𝑓 𝑥𝑥 2 𝑑𝑑𝑑𝑑
∫ 𝑔𝑔(𝑥𝑥) 2 𝑑𝑑𝑑𝑑
• Let 𝜓𝜓(𝑥𝑥) be a complex function and 𝜆𝜆 a complex constant such that
• 𝜓𝜓 𝑥𝑥 = 𝑓𝑓 𝑥𝑥 + 𝜆𝜆𝑔𝑔 𝑥𝑥 for some f and g.
• ∫ 𝜓𝜓 𝑥𝑥 𝜓𝜓 ∗ 𝑥𝑥 𝑑𝑑𝑑𝑑 = ∫ 𝜓𝜓 𝑥𝑥 𝑒𝑒 𝑗𝑗∠𝜓𝜓(𝑥𝑥) 𝜓𝜓 𝑥𝑥 𝑒𝑒 −𝑗𝑗∠𝜓𝜓(𝑥𝑥) 𝑑𝑑𝑑𝑑
2 𝑑𝑑𝑑𝑑
• = ∫ 𝜓𝜓 𝑥𝑥 ≥0
• ∫ 𝜓𝜓 𝑥𝑥 𝜓𝜓 ∗ 𝑥𝑥 𝑑𝑑𝑑𝑑 = ∫ 𝑓𝑓 𝑥𝑥 + 𝜆𝜆𝜆𝜆 𝑥𝑥 𝑓𝑓 ∗ 𝑥𝑥 + 𝜆𝜆∗ 𝑔𝑔∗ (𝑥𝑥) 𝑑𝑑𝑥𝑥
• = ∫ 𝑓𝑓 𝑥𝑥 𝑓𝑓 ∗ 𝑥𝑥 𝑑𝑑𝑑𝑑 + 𝜆𝜆 ∫ 𝑓𝑓 ∗ 𝑥𝑥 𝑔𝑔(𝑥𝑥)𝑑𝑑𝑑𝑑 + 𝜆𝜆∗ ∫ 𝑓𝑓 𝑥𝑥 𝑔𝑔∗ 𝑥𝑥 𝑑𝑑𝑑𝑑
• +𝜆𝜆𝜆𝜆∗ ∫ 𝑔𝑔 𝑥𝑥 𝑔𝑔∗ 𝑥𝑥 𝑑𝑑𝑑𝑑 ≥ 0

17
 Cauchy-Schwarz Inequality
• Equality holds when 𝜓𝜓 𝑥𝑥 = 0.
• 𝑓𝑓, 𝑓𝑓 ∗ = ∫ 𝑓𝑓 𝑥𝑥 𝑓𝑓 ∗ 𝑥𝑥 𝑑𝑑𝑑𝑑 = ∫ 𝑓𝑓 ∗ 𝑥𝑥 𝑓𝑓 𝑥𝑥 𝑑𝑑𝑑𝑑 = 𝑓𝑓 ∗ , 𝑓𝑓
• 𝑓𝑓, 𝑓𝑓 ∗ + 𝜆𝜆 𝑓𝑓 ∗ , 𝑔𝑔 + 𝜆𝜆∗ 𝑓𝑓, 𝑔𝑔∗ + 𝜆𝜆𝜆𝜆∗ 𝑔𝑔, 𝑔𝑔∗ ≥ 0 (1)
• Define
𝑓𝑓,𝑔𝑔∗ ∗ 𝑓𝑓∗ ,𝑔𝑔
• 𝜆𝜆 = − , 𝜆𝜆 = − (2)
𝑔𝑔,𝑔𝑔∗ 𝑔𝑔,𝑔𝑔∗
• Multiply (1) by 𝑔𝑔, 𝑔𝑔∗ and substitute (2).
• 𝑓𝑓, 𝑓𝑓 ∗ 𝑔𝑔, 𝑔𝑔∗ − 𝑓𝑓, 𝑔𝑔∗ 𝑓𝑓 ∗ , 𝑔𝑔 − 𝑓𝑓 ∗ , 𝑔𝑔 𝑓𝑓, 𝑔𝑔∗ + 𝑓𝑓, 𝑔𝑔∗ 𝑓𝑓 ∗ , 𝑔𝑔 ≥ 0
• 𝑓𝑓, 𝑓𝑓 ∗ 𝑔𝑔, 𝑔𝑔∗ ≥ 𝑓𝑓 ∗ , 𝑔𝑔 𝑓𝑓, 𝑔𝑔∗ = 𝑓𝑓, 𝑔𝑔∗ 𝑓𝑓 ∗ , 𝑔𝑔 = 𝑓𝑓, 𝑔𝑔∗ 2
18
 Cauchy-Schwarz Inequality
• 𝑓𝑓, 𝑔𝑔∗ 2 ≤ 𝑓𝑓, 𝑓𝑓 ∗ 𝑔𝑔, 𝑔𝑔∗
2
• ∫ 𝑓𝑓 𝑥𝑥 𝑔𝑔∗ 𝑥𝑥 𝑑𝑑𝑑𝑑 ≤ ∫ 𝑓𝑓 𝑥𝑥 2 𝑑𝑑𝑑𝑑
∫ 𝑔𝑔 𝑥𝑥 2 𝑑𝑑𝑑𝑑

• Equality holds when 𝑓𝑓 𝑥𝑥 = −𝜆𝜆𝜆𝜆 𝑥𝑥 .

19
 Representation of Arbitrary Signal by
Orthogonal Functions
• An arbitrary signal f(t) can be approximated over the interval (t0, t1)
by a linear combination of an arbitrary set of orthogonal functions
[φ0(t), φ1(t), ......., φN−1(t)] if the energy of the signal over the interval
(t0, t1) is finite, i.e.,
𝑡𝑡1
• 𝐸𝐸𝑓𝑓 = ∫𝑡𝑡 𝑓𝑓(𝑡𝑡) 2 𝑑𝑑𝑑𝑑 < ∞
0
̂ be the approximation. Then,
• Let 𝑓𝑓(𝑡𝑡)
• 𝑓𝑓̂ 𝑡𝑡 = 𝑎𝑎0 𝜙𝜙0 𝑡𝑡 + 𝑎𝑎1 𝜙𝜙1 𝑡𝑡 + ⋯ + 𝑎𝑎𝑁𝑁−1 𝜙𝜙𝑁𝑁−1 𝑡𝑡 = ∑𝑁𝑁−1
𝑛𝑛=0 𝑎𝑎𝑛𝑛 𝜙𝜙𝑛𝑛 𝑡𝑡
• The set of orthogonal functions [φ0(t), φ1(t), ......., φN−1(t)] is called
basis functions because they form the basis for the representation.
20
 Representation of Arbitrary Signal by
Orthogonal Functions
• The difference between the original signal f(t) and its approximation
̂ is defined as the error signal:
𝑓𝑓(𝑡𝑡)
• 𝑒𝑒 𝑡𝑡 = 𝑓𝑓 𝑡𝑡 − 𝑓𝑓̂ 𝑡𝑡 = 𝑓𝑓 𝑡𝑡 − ∑𝑁𝑁−1𝑛𝑛=0 𝑎𝑎𝑛𝑛 𝜙𝜙𝑛𝑛 𝑡𝑡
• The coefficients a0, a1, ......., aN−1 in the linear combination are
determined to minimize the energy of the error signal e(t) over the
interval (t0, t1) given by
𝑡𝑡1 𝑡𝑡1 2
• 𝐸𝐸𝑒𝑒 = ∫𝑡𝑡 𝑒𝑒(𝑡𝑡) 2 𝑑𝑑𝑑𝑑 = ∫𝑡𝑡 𝑓𝑓 𝑡𝑡 − ∑𝑁𝑁−1
𝑛𝑛=0 𝑎𝑎𝑛𝑛 𝜙𝜙𝑛𝑛 𝑡𝑡 𝑑𝑑𝑑𝑑
0 0
𝑡𝑡1
•= ∫𝑡𝑡 𝑓𝑓 𝑡𝑡 − ∑𝑁𝑁−1
𝑛𝑛=0 𝑎𝑎𝑛𝑛 𝜙𝜙𝑛𝑛 𝑡𝑡 𝑓𝑓 𝑡𝑡 − ∑𝑁𝑁−1
𝑛𝑛=0 𝑎𝑎𝑛𝑛 𝜙𝜙𝑛𝑛 𝑡𝑡
∗ 𝑑𝑑𝑑𝑑
0

21
 Representation of Arbitrary Signal by
Orthogonal Functions
𝑡𝑡1
• = ∫𝑡𝑡 𝑓𝑓 𝑡𝑡 − ∑𝑁𝑁−1 𝑎𝑎
𝑛𝑛=0 𝑛𝑛 𝑛𝑛𝜙𝜙 𝑡𝑡 𝑓𝑓 ∗ 𝑡𝑡 − ∑𝑁𝑁−1 𝑎𝑎 ∗ 𝜙𝜙 ∗ 𝑡𝑡 𝑑𝑑𝑑𝑑
𝑛𝑛=0 𝑛𝑛 𝑛𝑛
0
𝑡𝑡1 𝑡𝑡1
• = ∫𝑡𝑡 𝑓𝑓(𝑡𝑡)𝑓𝑓 (𝑡𝑡) 𝑑𝑑𝑑𝑑 − ∫𝑡𝑡 𝑓𝑓(𝑡𝑡) ∑𝑁𝑁−1
∗
𝑎𝑎 ∗ 𝜙𝜙 ∗ 𝑡𝑡 𝑑𝑑𝑑𝑑
𝑛𝑛=0 𝑛𝑛 𝑛𝑛
0 0
𝑁𝑁−1 𝑡𝑡1 ∗ 𝑁𝑁−1 𝑁𝑁−1 ∗ 𝑡𝑡1 ∗ 𝑡𝑡 𝑑𝑑𝑑𝑑
• − ∑𝑛𝑛=0 𝑎𝑎𝑛𝑛 ∫𝑡𝑡 𝑓𝑓 𝑡𝑡 𝜙𝜙𝑛𝑛 𝑡𝑡 𝑑𝑑𝑑𝑑 + ∑𝑛𝑛=0 ∑𝑚𝑚=0 𝑎𝑎𝑛𝑛 𝑎𝑎𝑚𝑚 ∫𝑡𝑡 𝜙𝜙𝑛𝑛 𝑡𝑡 𝜙𝜙𝑚𝑚
0 0
𝑡𝑡1 𝑁𝑁−1 𝑡𝑡1
• 2
= ∫𝑡𝑡 𝑓𝑓(𝑡𝑡) 𝑑𝑑𝑑𝑑 − ∑𝑛𝑛=0 𝑎𝑎𝑛𝑛 ∫𝑡𝑡 𝑓𝑓(𝑡𝑡) 𝜙𝜙𝑛𝑛∗ 𝑡𝑡 𝑑𝑑𝑑𝑑
∗
0 0
𝑡𝑡1 ∗
• − ∑𝑛𝑛=0 𝑎𝑎𝑛𝑛 ∫𝑡𝑡 𝑓𝑓 𝑡𝑡 𝜙𝜙𝑛𝑛 𝑡𝑡 𝑑𝑑𝑑𝑑 + ∑𝑁𝑁−1
𝑁𝑁−1
𝜇𝜇 𝑎𝑎
𝑛𝑛=0 𝑛𝑛 𝑛𝑛 𝑛𝑛 𝑎𝑎 ∗
0

22
 Representation of Arbitrary Signal by
Orthogonal Functions
• 𝑧𝑧𝑛𝑛 = 𝑥𝑥𝑛𝑛 + 𝑗𝑗𝑦𝑦𝑛𝑛 , 𝑧𝑧𝑛𝑛∗ = 𝑥𝑥𝑛𝑛 − 𝑗𝑗𝑦𝑦𝑛𝑛
• 𝑧𝑧𝑛𝑛 𝑧𝑧𝑛𝑛∗ = 𝑥𝑥𝑛𝑛 + 𝑗𝑗𝑦𝑦𝑛𝑛 𝑥𝑥𝑛𝑛 − 𝑗𝑗𝑦𝑦𝑛𝑛 = 𝑥𝑥𝑛𝑛2 + 𝑦𝑦𝑛𝑛2
∗
𝜕𝜕 𝑧𝑧𝑛𝑛 𝑧𝑧𝑛𝑛 𝜕𝜕 𝑧𝑧𝑛𝑛 𝑧𝑧𝑛𝑛∗ 𝜕𝜕 𝑧𝑧𝑛𝑛 𝑧𝑧𝑛𝑛∗
• = + 𝑗𝑗 = 2𝑥𝑥𝑛𝑛 + 𝑗𝑗𝑗𝑦𝑦𝑛𝑛 = 2𝑧𝑧𝑛𝑛
𝜕𝜕𝑧𝑧𝑛𝑛 𝜕𝜕𝑥𝑥𝑛𝑛 𝜕𝜕𝑦𝑦𝑛𝑛
𝜕𝜕 𝑧𝑧𝑛𝑛 𝜕𝜕 𝑧𝑧𝑛𝑛 𝜕𝜕 𝑧𝑧𝑛𝑛
• = + 𝑗𝑗 = 1 + 𝑗𝑗 × 𝑗𝑗 = 0
𝜕𝜕𝑧𝑧𝑛𝑛 𝜕𝜕𝑥𝑥𝑛𝑛 𝜕𝜕𝑦𝑦𝑛𝑛
∗
𝜕𝜕 𝑧𝑧𝑛𝑛 ∗
𝜕𝜕 𝑧𝑧𝑛𝑛 ∗
𝜕𝜕 𝑧𝑧𝑛𝑛
• = + 𝑗𝑗 = 1 + 𝑗𝑗 × (−𝑗𝑗) = 2
𝜕𝜕𝑧𝑧𝑛𝑛 𝜕𝜕𝑥𝑥𝑛𝑛 𝜕𝜕𝑦𝑦𝑛𝑛

23
 Representation of Arbitrary Signal by
Orthogonal Functions
𝜕𝜕𝐸𝐸𝑒𝑒 𝑡𝑡1
• =0− 2 ∫𝑡𝑡 𝑓𝑓 𝑡𝑡 𝜙𝜙𝑛𝑛∗ 𝑡𝑡 𝑑𝑑𝑑𝑑 − 0 + 2𝜇𝜇𝑛𝑛 𝑎𝑎𝑛𝑛 = 0
𝜕𝜕𝑎𝑎𝑛𝑛 0
1 𝑡𝑡1 ∗ 𝑡𝑡 𝑑𝑑𝑑𝑑
• 𝑎𝑎𝑛𝑛 = ∫ 𝑓𝑓 𝑡𝑡 𝜙𝜙𝑛𝑛
𝜇𝜇𝑛𝑛 𝑡𝑡0
1 𝑡𝑡1 ∗
• 𝑎𝑎𝑛𝑛∗ = ∫𝑡𝑡
𝑓𝑓 𝑡𝑡 𝜙𝜙𝑛𝑛 𝑡𝑡 𝑑𝑑𝑑𝑑
𝜇𝜇𝑛𝑛 0
𝑡𝑡1
• 𝐸𝐸𝑒𝑒 = ∫𝑡𝑡 𝑓𝑓(𝑡𝑡) 2 𝑑𝑑𝑑𝑑 − ∑𝑁𝑁−1 𝜇𝜇 𝑎𝑎 ∗ 𝑎𝑎 − ∑𝑁𝑁−1 𝜇𝜇 𝑎𝑎 𝑎𝑎 ∗ + ∑𝑁𝑁−1 𝜇𝜇 𝑎𝑎 𝑎𝑎 ∗
𝑛𝑛=0 𝑛𝑛 𝑛𝑛 𝑛𝑛 𝑛𝑛=0 𝑛𝑛 𝑛𝑛 𝑛𝑛 𝑛𝑛=0 𝑛𝑛 𝑛𝑛 𝑛𝑛
0
𝑡𝑡1 𝑡𝑡1 𝑡𝑡1 2
• 𝐸𝐸𝑒𝑒 = 2 𝑁𝑁−1 2 2
∫𝑡𝑡 𝑓𝑓(𝑡𝑡) 𝑑𝑑𝑑𝑑 − ∑𝑛𝑛=0 𝜇𝜇𝑛𝑛 𝑎𝑎𝑛𝑛 = ∫𝑡𝑡 𝑓𝑓 𝑡𝑡 𝑑𝑑𝑑𝑑 − ∫𝑡𝑡 𝑓𝑓 (𝑡𝑡) 𝑑𝑑𝑑𝑑 ̂
0 0 0
𝑡𝑡1 𝑡𝑡1 𝑁𝑁−1
• 𝐸𝐸𝑓𝑓̂ = ̂ ̂
∫𝑡𝑡 𝑓𝑓 𝑡𝑡 𝑓𝑓 𝑡𝑡 𝑑𝑑𝑑𝑑 = ∫𝑡𝑡 ∑𝑛𝑛=0 𝑎𝑎𝑛𝑛 𝜙𝜙𝑛𝑛 𝑡𝑡 ∑𝑁𝑁−1
∗
𝑎𝑎 ∗ 𝜙𝜙 ∗ 𝑡𝑡 dt
𝑚𝑚=0 𝑚𝑚 𝑚𝑚
0 0

24
 Representation of Arbitrary Signal by
Orthogonal Functions
𝑡𝑡
• 𝐸𝐸𝑓𝑓̂ = ∑𝑁𝑁−1 ∑ 𝑁𝑁−1
𝑎𝑎 𝑎𝑎
𝑛𝑛=0 𝑚𝑚=0 𝑛𝑛 𝑚𝑚 𝑡𝑡
∗
∫
1
𝜙𝜙𝑛𝑛 𝑡𝑡 𝜙𝜙 ∗ 𝑡𝑡 𝑑𝑑𝑑𝑑 = ∑𝑁𝑁−1 𝜇𝜇 𝑎𝑎
𝑚𝑚 𝑛𝑛=0 𝑛𝑛 𝑛𝑛
2
0
• As N→∞, Ee→0,
∞ 2 𝑡𝑡1
• ∑𝑛𝑛=0 𝜇𝜇𝑛𝑛 𝑎𝑎𝑛𝑛 = ∫𝑡𝑡 𝑓𝑓(𝑡𝑡) 2 𝑑𝑑𝑑𝑑
0

25
 Exponential Fourier Series
• A set of complex exponential functions
• 𝑒𝑒 𝑖𝑖𝑖𝑖𝜔𝜔0 𝑡𝑡 , 𝑛𝑛 = 0, ±1, ±2, ±3, …
• are orthogonal and complete on the interval (t0, t0 + T0), where 𝜔𝜔0 =
2𝜋𝜋
= 2𝜋𝜋𝑓𝑓0 is the fundamental frequency in rad/s and f0 = 1/T0 is the
𝑇𝑇0
fundamental frequency in Hz and t0 is arbitrary.
• 𝑓𝑓 𝑡𝑡 = ∑∞ 𝐹𝐹
𝑛𝑛=−∞ 𝑛𝑛 𝑒𝑒 𝑗𝑗𝑗𝑗𝜔𝜔0 𝑡𝑡 , 𝑡𝑡0 ≤ 𝑡𝑡 ≤ 𝑡𝑡0 + 𝑇𝑇0
𝑡𝑡 +𝑇𝑇0
∫𝑡𝑡 0 𝑓𝑓(𝑡𝑡)𝑒𝑒 −𝑗𝑗𝑗𝑗𝜔𝜔0 𝑡𝑡 𝑑𝑑𝑑𝑑 1 𝑡𝑡0 +𝑇𝑇0
• 𝐹𝐹𝑛𝑛 = 0
= 𝑓𝑓(𝑡𝑡)𝑒𝑒 −𝑗𝑗𝑗𝑗𝜔𝜔0 𝑡𝑡 𝑑𝑑𝑑𝑑
𝑡𝑡 +𝑇𝑇 ∫
𝑇𝑇0 𝑡𝑡0
∫𝑡𝑡 0 0 𝑒𝑒 𝑗𝑗𝑛𝑛𝜔𝜔0 𝑡𝑡 𝑒𝑒 −𝑗𝑗𝑗𝑗𝜔𝜔0 𝑡𝑡 𝑑𝑑𝑑𝑑
0

26
 Gram-Schmidt Orthogonalization for
Waveforms
• Given a set of finite energy waveforms 𝑠𝑠𝑖𝑖 𝑡𝑡 , 𝑖𝑖 = 1,2, … , 𝑀𝑀 ,
construct a set of orthonormal waveforms.
𝑠𝑠1 (𝑡𝑡) ∞ 2
• 𝑓𝑓1 𝑡𝑡 = , 𝐸𝐸1 = ∫−∞ 𝑠𝑠1 𝑡𝑡 𝑑𝑑𝑑𝑑 = Energy of 𝑠𝑠1 (𝑡𝑡).
𝐸𝐸1
• 𝑓𝑓1 𝑡𝑡 has unit energy.
• Compute Projection of 𝑠𝑠2 (𝑡𝑡) onto 𝑓𝑓1 𝑡𝑡 .
∞
• 𝐶𝐶12 = ∫−∞ 𝑠𝑠2 𝑡𝑡 𝑓𝑓1 𝑡𝑡 𝑑𝑑𝑑𝑑
• Subtract 𝐶𝐶12 𝑓𝑓1 𝑡𝑡 from 𝑠𝑠2 (𝑡𝑡).
• 𝑓𝑓2′ 𝑡𝑡 = 𝑠𝑠2 𝑡𝑡 − 𝐶𝐶12 𝑓𝑓1 𝑡𝑡
27
 Gram-Schmidt Orthogonalization for
Waveforms
• Normalize.
𝑓𝑓2′ 𝑡𝑡 ∞
• 𝑓𝑓2 𝑡𝑡 =
𝐸𝐸2
, 𝐸𝐸2 = ∫−∞ 𝑓𝑓2′2 𝑡𝑡 𝑑𝑑𝑑𝑑
∞
• 𝐶𝐶13 = ∫−∞ 𝑠𝑠3 𝑡𝑡 𝑓𝑓1 𝑡𝑡 𝑑𝑑𝑑𝑑
∞
• 𝐶𝐶23 = ∫−∞ 𝑠𝑠3 𝑡𝑡 𝑓𝑓2 𝑡𝑡 𝑑𝑑𝑑𝑑
• 𝑓𝑓3′ 𝑡𝑡 = 𝑠𝑠3 𝑡𝑡 − 𝐶𝐶13 𝑓𝑓1 𝑡𝑡 − 𝐶𝐶23 𝑓𝑓2 𝑡𝑡
𝑓𝑓3′ 𝑡𝑡 ∞
• 𝑓𝑓3 𝑡𝑡 =
𝐸𝐸3
, 𝐸𝐸3 = ∫−∞ 𝑓𝑓3′2 𝑡𝑡 𝑑𝑑𝑑𝑑

28
 Gram-Schmidt Orthogonalization for
Waveforms
∞
• 𝐶𝐶14 = ∫−∞ 𝑠𝑠4 𝑡𝑡 𝑓𝑓1 𝑡𝑡 𝑑𝑑𝑑𝑑
∞
• 𝐶𝐶24 = ∫−∞ 𝑠𝑠4 𝑡𝑡 𝑓𝑓2 𝑡𝑡 𝑑𝑑𝑑𝑑
∞
• 𝐶𝐶34 = ∫−∞ 𝑠𝑠4 𝑡𝑡 𝑓𝑓3 𝑡𝑡 𝑑𝑑𝑑𝑑
• 𝑓𝑓4′ 𝑡𝑡 = 𝑠𝑠4 𝑡𝑡 − 𝐶𝐶14 𝑓𝑓1 𝑡𝑡 − 𝐶𝐶24 𝑓𝑓2 𝑡𝑡 − 𝐶𝐶34 𝑓𝑓3 𝑡𝑡
𝑓𝑓4′ 𝑡𝑡 ∞
• 𝑓𝑓4 𝑡𝑡 =
𝐸𝐸4
, 𝐸𝐸4 = ∫−∞ 𝑓𝑓4′2 𝑡𝑡 𝑑𝑑𝑑𝑑
• In general, 𝑓𝑓𝑘𝑘′ 𝑡𝑡 = 𝑠𝑠𝑘𝑘 𝑡𝑡 − ∑𝑘𝑘−1
𝑖𝑖=1 𝐶𝐶𝑖𝑖𝑖𝑖 𝑓𝑓𝑖𝑖 𝑡𝑡
∞
• 𝐶𝐶𝑖𝑖𝑖𝑖 = ∫−∞ 𝑠𝑠𝑘𝑘 𝑡𝑡 𝑓𝑓𝑖𝑖 𝑡𝑡 𝑑𝑑𝑑𝑑 , i = 1, 2, 3, … , k − 1
29
 EXAMPLE
• Find a set of orthonormal waveforms from s1(t), s2(t), s3(t), s4(t).
2 2

1 1

s (t)
0
s (t)

3
1

-1 -1

-2 -2
-2 -1 0 1 2 3 4 5 -2 -1 0 1 2 3 4 5

t t

2 2

1 1

s (t)
0
s (t)

4
2

-1 -1

-2 -2
-2 -1 0 1 2 3 4 5 -2 -1 0 1 2 3 4 5
t t

30
 EXAMPLE
1
2 2 2 2 𝑠𝑠1 (𝑡𝑡) ,0 ≤ 𝑡𝑡 ≤ 2
• 𝐸𝐸1 = ∫0 𝑠𝑠1 𝑡𝑡 𝑑𝑑𝑑𝑑 = ∫0 1 𝑑𝑑𝑑𝑑 = 2, 𝑓𝑓1 𝑡𝑡 = =� 2
2
0, 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒
2 1 1
• 𝐶𝐶12 = ∫0 𝑠𝑠2 𝑡𝑡 𝑓𝑓1 𝑡𝑡 𝑑𝑑𝑑𝑑 = 1 × −1× =0
2 2
2 2
• 𝑓𝑓2′ 𝑡𝑡 = 𝑠𝑠2 𝑡𝑡 − 𝐶𝐶12 𝑓𝑓1 𝑡𝑡 = 𝑠𝑠2 𝑡𝑡 , 𝐸𝐸2 = ∫0 1 𝑑𝑑𝑑𝑑 =2
1
,0 ≤ 𝑡𝑡 < 1
2
𝑠𝑠2 (𝑡𝑡)
• 𝑓𝑓2 𝑡𝑡 = = −1
,1 ≤ 𝑡𝑡 < 2
2 2
0, 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒
31
 EXAMPLE
• Orthonormal waveforms
2
2

1 1

f (t)
0
f (t)

3
1

-1 -1

-2 -2
-2 -1 0 1 2 3 4 5 -2 -1 0 1 2 3 4 5
t t

0
f (t)
2

-1

-2
-2 -1 0 1 2 3 4 5
t

32
 EXAMPLE
2 2 1
• 𝐶𝐶13 = ∫0 𝑠𝑠3 𝑡𝑡 𝑓𝑓1 𝑡𝑡 𝑑𝑑𝑑𝑑 = ∫0 1 × 𝑑𝑑𝑑𝑑 = 2
2
2 1 −1
• 𝐶𝐶23 = ∫0 𝑠𝑠3 𝑡𝑡 𝑓𝑓2 𝑡𝑡 𝑑𝑑𝑑𝑑 = 1 × +1× =0
2 2
−1, 2 ≤ 𝑡𝑡 < 3
•𝑓𝑓3′ 𝑡𝑡 = 𝑠𝑠3 𝑡𝑡 − 𝐶𝐶13 𝑓𝑓1 𝑡𝑡 − 𝐶𝐶23 𝑓𝑓2 𝑡𝑡 = �
0, 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒
3 2 𝑓𝑓3′ 𝑡𝑡 −1, 2 ≤ 𝑡𝑡 < 3
• 𝐸𝐸3 = ∫2 1 𝑑𝑑𝑑𝑑 = 1. 𝑓𝑓3 𝑡𝑡 = =�
𝐸𝐸3 0, 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒
• 𝑓𝑓3′ 𝑡𝑡 = 𝐸𝐸3 𝑓𝑓3 𝑡𝑡 = 𝑓𝑓3 𝑡𝑡
• 𝑓𝑓3 𝑡𝑡 = 𝑠𝑠3 𝑡𝑡 − 2𝑓𝑓1 𝑡𝑡
33
 EXAMPLE
2 2 1
• 𝐶𝐶14 = ∫0 𝑠𝑠4 𝑡𝑡 𝑓𝑓1 𝑡𝑡 𝑑𝑑𝑑𝑑 = ∫0 −1 × 𝑑𝑑𝑑𝑑 =− 2
2
2 1 1 2 −1
• 𝐶𝐶24 = ∫0 𝑠𝑠4 𝑡𝑡 𝑓𝑓2 𝑡𝑡 𝑑𝑑𝑑𝑑 = ∫0 −1 × 𝑑𝑑𝑑𝑑 + ∫1 −1 × 𝑑𝑑𝑑𝑑 =0
2 2
3 3
• 𝐶𝐶34 = ∫2 𝑠𝑠4 𝑡𝑡 𝑓𝑓3 𝑡𝑡 𝑑𝑑𝑑𝑑 = ∫2 −1 × (−1)𝑑𝑑𝑑𝑑 = 1
• 𝑓𝑓4′ 𝑡𝑡 = 𝑠𝑠4 𝑡𝑡 − 𝐶𝐶14 𝑓𝑓1 𝑡𝑡 − 𝐶𝐶24 𝑓𝑓2 𝑡𝑡 − 𝐶𝐶34 𝑓𝑓3 𝑡𝑡
• = 𝑠𝑠4 𝑡𝑡 + 2𝑓𝑓1 𝑡𝑡 − 𝑓𝑓3 𝑡𝑡 = 0
• 𝑠𝑠4 𝑡𝑡 = − 2𝑓𝑓1 𝑡𝑡 + 𝑓𝑓3 𝑡𝑡
• 𝑠𝑠4 𝑡𝑡 is a linear combination of 𝑓𝑓1 𝑡𝑡 and 𝑓𝑓3 𝑡𝑡 . 𝑠𝑠4 𝑡𝑡 is not linearly
independent to 𝑓𝑓1 𝑡𝑡 , 𝑓𝑓2 𝑡𝑡 , and 𝑓𝑓3 𝑡𝑡 .
34
 EXAMPLE
• Waveform Vector Representation
• 𝑠𝑠1 𝑡𝑡 = 2𝑓𝑓1 (𝑡𝑡) 𝒔𝒔1 = [ 2, 0,0]
• 𝑠𝑠2 𝑡𝑡 = 2𝑓𝑓2 (𝑡𝑡) 𝒔𝒔2 = [0, 2, 0]
• 𝑠𝑠3 𝑡𝑡 = 2𝑓𝑓1 𝑡𝑡 + 𝑓𝑓3 𝑡𝑡 𝒔𝒔3 = [ 2, 0,1]
• 𝑠𝑠4 𝑡𝑡 = − 2𝑓𝑓1 𝑡𝑡 + 𝑓𝑓3 𝑡𝑡 𝒔𝒔4 = [− 2, 0,1]

35