Control the World by Adding

and Multiplying
Andres Kwasinski
Department of Computer Engineering
Rochester Institute of Technology
2
What is this talk about?
It is about Digital Signal Processing (DSP): the technology of performing
operations on digital signals (signals represented as 1s and 0s).
DSP is used everywhere
3
Something Interesting
Many of the DSP algorithms are based on a
very simple operation:
y[n] =
M
X
k=N
c
k
x
n
[k]
Multiplications
Accumulate the
multiplications
A simple algorithm with many uses
4
Fourier Series

=
+
+
+ =
20
0
o
)
2
- t 1) cos((2n
) 1 2 (
2
2
1
x(t)
n
n
t
e
t
Periodic signals can be expressed as a sum of products
between weighting coefficients and sinusoids
5
Fourier Series

=
+
+
+ =
20
0
o
)
2
- t 1) cos((2n
) 1 2 (
2
2
1
x(t)
n
n
t
e
t
Instead of sinusoids, we use the more general expression
of complex exponentials:
) sin( ) cos( t j t e
t j
e e
e
+ =
We still use the same idea: periodic signals can be
expressed as a sum of products between weighting
coefficients and complex exponentials:
t k j
k
o
e k X t x
e

=
= ] [ ) (
Fourier Series
(Representation) of x(t)
6
Fourier Series
Each term of the sum multiplies a coefficient with a
complex exponential function:
Each complex exponentials function has a different frequency:
Two different complex exponential functions are orthogonal.
This gives rise to the idea of spectrum.
t k j
k
o
e k X t x
e

=
= ] [ ) (
Fourier Series
(Representation) of x(t)
(our reference algorithm)
Coefficients (weights) are complex numbers
k!
0
y[n] =
M
X
k=N
c
k
x
n
[k]
7
Spectrum
Spectrum: term originally used in the
17th century to describe the
decomposition of white light into its
colors (frequencies).
White light is made from the
composition of monochrome light.
Signals can be decomposed into separable (orthogonal,
independent) components determined by complex
exponentials (sinusoids) of different frequencies. The
result is the spectrum of the signal.
Signals are made from the composition of sinusoids.
8
Discrete Fourier Transform (DFT)
^
X[k] =
1
N
N1
X
n=0
x[n]e
jk-
o
n
; k = 0; : : : ; N 1
(our reference algorithm)
Same ideas can be extended to a digital signal
x[n]
y[n] =
M
X
k=N
c
k
x
n
[k]
0 50 100 150 200 250
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
time, n
x
[
n
]
0 5 10 15 20
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
time, n
90 100 110 120 130 140 150 160 170
0
5
10
15
20
|
X
[
k
]
|
9
DFT Application: OFDM
Data
IDFT
Network
DFT
Data
^
X[k] =
1
N
N1
X
n=0
x[n]e
j2kn=N
k = 0; : : : ; N 1
x[n] =
N1
X
k=0
^
X[k]e
j2kn=N
n = 0; : : : ; N 1
(our reference algorithm)
y[n] =
M
X
k=N
c
k
x
n
[k]
Each bit is sent through a separable (orthogonal) signal.
Reverse operation at receiver extracts each bit by exploiting signal
orthogonality.
10
DFT Application: OFDM
Data
IDFT
Network
DFT
Data
^
X[k] =
1
N
N1
X
n=0
x[n]e
j2kn=N
k = 0; : : : ; N 1
x[n] =
N1
X
k=0
^
X[k]e
j2kn=N
n = 0; : : : ; N 1
(our reference algorithm)
y[n] =
M
X
k=N
c
k
x
n
[k]
Applications:
DSL
4G Phones
WiFi
11
DFT Application: Speech Recognition
^
X[k] =
1
N
N1
X
n=0
x[n]e
j2kn=N
k = 0; : : : ; N 1
As a result, we get the evolution of the spectrum over time.
Computer use this information to recognize what is being said.
x[n]
Digital speech
samples
Every short period of time we take samples and calculate the DFT: N
12
Understanding System Response
Inverse Discrete Fourier Transform
0 50 100 150 200 250
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
time, n
x
[
n
]
0 5 10 15 20
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
time, n
90 100 110 120 130 140 150 160 170
0
5
10
15
20
|
X
[
k
]
|
^
X[k] =
1
N
N1
X
n=0
x[n]e
j2kn=N
x[n] =
N1
X
k=0
^
X[k]e
j2kn=N
The formula tells us that any signal can be thought of as
being made from adding sinusoids (after multiplying each
with an appropriate complex number).
13
Understanding System Response
To get started, two important observations:
Any signal can be thought of as being made from
adding sinusoids (after multiplying each with an
appropriate complex number).
If we test a linear system with a sinusoidal signal,
the system responds with a sinusoidal signal of the
same frequency as the input but with different
amplitude and phase.
14
Understanding System Response
If we test a linear system with a sinusoidal signal, the system
responds with a sinusoidal signal of the same frequency as
the input but with different amplitude and phase.
With math:
For input
Output is
)

cos( ] [
1
o e + = n A n x
)) (

cos( ) ( ] [
1 1

1
e e
o e
j j
e H n e H A n y Z + + =
Change in
amplitude
Change
in phase
The effect of the system at frequency is
represented through the complex number
1

e
) (
1

e j
e H
15
Understanding System Response
Any signal can be thought of as being made from adding sinusoids (after
multiplying each with an appropriate complex number).
Decompose input signal into sinusoidal components; get its spectrum.
If we test a linear system with a sinusoidal signal, the system responds with a
sinusoidal signal of the same frequency as the input but with different
amplitude and phase.
By testing the system at different frequencies, find the function that
maps the response of the system (amplitude and phase) as a function
of frequency. This is called the system frequency response.
Multiply the input signal spectrum with the system frequency response.
The result shows the components of the output signal at the sinusoidal
components of different frequencies. This is the output signal spectrum.
Assemble the output signal from its spectrum (e.g. inverse Discrete
Fourier Transform).
Decompose input signal into sinusoidal components; get its spectrum.
Multiply the input signal spectrum with the system frequency response. The
result shows the components of the output signal at the sinusoidal
components of different frequencies. This is the output signal spectrum.
Assemble the output signal from its spectrum (e.g. inverse Discrete Fourier
Transform).
16
Understanding System Response
FILTER
(LTI)
^
X[k] =
1
N
N1
X
n=0
x[n]e
j2kn=N
^
Y [k] =
^
H[k]
^
X[k]
y[n] =
N1
X
k=0
^
Y [k]e
j2kn=N
17
Z-transform
X(z) =
1
X
k=1
x[k]z
k
z is a complex variable.
18
Z-transform
X(z) =
1
X
k=1
x[k]z
k
(Think of it as a more general form of the DFT)
z is a complex variable.
^
X[k] =
1
N
N1
X
n=0
x[n]e
j2kn=N
19
Z-transform
X(z) =
1
X
k=1
x[k]z
k
(Think of it as a more general form of the DFT)
z is a complex variable.
^
X[k] =
1
N
N1
X
n=0
x[n]e
j2kn=N
(our reference algorithm)
y[n] =
M
X
k=N
c
k
x
n
[k]
Decompose input signal into sinusoidal components; get its spectrum.
Multiply the input signal spectrum with the system frequency response. The
result shows the components of the output signal at the sinusoidal
components of different frequencies. This is the output signal spectrum.
Assemble the output signal from its spectrum (e.g. inverse Discrete Fourier
Transform).
20
Understanding System Response
FILTER
(LTI)
^
Y [k] =
^
H[k]
^
X[k]
y[n] =
N1
X
k=0
^
Y [k]e
j2kn=N
^
X[k] =
1
N
N1
X
n=0
x[n]e
j2kn=N
Decompose input signal into sinusoidal components; get its spectrum.
Multiply the input signal spectrum with the system frequency response. The
result shows the components of the output signal at the sinusoidal
components of different frequencies. This is the output signal spectrum.
Assemble the output signal from its spectrum (e.g. inverse Discrete Fourier
Transform).
21
Understanding System Response
FILTER
(LTI)
^
X[k] =
1
N
N1
X
n=0
x[n]e
j2kn=N
^
Y [k] =
^
H[k]
^
X[k]
y[n] =
N1
X
k=0
^
Y [k]e
j2kn=N
22
Z-Transforms Properties Convolution/Multiplication
23
Z-Transforms Properties Convolution/Multiplication
24
Z-Transforms Properties Convolution/Multiplication
25
Z-Transforms Properties Convolution/Multiplication
26
Z-Transforms Properties Convolution/Multiplication
Convolution sum
27
Z-Transforms Properties Convolution/Multiplication
Convolution sum
28
Useful Z-Transforms Properties
Convolution in time, Multiplication in z-domain:
LTI FILTER x[n] y [n]
y[n] = h[n] x[n]
Z
!Y (Z) = H(z)X(z)
The convolution sum
tells us, in time domain,
how the filter (system)
processes the input to
generate the output
Time domain Z domain
(our reference algorithm)
y[n] =
M
X
k=N
c
k
x
n
[k]
29
Z-Transforms Properties Time Delay
30
Z-Transforms Properties Time Delay
31
Z-Transforms Properties Time Delay
32
Z-Transforms Properties Time Delay
33
A Digital Integrator
Rectangular rule (Eulers rule)
After digitalization, difference equation is:
Integration Increment: T
s
34
A Digital Integrator
Forward rectangular rule (Eulers rule)
This is a filter whose output is the integral of its input
The transfer function of the corresponding digital
integrator is (taking Z transforms)
35
A Digital Integrator
The problem for this system is that is unstable for some input signals
(not good if the system will be used for something that is not
integration).
0
2
4
6
8
10
12
14
0 5 10 15
x[n]
y[n]
When the input is just a
constant (turned on to a
value =1) the output
grows for ever
36
Digital Systems
How can we tell if and when a system will be unstable?
We look at the roots of the
denominator of the systems
transfer function.
If any root has magnitude larger of equal to 1, the system is
unstable.
37
Making an Unstable System Stable
+
-
Original unstable system
We connect other system to
control the original system
The resulting overall stable
system.
38
Making an Unstable System Stable
+
-
39
Making an Unstable System Stable
+
-
40
Making an Unstable System Stable
+
-
41
Making an Unstable System Stable
+
-
42
Making an Unstable System Stable
+
-
43
Making an Unstable System Stable
+
-
44
Making an Unstable System Stable
+
-
By design, we choose:
(this is called an FIR filter)
Control system design means to find the constants
45
Making an Unstable System Stable
+
-
Control system design means to find the constants
Stability will be determined by the location (value) of the roots
of the denominator
46
Making an Unstable System Stable
+
-
47
Making an Unstable System Stable
+
-
The particular value of is chosen based on the
specific behavior we want of the system to some
signals
48
Making an Unstable System Stable
+
-
Making the filter run in a computer:
49
Making an Unstable System Stable
+
-
Making the filter run in a computer:
50
Making an Unstable System Stable
+
-
Making the filter run in a computer:
Chosen so that there is no amplification
of attenuation when input is a constant
51
Making an Unstable System Stable
+
-
Making the filter run in a computer:
Chosen so that there is no amplification
of attenuation when input is a constant
52
Making an Unstable System Stable
+
-
0
2
4
6
8
10
12
14
0 5 10 15
x[n]
y[n]
Before
0
0.2
0.4
0.6
0.8
1
1.2
0 5 10 15
x[n]
w[n]
After