TSKS01  

Digital  Communication
Lecture  3
Stochastic Processes,  Hypothesis Testing,  Introduction to  Digital  Modulation

Emil  Björnson

Department  of  Electrical  Engineering  (ISY)

Division  of  Communication  Systems
Normalized  Filters

Definition:  A  filter  ℎ(𝑡) is  normalized if   Parseval’s relation

( (
% ℎ 𝑡 & 𝑑𝑡 = 1                         and   % 𝐻 𝑓 & 𝑑𝑓 = 1  
)( )(

Example:  Consider  input  𝑋(𝑡) with  𝑚4 = 0 and  𝑅4 (𝑓) = 𝑅7

Output  has  mean: 𝑚8 = 𝐻 0 𝑚4 = 0

(
has  variance: 𝜎8& = E 𝑌& (𝑡) = 𝑟8 0 = ∫)( 𝑅8 𝑓 𝑑𝑓
(
                                                 = 𝑅7 % 𝐻 𝑓 & 𝑑𝑓 = 𝑅7
)(

If  𝑿(𝒕) is  Gaussian  then  𝒀(𝒕) is  also  Gaussian

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 2

Cross-­Correlation  and  Cross-­Spectrum

Consider  two  processes:  𝑋(𝑡) and  𝑌(𝑡)

Definition:  Cross-­correlation 𝑟4,8 𝑡B ,𝑡& = 𝐸 𝑋 𝑡B 𝑌 𝑡&

Properties:        𝑟4,8 𝑡B,𝑡& = 𝑟8,4 𝑡& ,𝑡B and  𝑟4,4 𝑡B ,𝑡& = 𝑟4 𝑡B, 𝑡&

Definition:  𝑋 (𝑡) and  𝑌(𝑡) are  jointly wide-­sense  stationary  (WSS) if

§ 𝑋(𝑡) is  WSS  and  𝑌(𝑡) is  WSS.
§ 𝑟4,8 𝑡B ,𝑡& only  depends  on  𝑡B − 𝑡& .

Notation:  𝑟4,8 𝜏 = 𝑟4,8 𝑡, 𝑡 + 𝜏

Cross-­spectrum:  𝑅4,8 𝑓 = ℱ 𝑟4,8 𝜏
2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 3
Uncorrelated  and  Independent  Processes

Consider  two  processes:  𝑋(𝑡) and  𝑌(𝑡)

Definition:  𝑋 (𝑡) and  𝑌(𝑡) are  uncorrelated if

𝑟4,8 𝑡B,𝑡& = 𝐸 𝑋 𝑡B 𝐸 𝑌 𝑡& = 𝑚4 𝑡B 𝑚8 (𝑡& )
for  all  𝑡B and  𝑡&.

Definition:  Sample  𝑋(𝑡) and  𝑌(𝑡) at  time  instants  𝑡B̅ = (𝑡B,B ,… , 𝑡B,J )
and  𝑡&̅ = (𝑡&,B ,… , 𝑡&,J ).  The  processes  are  independent if
𝐹4 L̅M ,8 L̅N 𝑥̅,𝑦Q = 𝐹4 L̅M   (𝑥̅ )𝐹8 L̅N (𝑦Q)
for  every  𝑁,  𝑡B̅ ,  and  𝑡&̅ .

Relation:   Independent  à Uncorrelated

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 4
Jointly  Gaussian  Processes

Consider  two  processes:  𝑋(𝑡) and  𝑌(𝑡)

Definition:  Sample  𝑋(𝑡) and  𝑌(𝑡) at  time  instants  𝑡B̅ = (𝑡B,B ,… , 𝑡B,J )
and  𝑡&̅ = (𝑡&,B ,… , 𝑡&,J ).  The  processes  are  jointly  Gaussian  if  𝑋 𝑡B̅
and  𝑌 𝑡&̅ are  jointly  Gaussian  variables  for  every  𝑁,  𝑡B̅ ,  and  𝑡&̅ .

Theorem:  If 𝑋(𝑡) and  𝑌(𝑡) are  jointly  Gaussian  processes  and  
uncorrelated,  they  are  also  independent

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 5

 Stationarity of  the  Filtered  Output
Stochastic  process
(
WSS  stochastic LTI-­‐‑system
process  𝑋(𝑡) ℎ 𝑡 𝑌 𝑡 = % ℎ 𝜏 𝑋(𝑡 − 𝜏)𝑑𝜏
)(

Cross-­correlation:
(

𝑟4,8 𝑡B,𝑡& = 𝐸 𝑋 𝑡B 𝑌 𝑡& = 𝐸 % ℎ 𝜏 𝑋 𝑡B 𝑋(𝑡& − 𝜏)𝑑𝜏

)(
𝑋(𝑡) is  WSS (

= % ℎ 𝜏 𝑟4 𝑡B − 𝑡& + 𝜏 𝑑𝜏 = (ℎ ∗ 𝑟4 )(𝑡B − 𝑡& )

)(

𝑿(𝒕) and  𝒀(𝒕) are  jointly  WSS,  if  𝑿(𝒕) is  WSS

Notation:  𝑟4,8 𝜏 = (ℎ ∗ 𝑟4 )(𝜏) and        𝑅4,8 𝑓 = 𝐻 𝑓 𝑅4 (𝑓)

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 6

 Orthogonal  Filters

LTI-­‐‑system (
𝑌B 𝑡 = ∫)( ℎB 𝜏 𝑋(𝑡 − 𝜏)𝑑𝜏
ℎB 𝑡
WSS  stochastic
process  𝑋(𝑡)
LTI-­‐‑system (
𝑌& 𝑡 = ∫)( ℎ& 𝜏 𝑋(𝑡 − 𝜏)𝑑𝜏
ℎ& 𝑡

(
Definition:  ℎB 𝑡 and  ℎ& 𝑡 are  orthogonal  filters  if  ∫)( ℎB 𝜏 ℎ& 𝜏 𝑑𝜏 = 0

Property:  If  𝑋(𝑡) is  WSS  with  𝑚4 = 0 and  𝑅4 𝑓 = 𝑅7,  then
𝑚8e = 𝐻f 0 𝑚4 = 0      for      𝑘 = 1,2
(
𝑟8M ,8N 𝑡, 𝑡 = 𝐸 𝑌B 𝑡 𝑌& 𝑡 = ⋯ = 𝑅7 ∫)( ℎB 𝜏 ℎ& 𝜏 𝑑𝜏 = 0 = 𝑚8M 𝑚8N

Uncorrelated  outputs! Gaussian  input  à Independent  outputs

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 7

 A  One-­way  Telecommunication  System

Packing Error  control Digital to  analog Medium

Source   Channel  
Source Modulator
encoder encoder

Source   Channel   Digital   Channel

coding coding modulation

Source   Channel   De-­

Destination
decoder decoder modulator

Unpacking Error  correction Analog  to  digital

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 8

Situation
Noise

Source Sender Channel Receiver Destination

One message every 𝑇 seconds.

00 10 11 01 11 00 01 10
Represented using time-­delimited signals.  Duration  𝑇 seconds (T~1/bandwidth)

The  channel  adds  noise  to  the  signal.

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 9

 Analyze  Only  One  Interval
Noise

Source Sender Channel Receiver Destination

We consider only one interval [0, 𝑇).  Reasons:

§ Different  samples in  𝐴[𝑛] are  assumed independent.
⇒ No  information  gained about one sample from  another.
§ Signals  from  different  samples in  𝐴[𝑛] are non-­overlapping in  time.
⇒ No  intersymbol  interference due to  modulation.
§ The  channel does not  filter  the  signal.
⇒ No  intersymbol  interference due to  filtering.
§ Different  samples of WGN  are independent.
⇒ Noise from  non-­overlapping intervals are independent.

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 10

 Notation
Noise

Source Sender Channel Receiver Destination

Message  set: Size:

Signal  set: Zero  outside Duration:

Mapping: → for

Signal  energy:

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 11

 The  Channel  and  the  Receiver
Noise,  𝑾(𝒕)

Source Sender Channel Receiver Destination

AWGN  channel: Additive  White  Gaussian  Noise

Channel  output: 𝑋 𝑡 = 𝑠v 𝑡 + 𝑊(𝑡)

Receiver: Observes 𝑋(𝑡).    Generates estimate 𝐴x.

Goal: Minimize Pr  {𝐴x ≠ 𝐴}.

This lecture concentrates on  the  sender and  the  added noise

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 12

 Where  are  we  heading?
Noise

Source Sender Channel Receiver Destination

Noise

Vector   De-­ Vector  

Source Modulator Channel Destination
sender modulator detector

Vector  noise

Vector   Vector  
Source Vector  channel Destination
sender detector

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 13

Basics  of  Hypothesis  Testing

Hypothesis  Testing
§ Goal:  Guess  properties  of  a  continuous  stochastic  vector  𝑋Q
§ Observe  realization  𝑋Q = 𝑥̅
§ 𝑀 hypothesis  𝐻7 , … , 𝐻~)B with  probabilities  Pr 𝐻v for  𝑖 = 0, … , 𝑀 − 1
§ Select  the  “best”  hypothesis  based  on  𝑥̅

Application   in  Digital  Communication

§ Message  from  source:  Stochastic  variable  𝑆̅
§ Observation:  𝑋Q = 𝑥̅ correlated  with  𝑆̅ = 𝑠̅
§ Use  observation  to  guess  𝑠̅

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 14

Methodology  of  Hypothesis  Testing

• 𝑥̅
Decision  rule:  𝐻 − what  criterion  to  use?

Maximum  À-­Posteriori  (MAP):

•~‚ƒ 𝑥̅ = 𝑘          where        𝑘 = argmax7†v†~)B     Pr 𝐻v  is  true  |𝑋Q = 𝑥̅
𝐻

Maximum  Likelihood (ML):

•~‰ 𝑥̅ = 𝑘          where        𝑘 = argmax7†v†~)B     𝑓4Q|Š (𝑥̅|𝐻v  is  true)
𝐻

Which  to  choose?

§ MAP  minimizes  error  prob:  𝑃Œ = ∑~)B
v‘7 Pr 𝐻v Pr select  𝐻•  for  𝑗 ≠ 𝑖  |  𝐻v  is  true  

§ MAP  also  written  as  𝑘 = argmax7†v†~)B Pr 𝐻v 𝑓4Q|Š (𝑥̅ |𝐻v  is  true)
§ MAP=ML  if  hypotheses  are  equally  probable
2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 15
Example:  Signal  in  Noise

Additive  noise  channel:

𝑋 = 𝑆 + 𝑊  
§ 𝑆 is  +1 or  −1,  equal  probability
&
§ 𝑊 ∼ 𝑁(0, 𝜎“ ) is  independent  Gaussian  noise

&)
Hypothesis  0:  𝑆 = −1,  thus 𝑋|𝑆 ∼ 𝑁(−1, 𝜎“
— ˜B N
1 ) N
𝑓4|” 𝑥 −1 = 𝑒 & ™š
2𝜋𝜎“  

&)
Hypothesis  1:  𝑆 = +1,  thus 𝑋|𝑆 ∼ 𝑁(1, 𝜎“
— )B N
1 ) N
𝑓4|” 𝑥 +1 = 𝑒 & ™š
2𝜋𝜎“  
2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 16
Example:  Signal  in  Noise  cont’d
•žM N • M N
) )
NŸN NŸN
𝐻7 is  “best”:    𝑓4|” 𝑥 −1 > 𝑓4|” 𝑥 +1     ⟺     𝑒 š >𝑒 š       ⟺       𝑥 < 0
0.7
Hypothesis 0 most likely Hypothesis 1 most likely
0.6
Conditional probability density

0.5

0.4 fX|S (x| − 1) fX|S (x| + 1)

0.3

0.2

0.1

0
−4 −3 −2 −1 0 1 2 3 4
Observation x

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 17

Example:  Signal  in  Noise  cont’d

• 𝑥̅ = 𝐻B |𝐻7 + Pr 𝐻B Pr 𝐻
Error  prob.:  𝑃Œ = Pr 𝐻7 Pr 𝐻 • 𝑥̅ = 𝐻7|𝐻B
= 𝑄  (1/𝜎“ )
0.7

0.6
Conditional probability density

0.5

0.4

0.3

1 1
0.2 Area Q Area Q
σW
σW

0.1

0
−4 −3 −2 −1 0 1 2 3 4
Observation x

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 18

Indicator  Function

Definition: The  indicator function 𝐼‚ (𝑡) of the  set  𝐴 is  given  by

Examples:

𝐼 L¤7 𝑡 = 𝑢(𝑡) 𝐼 7†L¦§ (𝑡) 𝑡·𝐼 L ¦§ (𝑡)

1 1 T
–T
T T
–T

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 19

Function  Spaces  – Vector  Spaces

𝑀 signals,   span 𝑁 dimensions,                          .

Ortho-­normal  (ON)  basis: (spans the same dimensions)  

The  signals  expressed using this basis:

Corresponding vectors:

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 20

Result  – Split  the  Sender

Sender

Vector  
Modulator
sender

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 21

Example:  Vector representation
with respect to  {𝜙7 (𝑡), 𝜙B (𝑡)}

𝜙B

𝑠̅B

𝑠̅&

𝜙7

𝑠̅7

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 22

Example:  Vector representation
with respect to  {𝜃7 (𝑡), 𝜃B(𝑡)}

𝜃B

𝑠̅&
𝑠̅7
𝜃7

𝑠̅B

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 23

Example:  Vector  representation
with  respect  to  both  bases
𝜙B
𝑠̅B
𝜃7

𝑠̅&

𝜙7

𝑠̅7

𝜃B

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 24

Geometric  Interpretation

Signals:   and   .              Zero outside .

Vectors:  

ON-­basis:  

Inner  products:

Claim:

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 25

Geometric  Interpretation

Claim:

Proof:

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 26

 𝑎Q
Norms  and  Angles
| 𝑎Q |

Norms: of                  : Signal  energy.

of          : Squared  length.

Relation: ⇒

From  linear algebra:

Angle  between  signals:

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 27

Distances 𝑎Q
𝑏Q
𝑑(𝑎Q, 𝑏Q)
Euclidean   distances  between  vectors:

The  Pythagorean relation  in  𝑁 dimensions.

Euclidean   distances  between  signals:

These  computations  are  equivalent!

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 28

Example:  Norms,  Distances and  Angles
Example

𝜙B

𝑠̅B

𝑠̅&

𝜙7

𝑠̅7

2015-­09-­11 TSKS01  Digital  Communication  -­ Lecture  3 29

www.liu.se