Digital Communications

Instructor: Dr. Phan Van Ca

Lecture 08: The Union Bound

Kyung Hee
University
Modulation

‰ Modulation is used to transmit digital data over a channel

‰ Gramm-Schmidt procedure allows vector representation of

any signal constellation

‰ Optimal receiver consists of a correlator, weighted to

adjust for signal energy and a priori probabilities

‰ Receiver implementation can be simplified with reduced

correlations and matched filter

‰ Receiver decision rule corresponds to decision regions in

signal space
Young S. Kim Wireless Communication Systems Lab. - 2 -
Kyung Hee
University

Digital Communications
Spring 2006

Instructor: Young Soo Kim

Lecture #8 - The Union Bound

Young S. Kim Wireless Communication Systems Lab. - 1 -

Modulation

‰ Modulation is used to transmit digital data over a channel

‰ Gramm-Schmidt procedure allows vector representation of

any signal constellation

‰ Optimal receiver consists of a correlator, weighted to

adjust for signal energy and a priori probabilities

‰ Receiver implementation can be simplified with reduced

correlations and matched filter

‰ Receiver decision rule corresponds to decision regions in

signal space
 Performance Evaluation

‰ Probability of error can be calculated by integrating

probability density function of received signal vector over
decision regions.

‰ This integration can become extremely complicated

‰ Today we explore a simpler method for error probability

calculations called the Union Bound

Kyung Hee
University
The Union Bound

‰ In theory, we can compute any error probability by

integration over the appropriate regions.

‰ In practice, even with appropriate rotations and

translations of the signal set, integration will become
tedious for large signal constellations

‰ The Union Bound allows us to reduce error probability

calculations to a series of binary error calculations.

Young S. Kim Wireless Communication Systems Lab. - 4 -

Kyung Hee
University
Performance Evaluation

‰ Probability of error can be calculated by integrating

probability density function of received signal vector over
decision regions.

‰ This integration can become extremely complicated

‰ Today we explore a simpler method for error probability

calculations called the Union Bound

Young S. Kim Wireless Communication Systems Lab. - 3 -

The Union Bound

‰ In theory, we can compute any error probability by

integration over the appropriate regions.

‰ In practice, even with appropriate rotations and

translations of the signal set, integration will become
tedious for large signal constellations

‰ The Union Bound allows us to reduce error probability

calculations to a series of binary error calculations.
 Derivation of Union Bound

Pr[s ≠ si s = si ] = ∫ p(r si )dr

Ri C
M
= ∑ ∫ p( r si )dr
j =1, j ≠i R j

[ ]
M
Pr[ s ≠ si s = si ] ≤ ∑ Pr Z j ≥ Zi
j =1, j ≠i

‰ where N0 K 1 K
Zi = ln[ pi ] + ∑ rk si , k − ∑ si , k 2
2 k =1 2 k =1

Kyung Hee
University
Derivation of Union Bound (continued)

‰ Assume ML case:

⎡ K ⎤
[ ] 1 K 1
2
Pr Z j ≥ Zi = Pr ⎢ ∑ rk s j ,k − s j ,k ≥ ∑ rk si ,k − si ,k ⎥ 2
⎣ k =1 2 k =1 2 ⎦
⎡ K K ⎤
= Pr ⎢ ∑ s j ,k − 2rk s j , k + rk ≤ ∑ si , k 2 − 2rk si , k + rk2 ⎥
2 2
⎣ k =1 k =1 ⎦
⎡ K 2⎤
( )
K
2
= Pr ⎢ ∑ s j ,k − rk ≤ ∑ si , k − rk ⎥ ( )
⎣ k =1 k =1 ⎦

Young S. Kim Wireless Communication Systems Lab. - 6 -

Kyung Hee
University
Derivation of Union Bound

Pr[s ≠ si s = si ] = ∫ p(r si )dr

Ri C
M
= ∑ ∫ p( r si )dr
j =1, j ≠i R j

[ ]
M
Pr[ s ≠ si s = si ] ≤ ∑ Pr Z j ≥ Zi
j =1, j ≠i

‰ where N0 K 1 K
Zi = ln[ pi ] + ∑ rk si , k − ∑ si , k 2
2 k =1 2 k =1

Young S. Kim Wireless Communication Systems Lab. - 5 -

Derivation of Union Bound (continued)

‰ Assume ML case:

⎡ K ⎤
[ ] 1 K 1
2
Pr Z j ≥ Zi = Pr ⎢ ∑ rk s j ,k − s j ,k ≥ ∑ rk si ,k − si ,k ⎥ 2
⎣ k =1 2 k =1 2 ⎦
⎡ K K ⎤
= Pr ⎢ ∑ s j ,k − 2rk s j , k + rk ≤ ∑ si , k 2 − 2rk si , k + rk2 ⎥
2 2
⎣ k =1 k =1 ⎦
⎡ K 2⎤
( )
K
2
= Pr ⎢ ∑ s j ,k − rk ≤ ∑ si , k − rk ⎥ ( )
⎣ k =1 k =1 ⎦
 Expression of Decision Statistic
in terms of Distance Between Signals
⎡ K 2⎤
[ ] ( )
K
‰
2
Pr Z j ≥ Zi = Pr ⎢ ∑ s j ,k − rk ≤ ∑ si ,k − rk ⎥ ( )
⎣ k =1 k =1 ⎦
[( )
= Pr d r , s j ≤ d (r , si ) ]
‰ ( )
where d r , s j = ∑ k =1 s j , k − rk
K
( ) 2
is the Euclidean
distance between the points r and s j .

‰ This means the pairwise error probability between s j and si

is equal to the probability that the received signal vector r
is closer to s j than to si .

Kyung Hee
University
Computation of Pairwise Error Probability

‰ By appropriate rotation and translation, we can always

express the pairwise decision problem in the form:
Rj Ri

′ di, j 0 ′ di, j
sj = − si =
2 2

‰ ( )
Where di, j = d si ,s j = ∑ k =1 si,k − s j ,k
K 2(is the )
Euclidean distance between the points si and s j .

⎛ di , j ⎞
‰ [
This implies that Pr Z j ≥ Zi = Q⎜ ] ⎝ 2 N0 ⎠
⎟

Young S. Kim Wireless Communication Systems Lab. - 8 -

 Expression of Decision Statistic
Kyung Hee
University in terms of Distance Between Signals
⎡ K 2⎤
[ ] ( )
K
‰
2
Pr Z j ≥ Zi = Pr ⎢ ∑ s j ,k − rk ≤ ∑ si ,k − rk ⎥ ( )
⎣ k =1 k =1 ⎦
[( )
= Pr d r , s j ≤ d (r , si ) ]
‰ ( )
where d r , s j = ∑ k =1 s j , k − rk
K
( ) 2
is the Euclidean
distance between the points r and s j .

‰ This means the pairwise error probability between s j and si

is equal to the probability that the received signal vector r
is closer to s j than to si .

Young S. Kim Wireless Communication Systems Lab. - 7 -

Computation of Pairwise Error Probability

‰ By appropriate rotation and translation, we can always

express the pairwise decision problem in the form:
Rj Ri

′ di, j 0 ′ di, j
sj = − si =
2 2

‰ ( )
Where di, j = d si ,s j = ∑ k =1 si,k − s j ,k
K 2(is the )
Euclidean distance between the points si and s j .

⎛ di , j ⎞
‰ [
This implies that Pr Z j ≥ Zi = Q⎜ ] ⎝ 2 N0 ⎠
⎟
 The Union Bound (final form)

M ⎛ di , j ⎞
‰ Pr[ s ≠ si s = si ] ≤ ∑ Q⎜ ⎟
j =1, j ≠i ⎝ 2 N 0⎠
M 1 M ⎛ di , j ⎞
‰ Ps ( e) ≤ ∑ ∑ Q⎜ ⎟ (ML Assumption)
i =1 M j =1, j ≠i ⎝ 2 N 0⎠
‰ An improved form of the Union Bound:
M 1 ⎛ di , j ⎞
Ps ( e) ≤ ∑ ∑ ⎜ Q ⎟
i =1 M j ∈Ai ⎝ 2 N 0⎠
where Ai is the set of signals with decision regions directly
adjacent to Ri

Kyung Hee
University
Application of Union Bound to QPSK

‰ Distances between signals:

d1,2 = d1,4 = d2,3 = d3,4 = 2 E s = 4 Eb
d1,3 = d2,4 = 2 E s = 8 Eb
2
⎛ 2 Eb ⎞ ⎡ ⎛ 2 Eb ⎞ ⎤
‰ Exact Calculation: Ps ( e) = 2Q⎜ ⎟ − ⎢Q ⎜ ⎟⎥
⎝ N0 ⎠ ⎣ ⎝ N0 ⎠ ⎦

⎛ 2 Eb ⎞ ⎡ ⎛ 4 Eb ⎞ ⎤
‰ Union Bound: Ps ( e) ≤ 2Q⎜ ⎟ + ⎢Q⎜ ⎟⎥
⎝ N0 ⎠ ⎣ ⎝ N0 ⎠ ⎦

⎛ 2 Eb ⎞
‰ Improved Union Bound: Ps ( e) ≤ 2Q⎜ ⎟
⎝ N0 ⎠
Young S. Kim Wireless Communication Systems Lab. - 10 -
Kyung Hee
University
The Union Bound (final form)

M ⎛ di , j ⎞
‰ Pr[ s ≠ si s = si ] ≤ ∑ Q⎜ ⎟
j =1, j ≠i ⎝ 2 N 0⎠
M 1 M ⎛ di , j ⎞
‰ Ps ( e) ≤ ∑ ∑ Q⎜ ⎟ (ML Assumption)
i =1 M j =1, j ≠i ⎝ 2 N 0⎠
‰ An improved form of the Union Bound:
M 1 ⎛ di , j ⎞
Ps ( e) ≤ ∑ ∑ ⎜ Q ⎟
i =1 M j ∈Ai ⎝ 2 N 0⎠
where Ai is the set of signals with decision regions directly
adjacent to Ri

Young S. Kim Wireless Communication Systems Lab. - 9 -

Application of Union Bound to QPSK

‰ Distances between signals:

d1,2 = d1,4 = d2,3 = d3,4 = 2 E s = 4 Eb
d1,3 = d2,4 = 2 E s = 8 Eb
2
⎛ 2 Eb ⎞ ⎡ ⎛ 2 Eb ⎞ ⎤
‰ Exact Calculation: Ps ( e) = 2Q⎜ ⎟ − ⎢Q ⎜ ⎟⎥
⎝ N0 ⎠ ⎣ ⎝ N0 ⎠ ⎦

⎛ 2 Eb ⎞ ⎡ ⎛ 4 Eb ⎞ ⎤
‰ Union Bound: Ps ( e) ≤ 2Q⎜ ⎟ + ⎢Q⎜ ⎟⎥
⎝ N0 ⎠ ⎣ ⎝ N0 ⎠ ⎦

⎛ 2 Eb ⎞
‰ Improved Union Bound: Ps ( e) ≤ 2Q⎜ ⎟
⎝ N0 ⎠
 The Accuracy of the Union Bound
0
S ym b ol E rro r P rob ab ility of QP S K Using 3 M ethod s
10
E xact
Unio n B o und
10
-1 Im p ro ved Unio n B o und

-2
10
Ps(e)
-3
10

-4
10

-5
10

-6
10
0 2 4 6 8 10
E b/No (d B )

‰ The Union Bound is an upper bound because we may integrate

over some regions twice.
‰ Union Bound is very accurate, expecially for high Eb N o

Kyung Hee
University
Union Bound for M-ary PSK

‰ Signal Set: T
⎛ 2π ⎞⎛
sm ( t ) = 2P cos⎜ 2π fct + ( m−1) ⎟⎜ , m =1,…, M
⎝ M ⎠⎝0

‰ Basis Functions:
T T
2 2
f1 ( t ) = cos(2 πf ct ) , f 2 ( t ) = sin(2 πf ct )
T 0 T 0
‰ Signal Vectors:
⎡ ⎛ 2π (m − 1) ⎞ 2π (m − 1) ⎞⎤
sm = ⎢ Es cos⎜ ⎟ Es sin ⎛⎜ ⎟⎥,
⎣ ⎝ M ⎠ ⎝ M ⎠⎦
m = 1,…, M

Young S. Kim Wireless Communication Systems Lab. - 12 -

Kyung Hee
University
The Accuracy of the Union Bound
0
S ym b ol E rro r P rob ab ility of QP S K Using 3 M ethod s
10
E xact
Unio n B o und
10
-1 Im p ro ved Unio n B o und

-2
10
Ps(e)
-3
10

-4
10

-5
10

-6
10
0 2 4 6 8 10
E b/No (d B )

‰ The Union Bound is an upper bound because we may integrate

over some regions twice.
‰ Union Bound is very accurate, expecially for high Eb N o

Young S. Kim Wireless Communication Systems Lab. - 11 -

Union Bound for M-ary PSK

‰ Signal Set: T
⎛ 2π ⎞⎛
sm ( t ) = 2P cos⎜ 2π fct + ( m−1) ⎟⎜ , m =1,…, M
⎝ M ⎠⎝0

‰ Basis Functions:
T T
2 2
f1 ( t ) = cos(2 πf ct ) , f 2 ( t ) = sin(2 πf ct )
T 0 T 0
‰ Signal Vectors:
⎡ ⎛ 2π (m − 1) ⎞ 2π (m − 1) ⎞⎤
sm = ⎢ Es cos⎜ ⎟ Es sin ⎛⎜ ⎟⎥,
⎣ ⎝ M ⎠ ⎝ M ⎠⎦
m = 1,…, M
 8-ary PSK Signal Space

Signal Space and Decision Regions

1

0.8

0.6

0.4

0.2
Y

-0.2

-0.4

-0.6

-0.8

-1
-1 -0.5 0 0.5 1
X

Kyung Hee
University
Error Calculation for M-ary PSK

‰ Each signal has two adjacent decision regions.

‰ By symmetry, the distance di , j between all adjacent
signals is identical.

s2 R2
R1 Total Angle = 2π M
Half Angle = π M

s1 = [ Es 0]
di , j 2 = E s sin( π M ) = Eb log 2 M sin( π M )

Young S. Kim Wireless Communication Systems Lab. - 14 -

Kyung Hee
University
8-ary PSK Signal Space

Signal Space and Decision Regions

1

0.8

0.6

0.4

0.2
Y

-0.2

-0.4

-0.6

-0.8

-1
-1 -0.5 0 0.5 1
X

Young S. Kim Wireless Communication Systems Lab. - 13 -

Error Calculation for M-ary PSK

‰ Each signal has two adjacent decision regions.

‰ By symmetry, the distance di , j between all adjacent
signals is identical.

s2 R2
R1 Total Angle = 2π M
Half Angle = π M

s1 = [ Es 0]
di , j 2 = E s sin( π M ) = Eb log 2 M sin( π M )
 Symbol Error Probability for M-ary PSK

‰ Union Bound (M>2):

Ps ( e) = 2Q( 2 Eb log 2 M N 0 sin( π M ))

Kyung Hee
University
Performance Comparison for M-ary PSK

0
Symbol Error Probability for M-ary PSK
10

-1
10

-2
10
Ps(e)

-3
10

-4
M=2
10 M=4
M=8
M=16
-5 M=32
10

-6
10
0 5 10 15 20
Eb/No (dB)

Young S. Kim Wireless Communication Systems Lab. - 16 -

Kyung Hee
University
Symbol Error Probability for M-ary PSK

‰ Union Bound (M>2):

Ps ( e) = 2Q( 2 Eb log 2 M N 0 sin( π M ))

Young S. Kim Wireless Communication Systems Lab. - 15 -

Performance Comparison for M-ary PSK

0
Symbol Error Probability for M-ary PSK
10

-1
10

-2
10
Ps(e)

-3
10

-4
M=2
10 M=4
M=8
M=16
-5 M=32
10

-6
10
0 5 10 15 20
Eb/No (dB)
 Orthogonal Signal Sets

‰ M Basis Functions: { f1(t ),… , f M (t )}

‰ M Signals:
sm ( t ) = E s f m (t ) = Eb log 2 M f m ( t ), m = 1,… , M

‰ Signal Space Representation:

⎧ Es , m = i
[
sm = sm,1 ]
sm, M , sm,i = ⎨
⎩ 0, m≠i
‰ Example: M-ary FSK

sm ( t ) = 2 P cos(2 πf mt )
T
0

Kyung Hee
University
3-ary Orthogonal Signal Space

Young S. Kim Wireless Communication Systems Lab. - 18 -

Kyung Hee
University
Orthogonal Signal Sets

‰ M Basis Functions: { f1(t ),… , f M (t )}

‰ M Signals:
sm ( t ) = E s f m (t ) = Eb log 2 M f m ( t ), m = 1,… , M

‰ Signal Space Representation:

⎧ Es , m = i
[
sm = sm,1 ]
sm, M , sm,i = ⎨
⎩ 0, m≠i
‰ Example: M-ary FSK

sm ( t ) = 2 P cos(2 πf mt )
T
0

Young S. Kim Wireless Communication Systems Lab. - 17 -

3-ary Orthogonal Signal Space

 Union Bound Calculation
for M-ary Orthogonal Signaling

‰ di , j = 2 E s , i ≠ j

‰ Ps ( e) = ( M − 1)Q( E s N 0 )
= ( M − 1)Q( Eb log 2 M N0 )

Performance Comparison
Kyung Hee
University for M-ary Orthogonal Signaling

0
Symbol Error Probability for Coherent M-ary FSK
10

M=2
-1
10 M=4
M=8
M=16
-2
10
Ps(e)

-3
10

-4
10

-5
10

-6
10
0 2 4 6 8 10 12
Eb/No (dB)

Young S. Kim Wireless Communication Systems Lab. - 20 -

 Union Bound Calculation
Kyung Hee
University for M-ary Orthogonal Signaling

‰ di , j = 2 E s , i ≠ j

‰ Ps ( e) = ( M − 1)Q( E s N 0 )
= ( M − 1)Q( Eb log 2 M N0 )

Young S. Kim Wireless Communication Systems Lab. - 19 -

Performance Comparison
for M-ary Orthogonal Signaling

0
Symbol Error Probability for Coherent M-ary FSK
10

M=2
-1
10 M=4
M=8
M=16
-2
10
Ps(e)

-3
10

-4
10

-5
10

-6
10
0 2 4 6 8 10 12
Eb/No (dB)
 Comments on Error Probability for
Orthogonal Signals
‰ Performance improves as M increases

‰ In the limit (as M → ∞ ), error probability can be made

arbitrarily small as long as Eb N 0 > −1.6dB

‰ Most practical systems use noncoherent FSK rather than

coherent FSK. We will discuss noncoherent FSK soon.

Kyung Hee
University
Improvements on Orthogonal Signaling

‰ Orthogonal signalling achieves optimum performance as

M → ∞ , but for finite M improvements are possible.

‰ Biorthogonal Signaling:
z sm ( t ) are orthogonal for m = 1,… , M 2
z sm ( t ) = − sm− M 2 ( t ) for
m = 1 + M 2 ,… , M

‰ Union Bound:
Ps ( e) = ( M − 2)Q( Eb log 2 M N0 )

Young S. Kim Wireless Communication Systems Lab. - 22 -

 Comments on Error Probability for
Kyung Hee
University Orthogonal Signals
‰ Performance improves as M increases

‰ In the limit (as M → ∞ ), error probability can be made

arbitrarily small as long as Eb N 0 > −1.6dB

‰ Most practical systems use noncoherent FSK rather than

coherent FSK. We will discuss noncoherent FSK soon.

Young S. Kim Wireless Communication Systems Lab. - 21 -

Improvements on Orthogonal Signaling

‰ Orthogonal signalling achieves optimum performance as

M → ∞ , but for finite M improvements are possible.

‰ Biorthogonal Signaling:
z sm ( t ) are orthogonal for m = 1,… , M 2
z sm ( t ) = − sm− M 2 ( t ) for
m = 1 + M 2 ,… , M

‰ Union Bound:
Ps ( e) = ( M − 2)Q( Eb log 2 M N0 )
 6-ary Biorthogonal Signal Constellation

Kyung Hee
University
Simplex Signal Constellation:

‰ Note that the centroid of an orthogonal constellation is

[
located at: c = E s M Es M ]
‰ A more energy efficient signal constellation can be
achieved by moving the centroid to the origin.

‰ This can be accomplished by letting:

z sm′ = sm − c for m = 1,… , M

‰ Simplex Conjecture: The Simplex signal constellation is

believed to be the most energy efficient signal
constellation for any value of M
Young S. Kim Wireless Communication Systems Lab. - 24 -
Kyung Hee
University
6-ary Biorthogonal Signal Constellation

Young S. Kim Wireless Communication Systems Lab. - 23 -

Simplex Signal Constellation:

‰ Note that the centroid of an orthogonal constellation is

[
located at: c = E s M Es M ]
‰ A more energy efficient signal constellation can be
achieved by moving the centroid to the origin.

‰ This can be accomplished by letting:

z sm′ = sm − c for m = 1,… , M

‰ Simplex Conjecture: The Simplex signal constellation is

believed to be the most energy efficient signal
constellation for any value of M
 3-ary Simplex Signal Constellation

Note that simplex signal constellation reduces dimension

by one from orthogonal signaling

Kyung Hee
University
Summary of Union Bound

‰ Union Bound allows evaluation of error probability for

arbitrary signal constellations

‰ High dimensional signal constellations (such as FSK) can

become very energy efficient as M becomes large

‰ Fixed dimensional signal constellations (such as PSK and

QAM) use energy inefficiently as M becomes large

Young S. Kim Wireless Communication Systems Lab. - 26 -

Kyung Hee
University
3-ary Simplex Signal Constellation

Note that simplex signal constellation reduces dimension

by one from orthogonal signaling
Young S. Kim Wireless Communication Systems Lab. - 25 -

Summary of Union Bound

‰ Union Bound allows evaluation of error probability for

arbitrary signal constellations

‰ High dimensional signal constellations (such as FSK) can

become very energy efficient as M becomes large

‰ Fixed dimensional signal constellations (such as PSK and

QAM) use energy inefficiently as M becomes large