Digital Communication Systems by Simon Haykin-124
Digital Communication Systems by Simon Haykin-124
Digital Communication Systems by Simon Haykin-124
APPENDIX
D Method of Lagrange Multipliers
D.1 Optimization Involving a Single Equality Constraint
The Re[] and Im[] in (D.4) and (D.5) denote real and imaginary operators, respectively.
We may then rewrite (D.4) in the form
h w = f w + Re *c w (D.7)
where the asterisk denotes complex conjugation.
A19
Haykin_app_D_pp3.fm Page 20 Tuesday, January 8, 2013 1:14 PM
Next, we minimize the function h(w) with respect to the vector w. To do this, we set the
conjugate derivative h w* equal to the null vector:
f
----------- + ----------- Re *c w = 0 (D.8)
w* w*
The system of simultaneous equations consisting of (D.8) and the original constraint given
in (D.2) defines the optimum solutions for the vector w and the Lagrange multiplier . We
call (D.8) the adjoint equation and (D.2) the primal equation (Dorny, 1975).
Haykin_app_E_pp2.fm Page 21 Friday, December 7, 2012 10:17 AM
APPENDIX
E Information Capacity of MIMO Channels
The topic of multiple-input multiple-output (MIMO) links for wireless communications
was discussed in Chapter 9 on signaling over fading channels. To get a measure of the
transmission efficiency of MIMO links therein, we resorted to the notion of outage
capacity, which is naturally of practical interest. However, in light of its mathematical
sophistication, we deferred discussion of the information capacity of MIMO links rooted
in Shannon’s information theory to this appendix.
To be specific, in this appendix we discuss two different aspects of information
capacity:
1. The channel state is known to the receiver but not the transmitter;
2. The channel state is known to both the receiver and the transmitter.
The discussion will proceed in this order.
Consider a communication channel with multiple antennas.1 Let the Nt-by-1 vector s
denote the transmitted signal vector and the Nr-by-1 vector x denote the received signal
vector. These two vectors are related by the input–output relation of the channel:
x = Hs + w (E.1)
where H is the channel matrix of the link and w is the additive channel noise vector. The
vectors s, w, and x are realizations of the random vectors S, W, and X, respectively.
In what follows in this appendix, the following assumptions are made:
1. The channel is stationary and ergodic.
2. The channel matrix H is made up of iid Gaussian elements.
3. The transmitted signal vector s has zero mean and correlation matrix Rs.
4. The additive channel noise vector w has zero mean and correlation matrix Rw.
5. Both s and w are governed by Gaussian distributions.
In this section, we also assume that the channel state H is known to the receiver but not the
transmitter. With both H and x unknown to the transmitter, the primary issue of interest is
to determine I(s;x,H), which denotes the mutual information between the transmitted
signal vector s and both the received signal vector x and the channel matrix H. Extending
the definition of mutual information introduced in Chapter 5 to the problem at hand, we
write
f S X H s x H
I S ; X H = fS X H s x H log 2 ------------------------------------
f X H x H
ds dx dH (E.2)
ℋ 𝒳𝒮
where 𝒮, 𝒳, and ℋ are the respective spaces pertaining to the random vectors S and X and
matrix H.
A21
Haykin_app_E_pp2.fm Page 22 Friday, December 7, 2012 10:17 AM
Using the definition of a joint probability density function (pdf) as the product of a
conditional pdf and an ordinary pdf, we write
f S X H s x H = f S X H s x H f H H
We may therefore rewrite (E.2) in the equivalent form
f S X H s x H
I S ; X H = fH H f S X H s x H log ------------------------------------ ds dx dH
2 f X H x H
ℋ 𝒳𝒮
f S X H s x H (E.3)
= ⺕H f S X H s x H log ------------------------------------ ds dx
2 f X H x H
𝒳𝒮
= ⺕ H I s ;x H
f S X H s x H
I s ;x H = f S X H s x H log ------------------------------------ ds dx
2 f X H x H
𝒳𝒮
is the conditional mutual information between the transmitted signal vector s and received
signal vector x, given the channel matrix H. However, by assumption, the channel state is
unknown to the transmitter. Therefore, it follows that, insofar as the receiver is concerned,
I(s;x|H) is a random vector; hence the expectation with respect to H in (E.3). The quantity
resulting from this expectation is therefore deterministic, defining the mutual information
jointly between the transmitted signal vector s and both the received signal vector x and
channel matrix H. The result so obtained is indeed consistent with what we know about
the notion of joint mutual information.
Next, applying the vector form of the first line in (5.81) to the mutual information
I(s;x|H), we have
I s ;x H = h x H – h x s H (E.4)
where h(x|H) is the conditional differential entropy of the channel output x given H, and
h(x|s,H) is the conditional differential entropy of x, given both s and H. Both of these
entropies are random quantities, because they both depend on H.
To proceed further, we now invoke the assumed Gaussianity of both s and H, in which
case x also assumes a Gaussian description. Under these circumstances, we may use the
result of Problem 5.32 to express the entropy of the received signal x of dimension Nr,
given H, as
h x H = N r + N r log 2 + log 2 det R x bits (E.5)
2
where Rx is the correlation matrix of x and det(Rx) is its determinant. Recognizing that the
transmitted signal vector s and channel noise vector w are independent of each other, we
find from (E.1) that the correlation matrix of the received signal vector x is given by
Haykin_app_E_pp2.fm Page 23 Friday, December 7, 2012 10:17 AM
R x = xx†
= ⺕ Hs + w Hs + w †
= ⺕ Hs + w s† H† + w†
(E.6)
= ⺕ Hss† H† + ⺕ ww† ⺕ sw† = 0
= H⺕ ss† H† + R w
= HR s H† + R w
where † denotes Hermitian transposition,
R s = ⺕ ss† (E.7)
is the correlation matrix of the channel noise vector w. Hence, using (E.6) in (E.5), we get
h x H = N r + N r log 2 2 + log 2 det R w + HR s H† bits (E.9)
where Nr is the number of elements in the receiving antenna. Next, we note that since the
vectors s and w are independent and the sum of w plus Hs equals x as indicated in (E.1),
then the conditional differential entropy of x, given both s and H, is simply equal to the
differential entropy of the additive channel noise vector w; that is,
h x s H = h w (E.10)
det R w + HR s H† (E.12)
= log -----------------------------------------------------
2
det R w
As remarked previously, the conditional mutual information I(s;x|H) is a random variable.
Hence, using (E.12) in (E.3), we finally formulate the ergodic capacity of the MIMO link
as the expectation
det R w + HR s H†
C = ⺕ H log ----------------------------------------------------- bits/(sHz) (E.13)
2
det R w
which is subject to the constraint
max tr R s P
Rs
Haykin_app_E_pp2.fm Page 24 Friday, December 7, 2012 10:17 AM
where P is constant transmit power and tr[.] denotes the trace operator, which extracts the
sum of the diagonal elements of the enclosed matrix.
Equation (E.13) is the desired log-det formula for the ergodic capacity of the MIMO
link. This formula is of general applicability, in that correlations among the elements of
the transmitted signal vector s and among those of the channel noise vector w are
permitted. However, the assumptions made in its derivation involve the Gaussianity of s,
H, and w.
The log-det formula of (E.13) for the ergodic capacity of a MIMO flat-fading channel
assumes that the channel state is only known at the receiver. What if the channel state is
also known perfectly at the transmitter? Then the channel state becomes known to the
entire system, which means that we may treat the channel matrix H as a constant. Hence,
unlike the partially known case treated in Section E.1, there is no longer the need for
invoking the expectation operator in formulating the log-det capacity. Rather, the problem
becomes one of constructing the optimal Rs (i.e., the correlation matrix of the transmitted
signal vector s) that maximizes the ergodic capacity. To simplify the construction
procedure, we consider a MIMO channel for which the number of elements in the
receiving antenna Nr and the number of elements in the transmitting antenna Nt have a
common value, denoted by N.
2
Accordingly, using the assumption of additive white Gaussian noise with variance w
in the log-det capacity formula of (E.13), we get
1
C = log 2 det I N + ------- HR s H† bits/(sHz) (E.14)
2
w
We can now formally postulate the optimization problem at hand as follows:
Maximize the ergodic capacity C of (E.14) with respect to the correlation
matrix Rs, subject to two constraints, expressed as
1. Nonnegative definite Rs, which is a necessary requirement for a correlation
matrix.
2. Global power constraint
tr R s = P (E.15)