Digital Communication Systems by Simon Haykin-124

Download as pdf or txt
Download as pdf or txt
You are on page 1of 6

Haykin_app_D_pp3.

fm Page 19 Tuesday, January 8, 2013 1:14 PM

APPENDIX
D Method of Lagrange Multipliers
D.1 Optimization Involving a Single Equality Constraint

Consider the minimization of a real-valued function f(w) that is a quadratic function of a


parameter vector w, subject to the constraint
w† s = g (D.1)

where s is a prescribed vector and g is a complex constant; the superscript denotes
Hermitian transposition. We may redefine the constraint by introducing a new function
c(w) that is linear in w, as shown by
c  w  = w† s – g
(D.2)
= 0 + j0
In general, the vectors w and s and the function c(w) are all complex. For example, in a
beamforming application, the vector w represents a set of complex weights applied to the
individual sensor outputs and s represents a steering vector whose elements are defined by
a prescribed “look” direction; the function f(w) to be minimized represents the mean-
square value of the overall beamformer output. In a harmonic retrieval application, for
another example, w represents the tap-weight vector of an FIR filter and s represents a
sinusoidal vector whose elements are determined by the angular frequency of a complex
sinusoid contained in the filter input; the function f(w) represents the mean-square value
of the filter output. In any event, assuming that the issue is one of minimization, we may
state the constrained optimization problem as follows:
Minimize a real-valued function f(w), subject to the constraint c(w) = 0 +j0 (D.3)
The method of Lagrange multipliers converts the problem of constrained minimization
just described into one of unconstrained minimization by the introduction of Lagrange
multipliers. First, we use the real function f(w) and the complex constraint function c(w)
to define a new real-valued function
h  w  = f  w  +  1 Re  c  w   +  2 Im  c  w   (D.4)

where 1 and 2 are real Lagrange multipliers and


c  w  = Re  c  w   + jIm  c  w   (D.5)
Now we define a complex Lagrange multiplier:
 = 1 + j 2 (D.6)

The Re[] and Im[] in (D.4) and (D.5) denote real and imaginary operators, respectively.
We may then rewrite (D.4) in the form
h  w  = f  w  + Re   *c  w   (D.7)
where the asterisk denotes complex conjugation.

A19
Haykin_app_D_pp3.fm Page 20 Tuesday, January 8, 2013 1:14 PM

A20 Appendix D Method of Lagrange Multipliers

Next, we minimize the function h(w) with respect to the vector w. To do this, we set the
conjugate derivative h   w*  equal to the null vector:
f 
----------- + -----------  Re   *c  w    = 0 (D.8)
w* w*
The system of simultaneous equations consisting of (D.8) and the original constraint given
in (D.2) defines the optimum solutions for the vector w and the Lagrange multiplier . We
call (D.8) the adjoint equation and (D.2) the primal equation (Dorny, 1975).
Haykin_app_E_pp2.fm Page 21 Friday, December 7, 2012 10:17 AM

APPENDIX
E Information Capacity of MIMO Channels
The topic of multiple-input multiple-output (MIMO) links for wireless communications
was discussed in Chapter 9 on signaling over fading channels. To get a measure of the
transmission efficiency of MIMO links therein, we resorted to the notion of outage
capacity, which is naturally of practical interest. However, in light of its mathematical
sophistication, we deferred discussion of the information capacity of MIMO links rooted
in Shannon’s information theory to this appendix.
To be specific, in this appendix we discuss two different aspects of information
capacity:
1. The channel state is known to the receiver but not the transmitter;
2. The channel state is known to both the receiver and the transmitter.
The discussion will proceed in this order.

E.1 Log-Det Capacity Formula of MIMO Channels

Consider a communication channel with multiple antennas.1 Let the Nt-by-1 vector s
denote the transmitted signal vector and the Nr-by-1 vector x denote the received signal
vector. These two vectors are related by the input–output relation of the channel:
x = Hs + w (E.1)
where H is the channel matrix of the link and w is the additive channel noise vector. The
vectors s, w, and x are realizations of the random vectors S, W, and X, respectively.
In what follows in this appendix, the following assumptions are made:
1. The channel is stationary and ergodic.
2. The channel matrix H is made up of iid Gaussian elements.
3. The transmitted signal vector s has zero mean and correlation matrix Rs.
4. The additive channel noise vector w has zero mean and correlation matrix Rw.
5. Both s and w are governed by Gaussian distributions.
In this section, we also assume that the channel state H is known to the receiver but not the
transmitter. With both H and x unknown to the transmitter, the primary issue of interest is
to determine I(s;x,H), which denotes the mutual information between the transmitted
signal vector s and both the received signal vector x and the channel matrix H. Extending
the definition of mutual information introduced in Chapter 5 to the problem at hand, we
write

 f S X H  s x H 
I  S ; X H  =    fS X H  s x H  log 2 ------------------------------------
f X H  x H  
 ds dx dH (E.2)
ℋ 𝒳𝒮
where 𝒮, 𝒳, and ℋ are the respective spaces pertaining to the random vectors S and X and
matrix H.

A21
Haykin_app_E_pp2.fm Page 22 Friday, December 7, 2012 10:17 AM

A22 Appendix E Information Capacity of MIMO Channels

Using the definition of a joint probability density function (pdf) as the product of a
conditional pdf and an ordinary pdf, we write
f S X H  s x H  = f S X H  s x H f H  H 
We may therefore rewrite (E.2) in the equivalent form

 f S X H  s x H 
I  S ; X H  =  fH  H   f S X H  s x H  log  ------------------------------------ ds dx dH
2 f X H  x H  
ℋ 𝒳𝒮

 f S X H  s x H  (E.3)
= ⺕H  f S X H  s x H  log  ------------------------------------ ds dx
2 f X H  x H  
𝒳𝒮
= ⺕ H  I  s ;x H  

where the expectation is with respect to the channel matrix H and

 f S X H  s x H 
I  s ;x H  = f S X H  s x H  log  ------------------------------------ ds dx
2 f X H  x H  
𝒳𝒮
is the conditional mutual information between the transmitted signal vector s and received
signal vector x, given the channel matrix H. However, by assumption, the channel state is
unknown to the transmitter. Therefore, it follows that, insofar as the receiver is concerned,
I(s;x|H) is a random vector; hence the expectation with respect to H in (E.3). The quantity
resulting from this expectation is therefore deterministic, defining the mutual information
jointly between the transmitted signal vector s and both the received signal vector x and
channel matrix H. The result so obtained is indeed consistent with what we know about
the notion of joint mutual information.
Next, applying the vector form of the first line in (5.81) to the mutual information
I(s;x|H), we have
I  s ;x H  = h  x H  – h  x s H  (E.4)

where h(x|H) is the conditional differential entropy of the channel output x given H, and
h(x|s,H) is the conditional differential entropy of x, given both s and H. Both of these
entropies are random quantities, because they both depend on H.
To proceed further, we now invoke the assumed Gaussianity of both s and H, in which
case x also assumes a Gaussian description. Under these circumstances, we may use the
result of Problem 5.32 to express the entropy of the received signal x of dimension Nr,
given H, as
h  x H  = N r + N r log  2  + log 2 det  R x   bits (E.5)
2
where Rx is the correlation matrix of x and det(Rx) is its determinant. Recognizing that the
transmitted signal vector s and channel noise vector w are independent of each other, we
find from (E.1) that the correlation matrix of the received signal vector x is given by
Haykin_app_E_pp2.fm Page 23 Friday, December 7, 2012 10:17 AM

E.1 Log-Det Capacity Formula of MIMO Channels A23

R x =  xx† 
= ⺕   Hs + w   Hs + w † 
= ⺕   Hs + w   s† H† + w†  
(E.6)
= ⺕  Hss† H†  + ⺕  ww†   ⺕  sw†  = 0 
= H⺕  ss† H† + R w
= HR s H† + R w
where † denotes Hermitian transposition,
R s = ⺕  ss†  (E.7)

is the correlation matrix of the transmitted signal vector s, and


R w = ⺕  ww†  (E.8)

is the correlation matrix of the channel noise vector w. Hence, using (E.6) in (E.5), we get
h  x H  = N r + N r log 2 2  + log 2 det  R w + HR s H†   bits (E.9)

where Nr is the number of elements in the receiving antenna. Next, we note that since the
vectors s and w are independent and the sum of w plus Hs equals x as indicated in (E.1),
then the conditional differential entropy of x, given both s and H, is simply equal to the
differential entropy of the additive channel noise vector w; that is,
h  x s H  = h  w  (E.10)

The entropy h(w) is given by (see Problem 5.32)


h  w  = N r + N r log  2  + log 2 det  R w   bits (E.11)
2
Thus, using (E.9), (E.10), and (E.11) in (E.4), we get

I  s ;x H  = log 2 det  R w + HR s H†   – log 2 det  R w  

  det  R w + HR s H†    (E.12)
= log  ----------------------------------------------------- 
2
  det  R w   
As remarked previously, the conditional mutual information I(s;x|H) is a random variable.
Hence, using (E.12) in (E.3), we finally formulate the ergodic capacity of the MIMO link
as the expectation

  det  R w + HR s H†   
C = ⺕ H log  -----------------------------------------------------  bits/(sHz) (E.13)
2
  det  R w   
which is subject to the constraint
max tr  R s   P
Rs
Haykin_app_E_pp2.fm Page 24 Friday, December 7, 2012 10:17 AM

A24 Appendix E Information Capacity of MIMO Channels

where P is constant transmit power and tr[.] denotes the trace operator, which extracts the
sum of the diagonal elements of the enclosed matrix.
Equation (E.13) is the desired log-det formula for the ergodic capacity of the MIMO
link. This formula is of general applicability, in that correlations among the elements of
the transmitted signal vector s and among those of the channel noise vector w are
permitted. However, the assumptions made in its derivation involve the Gaussianity of s,
H, and w.

E.2 MIMO Capacity for Channel Known at the Transmitter

The log-det formula of (E.13) for the ergodic capacity of a MIMO flat-fading channel
assumes that the channel state is only known at the receiver. What if the channel state is
also known perfectly at the transmitter? Then the channel state becomes known to the
entire system, which means that we may treat the channel matrix H as a constant. Hence,
unlike the partially known case treated in Section E.1, there is no longer the need for
invoking the expectation operator in formulating the log-det capacity. Rather, the problem
becomes one of constructing the optimal Rs (i.e., the correlation matrix of the transmitted
signal vector s) that maximizes the ergodic capacity. To simplify the construction
procedure, we consider a MIMO channel for which the number of elements in the
receiving antenna Nr and the number of elements in the transmitting antenna Nt have a
common value, denoted by N.
2
Accordingly, using the assumption of additive white Gaussian noise with variance  w
in the log-det capacity formula of (E.13), we get
  1 
C = log 2 det  I N + ------- HR s H†  bits/(sHz) (E.14)
  
2
w
We can now formally postulate the optimization problem at hand as follows:
Maximize the ergodic capacity C of (E.14) with respect to the correlation
matrix Rs, subject to two constraints, expressed as
1. Nonnegative definite Rs, which is a necessary requirement for a correlation
matrix.
2. Global power constraint
tr  R s  = P (E.15)

where P is the total transmit power.


To proceed with construction of the optimal Rs, we first use the determinant identity:
det  I + AB  = det  I + BA  (E.16)
Application of this identity to (E.14) yields
  1 
C = log 2 det  I N + ------- R s H† H  bits/(sHz) (E.17)
  
2
w

You might also like