Basic Probability Theory

1. Consider the binary communication channel shown in Fig. 1. The channel

input symbol X may assume the state 0 or the state 1, and, similarly, the channel
output symbol Y may assume either the state 0 or the state 1. Because of the
channel noise, an input 0 may convert to an output 1 and vice versa. The channel
is characterized by the channel transition probabilities
 ,  ,
   , defined
by

Fig. 1

where    denote the events (X=0)  ( = 1), respectively, and

   , denote the events(Y=0)  ( = 1), respectively.
Note that
 +  = 1 =
 +  . Let ( ) = 0.5,  = 0.3   = 0.1.
(a) Find (  )  (  ).
(b) If a 0 was observed at the output, what is the probability that a 0 was the
input state?
(c) If a 1 was observed at the output, what is the probability that a 1 was the
input state?
(d) Calculate the probability of error
 .
Random Variables
1. An information source generates symbols at random from. a four-letter
alphabet (a, b, c, d} with probabilities P(a) = 0.5, P(b) = 0.3, and P(c) = P(d) =0.1
A coding scheme encodes these symbols into binary codes as follows:

1|Page
Let X be the r.v. denoting the length of the code, that is, the number of binary
symbols (bits).
(a) What is the range of X?
(b) Assuming that the generations of symbols are independent, find the
probabilities P(X = 1), P(X = 2), P(X = 3), and P(X > 3).
2. Show that

3. Let a r.v. X denote the outcome of throwing a fair die. Find the mean and
variance of X.
4. Find the mean and variance of the geometric r.v. X
5. A binary source generates digits 1 and 0 randomly with probabilities 0.7 and
0.3, respectively.
(a) What is the probability that three 1s and four 0s will occur in a seven-digit
sequence?
(b) What is the probability that at least three 1s will occur in a seven-digit
sequence?
6. A noisy transmission channel has a per-digit error probability p = 0.03.
(a) Calculate the probability of more than one error in 5 received digits.
(b) Repeat (a), using the Poisson approximation.
(c) calculate the probability of less than one error in 5 received digits.
7. Assume that the length of a phone call in minutes is an exponential r.v. X with
parameter  = 0.5= 0.1. If someone arrives at a phone booth just before you
arrive, find the probability that you will have to wait
(a) less than 5 minutes, and (b) between 5 and 10 minutes.
8. A production line manufactures 1000-ohm (R) resistors that have 10 percent
tolerance. Let X denote the resistance of a resistor. Assuming that X is a normal
r.v. with mean 500 and variance 1000, find the probability that a resistor picked
at random will be rejected.
2|Page
9. Binary data are transmitted over a noisy communication channel in block of
16 binary digits. The probability that a received digit is in error as a result of
channel noise is 0.01. Assume that the errors occurring in various digit
positions within a block are independent.
(a) Find the mean and the variance of the number of errors per block.
(b) Find the probability that the number of errors per block is greater than or
equal to 4.
10. Let the continuous r.v. X denote the weight (in pounds) of a package. The
range of weight of packages is between 30 and 100 pounds.
(a) Determine the probability that a package weighs more than 50 pounds.
(b) Find the mean and the variance of the weight of packages.
11. Consider the experiment of observing a sequence of Bernoulli trials until
exactly r successes occur. Let the r.v. X denote the number of trials needed to
observe the rth success. The r-v. X is known as the negative binomial r.v. with
parameter p, where p is the probability of a success at each trial.
(a) Find the pmf of X. (b) Find the mean and variance of X.

Multiple Random Variables

1. Consider a bivariate r.v. (X, Y). Find the region of the xy plane corresponding
to the events

A = X + Y ≥ 3 B = X ! + Y ! < 3
C = min (X, Y) ≤ 3 D = max (X, Y) ≤ 3
2. The joint cdf of a bivariate r.v. (X, Y) is given by
234 ( 267 )
/1 − 1 5 1 − 1 ≥ 0,  ≥ 0, 8, 9 > 0 B
+,- ( , ) = .
0 ;<ℎ1>?@A1
(a) Find the marginal cdf's of X and Y.
(b) Show that X and Y are independent.
(c) find
( ≤ 5,  ≤ 3), ( ≤ 3), ( > 3),  ( > ,  > ).

3|Page
3. Verify each of the following

4. Consider the binary communication channel shown in Fig. 2. Let (X, Y) be a

bivariate r.v., where X is the input to the channel and Y is the output of the
channel. Let ( = 0) = 0.5, ( = 1| = 0)B = 0.1  ( = 0| = 1)B = 0.2
(a) Find the joint pmf's of (X, Y).
(b) Find the marginal pmf's of X and Y.
(c) Are X and Y independent?
(d) Find the mean and the variance of X.
(e) Find the mean and the variance of Y.
(f) Find the covariance of X and Y.
(g) Find the correlation coefficient of X and Y.

Fig 2: Binary communication channel

4|Page
5. Consider an experiment of drawing randomly three balls from an urn
containing two red, three white, and four blue balls. Let (X, Y) be a bivariate
r.v. where X and Y denote, respectively, the number of white and blue balls
chosen.
(a)Find the range of (X, Y).
(b)Find the joint pmf's of (X, Y).
(c) Find the marginaI pmf's of X and Y.
(d) Are X and Y independent?
6. The joint pmf of a bivariate r.v. (X, Y) is given by
k (3 xi + 5 y j ) , xi = 1, 2,3; y = 1, 2
PXY ( xi , y j ) = 
0 , otherwise
where k is a constant.
(a) Find the value of k.
(b) Find the marginal pmf's of X and Y.
(c) Are X and Y independent?
(d) Find the conditional pmf’s B-|, (BE F G )  B,|- (B G |E )
(e) find ( = 1| = 1)B  ( = 1| = 1)B
(f) Find the mean and the variance of X.
(g) Find the mean and the variance of Y.
(h) Find the covariance of X and Y.
(i) Find the correlation coefficient of X and Y.
7. Let (X, Y, Z) be a trivariate r.v. with joint pdf
2(L4MN7MOP)
H,-I ( , , J) = . K1 > 0,  > 0, J > 0 B
0 ;<ℎ1>?@A1
where k a constant.
(a) Determine the value of k.
(b) Find the marginal joint pdf of X and Y.
(c) Find the marginal pdf of X.
(d) Are X, Y, and Z independent?

5|Page
 Functions of Random Variables
1. A voltage V is a function of time t and is given by Q (<) = R;AS< + A@S<
in which S is a constant angular frequency and =  = T(0; V ! ) and they are
independent.
(a). Show that Q (<) may be written as Q (<) = WR;A(S< − X).
(b). Find the pdf’s of r.v.’s W  X and show that they are independent.
2. Let Z=aX+ bY,where a and b are constants.
(a). Show that Y (Z) = Y ( ) + [Y()
(b). Find the correlation & covariance of X and Y.
(c). Find the correlation coefficient of X and Y.
3. Let X and Y be defined by = R;A\  = A@\
where \ is a random variable uniformly distributed over (0,2]).
(a) Show that X and Y are uncorrelated.
(b) Show that X and Y are not independent.
4. Let X be a Bernoulli r.v.
(a) Find the moment generating function of X.
(b) Find the mean and variance of X.
5. Let X be a binomial r.v. with parameters (n, p).
(a) Find the moment generating function of X.
(b) Find the mean and variance of X.
6. Let X be a Poisson r.v. with parameter A.
(a) Find the moment generating function of X.
(b) Find the mean and variance of X.
7. Let X be an exponential r.v. with parameter 1.
(a) Find the moment generating function of X.
(b) Find the mean and variance of X.
8. Find the moment generating function of the standard normal r.v. X = N(0; 1)
and calculate the first three moments of X.

6|Page