Final Practice
Final Practice
This is a collection of practice questions, gathered randomly from previous exams and
quizzes. They may not be representative of what will be on the final. In particular, topics
not represented in this list are still likely to be on the exam.
1. The phrase “peck pickled peppers” has the following letter frequencies:
c d e i k l p r s
2 2 1 4 1 2 1 5 1 1
Solution: View these frequencies as probabilities. The answer is not unique; just
make sure you follow the algorithm described in the notes.
Solution: 001101.
This matrix satisfies the conditions for being 1-error correcting: considering just
the 3 columns, all rows are distinct and have ≥ 2 1’s.
3. Let S1 be a source which outputs independent letters from the alphabet A = {a, b, c, d, e}
with probabilities pa = 1/8, pb = 1/16 pc = 1/2 pd = 1/4 and pe = 1/16. Let S2 be
a source which outputs independent letters from the alphabet Z = {w, x, y, z} with
probabilities pw = 1/4, px = 1/4 py = 1/4 pz = 1/4. Suppose one wants to compress
the random sequence coming out of these sources. Which source can be compressed to
shorter code in average?
Solution: You just need to compute the entropies of both sources; the one with
lower entropy is the more compressible.
4. Consider a source which outputs independent random letters from the alphabet A =
{a, b, c, d, e} with probabilities pa = 1/4, pb = 1/4, pc = 1/6, pd = 1/6 and pe = 1/6.
(a) Give a Huffman code for this source.
a c b
d e
(b) Let Ln be the random length of the Huffman code for a random sequence of n letters
from that source. Compute the expectation E(Ln ).
So E(Ln ) = 7n/3.
(c) Let L0n be the (random) length of the Lempel-Ziv code for a random sequence of
0
n letters from that source. Say if limn→∞ E(Ln n ) and limn→∞ E(Ln n ) are equal, and if
not which one is larger.
Solution:
E(L0n ) E(Ln )
lim = H < lim
n→∞ n n→∞ n
Page 2
5. For the sequence
0101101011101101110111
construct the Lempel-Ziv parsing of it (i.e., its decomposition into phrases). How would
it be encoded by the Lempel-Ziv code?
Solution: (Assume that the input alphabet is {0, 1}.) The decomposition into
phrases is
|0|1|01|10|101|11|011|0111|0111.
This yields the following list of reference-alphabet pairs:
0, 0 0, 1 1, 1 2, 0 4, 1 2, 1 3, 1 7, 1 7.
(The last phrase has no alphabet part, since it’s a repeated phrase.) Encoding each
dictionary reference with the correct number of bits gives
6. Consider the simple encoding scheme which simply repeats every bit of the input 3 times,
and decoded by taking the majority vote.
(a) What is the rate of this encoding scheme?
Solution: 1/3.
(b) Suppose a message of length k is encoded and sent through a binary symmetric
channel which flips each bit with probability p. What is the probability that a
message sent using this encoding scheme is decoded correctly?
(c) Suppose p = 0.125. Is it true that for k (the message length) large enough, there
exists an encoding scheme with the same rate as the above simple encoding scheme
and which decodes correctly with a probability of at least 99%? Explain why or
why not.
Solution: The entropy of the source is H(p) = − 18 log2 ( 81 )− 78 log2 ( 78 ) ≈ 0.377 <
0.5. Thus the channel capacacity is at least 1 − H(p) ≥ 0.5. Thus 1/3 is above
Page 3
the capacity, and by Shannon’s theorem there exist good codes with the required
property.
9. Let p be an odd prime. Consider the group (Z∗p , ×), the group operation being multi-
plication modulo p. Let S be a subset of size (p − 1)/2. Show that for some element
a ∈ Z∗p , a ∈ S and a−1 ∈ S.
10. Alice chooses primes p = 5 and q = 11. The public key she generates is the pair (55, 7)
and the private key is (55, 23). Bob wants to encrypt the message m = 53 and send it
to Alice. What is the encrypted message?
11. Here is a modified RSA protocol. Instead of using a composite number N = pq in the
RSA cryptosystem, we just use a prime p as the main modulus. We further choose an
encryption exponent e which is relatively prime with p − 1,yielding a public key (p, e).
A message m is then encoded as s = me mod p. Show that this system is not secure
by giving a simple (and efficient) algorithm that, given p, e and s recovers m (where
s = me mod p).
Page 4
Solution: Compute the inverse d of e modulo p − 1 using the Euclidean algorithm
(the inverse exists since gcd(e, p − 1) = 1). So de = k(p − 1) + 1 for some integer k.
Then compute sd mod p:
12. In implementing Pollard’s rho algorithm, a colleague suggests that the following alter-
native algorithm might be faster:
Modified tortoise and hare algorithm:
1. Let y0 = x0 .
2. For i = 1, 2, . . .
(a) xi = f (xi−1 ).
(b) yi = f (f (f (yi−1 ))) ← the difference is here
(c) If gcd(xi − yi , n) 6= 1, return this discovered factor.
(Recall f (x) = x2 + 1 mod n, and x0 is some random starting element in Zn .) Does this
work? Why, or why not? If it does work, is it faster?
Solution: This does not necessarily work. Not only might the hare jump over the
tortoise, but they might never meet! Consider a cycle of length 6, label the vertices
1 through 6 in order. Suppose that at time 1, the hare is at node 3, and the tortoise
at node 2. Then the hare will be at position 3 at every odd time, and 6 at every
even time. But the tortoise will always be on an even position at odd times and an
odd position at even times, so they will never be at the same position at the same
time. Good news for the tortoise.
Suppose
2 + 2x
F (x) = .
1 − 2x − x2
Give an expression for an . Can you give a recursion for an and initial conditions that
would give rise to this generating function?
Page 5
Solution: Expanding as partial fractions,
1 1
F (x) = − ,
λ− − x λ+ + x
√
where λ± = 2 ± 1. Hence
∞ ∞
1 X n 1 X
F (x) = (x/λ− ) − (x/λ+ )n .
λ− n=0 λ+ n=0
So
an = λ−n−1
− + λ−n−1
+ .
Hence ∞ ∞ ∞
X X X
n n
an x − 2 an−1 x − an−2 xn = 2 + 2x.
n=0 n=1 n=2
This gives
14. Suppose we have two polynomials p(x) = di=0 pi xi and q(x) = di=0 qi xi , each of degree
P P
d and with integer coefficients satisfying |pi | ≤ 2 and |qi | ≤ 2 for all i. Describe how one
can use fast discrete fourier transforms to compute the coefficients of p(x)q(x). Specify
how you would select the required parameters.
15. Let n be a positive integer. Give a concise formula for the number of sequences a1 <
a2 < · · · < an of integers that satisfy 1 ≤ ai ≤ 2i for each 1 ≤ i ≤ n.
Page 6
Solution: This one is tricky! First, let’s write bi = ai − i for each i. Then bi+1 − bi =
ai+1 − ai − 1, so b1 ≤ b2 ≤ · · · ≤ bn (but note that equality is now possible). Also,
ai ≥ i (since a1 ≥ 1, so a2 > a1 ≥ 1, etc.). Hence 0 ≤ bi ≤ i.
Now the trick is to realize that these are counted by the Catalan numbers; the answer
is Cn+1 ; you might discover this by computing the first few values. So the goal is
to find a bijection to some objects counted by the (n + 1)’st Catalan number. Here
is one way. Plot the points (1, b1 ), (2, b2 ), . . . , (n, bn ) on the plane. Note that all the
points lie below the line y = x. Now draw a path with only horizontal and vertical
steps, starting from (0, 0) and ending at (n+1, n+1), that stays below the line y = x.
We begin with a step to the right, and then move up if necessary to touch the point
(1, b1 ). Then take another step to the right, and again move up if necessary to reach
(2, b2 ), etc., with a final step to the right and then vertical segment after (n, bn ) to
reach (n1 , n + 1). You can prove that this map is a bijection. Since the number of
such paths is counted by the Catalan numbers (as you saw in class), you’re done.
16. Let a0 , a1 , . . . , an−1 be real numbers. Let c0 , c1 , . . . , cn−1 be the coefficients associated
to the sequence a0 , a1 , . . . , an−1 by the Discrete Fourier Transform. Let c0k = ck for all
k = {0, . . . , n − 1} except c01 = c1 + 1. Let a00 , a01 , . . . , a0n−1 be the numbers obtained from
c00 , c01 , . . . , c0n−1 by the inverse Discrete Fourier Transform.
Express a00 , a01 , . . . , a0n−1 in terms of a0 , a1 , . . . , an−1 .
17. Let ~y = (y0 , y1 , . . . , yn−1 ) be a given vector of n complex numbers, and let ~c = (c0 , c1 , . . . , cn−1 )
be the discrete Fourier transform of ~y . Prove that
n−1
X n−1
X
2
|yj | = |ck |2 .
j=0 k=0
Page 7
Solution: Let z = e−2πı/n . Since |ck |2 = ck c¯k , we have
n−1
X n−1 X
X n−1 n−1
X
2
|ck | = yj z jk
ȳ` z −`k
k=0 k=0 j=0 `=0
n−1 X
X n−1 n−1
X
= yj ȳ` z jk z −`k
j=0 `=0 k=0
n−1 X
X n−1
= yj ȳ` nδj,`
j=0 `=0
n−1
X
= yj ȳj
j=0
n−1
X
= |yj |2 .
j=0
min 6x − 3y − z
s.t. 4x − 2y + z = 4
x + 3y − z ≥ 2
x, y, z ≥ 0.
Solution:
max 4α + 2β
s.t. 4α + β ≤ 6
−2α + 3β ≤ −3
α − β ≤ −1
α∈R
β ≥ 0.
19. Consider the following flow problem instance, with source s and sink t:
Page 8
2 1 4
6 8
7 2
s t
5
7
2
3 3 5
Find a maximum flow, and give an argument that there is no flow of any larger value.
Solution: The maximum flow has value 10 (you should be able to find such a flow);
the set S = {s, 2, 3, 5} has capacity 10, so it cannot be larger.
20. Solve the following LP using the simplex method, showing your tableau and choice of
pivot clearly at each step, and writing the final answer and objective value clearly.
maximize 3x1 + 2x2 + 4x3
subject to x1 + x2 + 2x3 ≤4
2x1 + 3x3 ≤5
2x1 + x2 + 3x3 ≤7
x1 , x2 , x3 ≥ 0.
Solution: After adding slack variables x4 , x5 and x6 , the starting tableau is:
−z x1 x2 x3 x4 x5 x6
1 3 2 4 0
1 1 2 1 4
2 0 3 1 5
2 1 3 1 7
After one pivot (in column x3 , 2nd row)
−z x 1 x2 x3 x4 x5 x6
1 1/3 2 −4/3 −20/3
−1/3 1 1 −2/3 2/3
2/3 0 1 1/3 5/3
0 1 −1 1 2
Next pivot is column x2 , row 1:
−z x 1 x2 x3 x4 x5 x 6
1 1 −2 0 −8
−1/3 1 1 −2/3 2/3
2/3 1 0 1/3 5/3
1/3 −1 −1/3 1 4/3
Page 9
Finally, pivot on column x1 , row 2:
−z x1 x2 x3 x4 x5 x6
1 −1.5 −2 −0.5 −10.5
1 0.5 1 −0.5 1.5
1 1.5 0 0.5 2.5
−0.5 −1 −0.5 1 0.5
At this point, all coefficients in the objective row are nonpositive, so we are finished.
The objective value is 10.5, and the optimal solution is
(In terms of the original problem without slack variables, we only need to specify
x1 = 2.5, x2 = 1.5 and x3 = 0.)
21. Consider the matrix game based on the following payoff matrix:
0 −2 1
A= 2 0 3 .
−1 −3 0
max g
s.t. − 2y2 + y3 ≥ g
−2y1 − 3y2 ≥ g
y1 + 3y2 ≥ g
y1 + y2 + y3 = 1
y1 , y2 , y3 ≥ 0
g∈R
Page 10
For player II, you should get
min l
s.t. − 2x2 + x3 ≤ l
−2x1 − 3x2 ≤ l
x1 + 3x2 ≤ l
x1 + x2 + x3 = 1
x1 , x2 , x3 ≥ 0
g∈R
(b) Using part 1 and using complementary slackness, find the optimal strategies for
both players.
Solution: Player I can play row 2 only (i.e., with probability 1) and symmetri-
cally, player II can play column 2 only. This gives player I a guaranteed expected
payoff of 0, and similarly player 1 a guaranteed expected loss of 0. Since these
are equal, both strategies are optimal.
(Complementary slackness is not needed in the above argument.)
22. Consider the problem of tiling a rectangle of dimension n × 1 with tiles of three types:
white tiles of dimension 1 × 1, gray tiles of dimension 2 × 1 and black tiles of dimension
2×1. An example is given in Figure 1. Let an be the number of ways of tiling a rectangle
∞
X
of dimension n × 1 (by convention a0 = 1), and let A(x) = an x n .
n=0
Solution: a1 = 1, a2 = 3, a3 = 5.
Page 11
(b) Find an expression for the generating function A(x). (You do not need to find a
formula for the coefficients an .)
where C(x) is the generating function of the finite set C = {white tile, gray tile, black tile}.
Since C contains 1 element of size 1 and 2 elements of size 2 one gets C(x) =
x + 2x2 and
1
A(x) = .
1 − x − 2x2
Page 12