EWSCS’06 Palmse, Estonia

5-10 March 2006

Shannon’s Theory of Secrecy

Lecture 1:

and its Extension to Authenticity

James L. Massey

Prof.-em. ETH Zürich, Adjunct Prof., Lund Univ.,

Sweden, and Tech. Univ. of Denmark
Trondhjemsgade 3, 2TH
DK-2100 Copenhagen East

[email protected]

1
 Cryptology
(“hidden word”)

Cryptography Cryptanalysis
(code making) (code breaking)

The “good guys” The “bad guys”

2
 Goals of cryptography

Secrecy Authenticity

Xuejia Lai has given a useful razor for

deciding whether something is a matter of
secrecy or a matter of authenticity.

3
 Secrecy - concerned with who has access to
(or can read) a legitimate message.

Secrecy deals with safeguarding the future by

ensuring that only authorized recipients will be able
to gain access to (or read) a legitimate message.

4
Authenticity - concerned with who can create
(or write) a legitimate message.

Authenticity deals with protecting the past by

• ensuring that the creator (or author) was
entitled to create (or write) the message
• ensuring that the contents of the message
have not been altered

5
 A secrecy scheme: The Caesar Cipher
0
25 1 Arithmetic on a CIRCLE
24 A (Modulo 26 arithmetic)
Z B 2
Y Encrypt = Add 3
23 C
. (move clockwise 3 places)
.
X
. Decrypt = Subtract 3
.. . (move counterclockwise 3 places)
SECRET KEY = 3

C A E S A R plaintext
F D H V D U ciphertext

M A S S E Y plaintext
P D V V H B ciphertext 6
 Today we use a SMALLER CIRCLE!

Arithmetic on this CIRCLE

0 (Modulo 2 arithmetic)
Encrypt = Add
(move clockwise)
Decrypt = Subtract
(move counterclockwise)
= (move clockwise)

1 ⇒ Decrypt = Encrypt

and a LONGER SECRET KEY!

1 0 0 1 1 1 0 1 plaintext
0 1 1 0 0 1 1 1 secret key
1 1 1 1 1 0 1 0 ciphertext
7
Everybody likes to make secret codes!

8
Photograph of
Shannon at home
in 1962. (from the
New York Times
Magazine, 30
December 2001)

9
“As a first step in the mathematical analysis of
cryptography, it is necessary to idealize the
situation suitably, and to define in a
mathematically acceptable way what we shall
mean by a secrecy system.”
C.E. Shannon, "Communication Theory of Secrecy Systems",
Bell System Tech. J., vol. 28, pp. 656-715, Oct., 1949.

This was a radical departure from previous

papers in cryptography where (as in Steen and
Stoffer) conjecture and imprecision reigned.

Just how did Shannon define a secrecy system in

“a mathematically acceptable way”?

10
He drew a picture!

(Shannon, 1949)
11
 Claude Elwood Shannon (1916-2001)
(photographed 17 April 1961 by Göran Einarsson) 12
Shannon’s
Fig. 1—Schematic of a general secrecy system
makes the following assumptions crystal clear:
• The message M and the key K are independent
random variables.
• The sender and receiver both know the key.
• The attacker knows only the cryptogram E (i.e., a
ciphertext-only attack is assumed).
• The receiver is able to recover the message M from
knowledge of the cryptogram E and key K.
• No assumption is made about who generates the key.

You don’t need a lot of words and/or equations

to make yourself mathematically precise!
13
 Kerckhoffs’ Principle

A cipher should be secure when the enemy

cryptanalyst knows all details of the
enciphering process and deciphering process
except for the value of the secret key.

This principle was first stated in 1881 by the

Dutchman Auguste Kerckhoffs (1835 - 1903).

When evaluating security, one assumes that

the enemy cryptanalyst knows everything
(including the source statistics and key
statistics) except the secret key.
14
 What does “unbreakable” mean?
To Shannon, a cipher is unbreakable in a
ciphertext-only attack if it provides
unconditional security, i.e., no matter how hard
or how long the attacker works, he/she can do
no better than to guess the plaintext by
the best guessing rule that he/she would
use without having seen the ciphertext.

Shannon ‘s 1949 definition: A cipher provides

perfect secrecy against a ciphertext-only
attack if the plaintext and the ciphertext,
considered as random variables, are independent.

15
 Vernam’s 1926 Cipher:
Enemy cryptanalyst
in a ciphertext-only
attack.

Binary M E M
Plaintext Destination
Source R R
Secure Channel
R
R is the secret key,
a “totally random” BSS
sequence. Binary Symmetric Source

Vernam claimed that his cipher was unbreakable!

16
Vernam not only claimed that his cipher
was unbreakable, but also stated that he
had confirmed this in “field trials with
the U. S. Army Signal Corps”.

Was Vernam right? Was his

cipher the first unbreakable
cipher in the many thousands
of years of cryptographic
history?

17
The Binary Symmetric Source (BSS) of
information theory is a monkey with a fair
binary coin ( 0 on one side and 1 on the other).

18
 Cryptographic property of the BSS:

The modulo-two sum of a BSS output and

an arbitrary random sequence is another
BSS output that is INDEPENDENT of
the arbitrary random sequence.

Example:
BSS output: 0 1 0 0 1 0 1 0 1 1 1 0 1 ...
Arb. Ran. Seq. 1 1 1 1 1 1 1 1 1 1 1 1 1 ...
Modulo-2 sum 1 0 1 1 0 1 0 1 0 0 0 1 0 ...

19
 Vernam’s cipher provides perfect secrecy
against a ciphertext-only attack!

Binary M E M
Plaintext Destination
Source R R
Secure Channel
R
BSS
The cryptogram E that the enemy cryptanalyst
sees is independent of the plaintext message M.
This simple proof of unbreakability of Vernam’s
1926 cipher was first given by Shannon in 1949! 20
Vernam’s cipher is usually today called the
“one-time pad” to emphasize that the key is
to be used for only one message . It was
used by spies on both sides in World War II
and is still the cipher of choice for
extremely important secret communications.

(What Shannon called the plaintext is the

total data that will be encrypted before the
key is changed, i.e., Shannon specified a
“one-time key” in his theory of secrecy
systems.)

21
Vernam’s cipher needs as many binary
digits of secret key as there are bits of
plaintext to be encrypted.

Vernam was right about his cipher being

unbreakable, but does an unbreakable
cipher really need this huge amount of
secret key???

22
 Shannon’s 1949 Lower Bound on Key Length:
For perfect secrecy, the number of different
keys must be AT LEAST AS GREAT as the
number of different plaintexts.

Proof:
• For any fixed key k, the number of different ciphertexts
e equals the number of different plaintexts m.
• Perfect secrecy ⇒ for all possible e and any fixed m,
P(E=e|M=m) = P(E=e) ≠ 0
• ⇒ For a fixed m, the number of different ciphertexts e
must equal at least the number of different plaintexts m.
• But all keys from a fixed m to different e’s must be
different.

23
Shannon later gave the following proof of a slightly
weaker lower bound on key length, namely

H(K) ≥ H(E).

Perfect secrecy ⇒ H(E) = H(M|E)

≤ H(MK|E)
= H(K|E) + H(M|EK)
= H(K|E)
=0
≤ H(K)

Thus, if the cipher is to give perfect secrecy

regardless of the source statistics, it must also
give perfect secrecy for the BSS for which
H(M) = N bits. Thus H(K) ≥ N so that the key
must be at least N binary digits long.
24
The number of different plaintext messages is about
2H(M) where H(M) is the entropy of the plaintext
message. Equivalently, one says that H(M) is the
number of bits of information in the plaintext
message. An ideal data compressor will compress M
to about H(M) binary digits. Consider the system:

Ideal Data Vernam’s

M Compressor Cipher E

K
Achieves perfect secrecy and the number of
binary digits of the key K is H(M), which
satisfies Shannon’s lower bound with equality.
25
 Simmons’ Model of a
Substitution Attack on an Authenticity System

ACCEPTED
MESSAGE M ENCIPHERER
E E' DECIPHERER
SOURCE TK TK-1 M'
ENEMY CRYPTANALYST
K K
Secure Channel
K
RECOGNIZED
KEY UNAUTHENTIC
SOURCE

E' can be the legitimate cryptogram E or a phony

cryptogram E' (E' ≠ E) inserted by the attacker.

E' is accepted if and only if it is a valid cryptogram

for the key K.
26
 In an impersonation attack, the attacker forms E'
without seeing a legitimate cryptogram E and wins if
his cryptogram is accepted.
PI = Probability of successful impersonation when the
attacker uses an optimum attack.
PS = Probability of successful substitution when the
attacker uses an optimum attack.
Pd = Probability of deception = max(PI, PS)

Simmons’ 1984 bound on the probability of deception:

Pd ≥ 2-I(E; K)
where I(E; K) = H(K) - H(K|E) is the mutual
information between E and K.

The only way to get unconditionally secure authenticity

is to let the cryptogram give information about the key!
27
 Example of an authenticity system meeting
Simmon’s lower bound on PI with equality:

Plaintext is sent in the clear and the key

is added as a signature: E = [M : K]

If the key has length n binary digits, then

PI = 2-n
because the attacker can only make a random
guess at the secret key in an impersonation attack.
I(E; K) = n bits so that Simmons’ bound on PI
holds with equality!
This authenticity system gives no secrecy!
In a substitution attack, the attacker can achieve
PS = 1.
28
 Example of an authenticity system meeting
Simmon’s lower bound on PS and Pd with equality:
1-bit messages with individual signatures.
K = (K1, K2, . . . Kν , Kν +1, . . . K2ν ) [n = 2ν-bit
key] assumed generated by a BSS.
M is 0 or 1.
M = 0 ⇒ E = (0, K1, K2, . . . Kν )
M = 1 ⇒ E = (1, Kν +1, Kν +2, . . . K2ν )
Note that again there is no secrecy!
Whether the attacker observes E or not, he must
guess ν bits of key to produce a cryptogram E'
that will be accepted as authentic.
⇒ PI = PS = Pd = 2-ν.
But I(E; K) = ν bits so that Simmons’ bound on Pd
holds with equality! 29
This example shows that we can have unconditionally
secure authenticity with no secrecy.

Vernam’s cipher gives perfect secrecy against a

ciphertext-only attack but no protection against
an impersonation attack, i.e., PI = 1.

The important conclusion to make is that

secrecy and authenticity are independent
attributes of a cryptographic system.

30
The informational divergence (or the “Kullbach-
Leibler distance” or the “relative entropy” or the
“discrimination”) from P to Q, two probability
distributions on the same alphabet, is the quantity
Q(x)
D ( P || Q ) = − ∑ P(x) log .
x∈supp ( P ) P(x)

Fundamental property of informational divergence:

D ( P || Q ) ≥ 0 with equality if and only if P = Q.

Let H0 and H1 be the two possible hypotheses and let

Y be the observation used to determine which
hypothesis is true. Let D0 and D1 be the regions of Y
values in which one decides for H0 or H1, respectively.
Let α or β be the error probabilities when H0 or H1 is
true, respectively.
31
 Let V (0 or 1) be the decision as to which
hypothesis is true so that
α = PV |H (1)
0
and β = PV |H (0) .
1

Direct calculation gives

1- β β
D( PV |H 0 || PV |H1 ) = −α log − (1 − α ) log .
α 1-α
Information-theoretic bound for hypothesis testing:
1- β β
D ( PY |H 0 || PY |H1 ) ≥ −α log − (1 − α ) log
α 1-α
PY|H 0 ( y)
with equality if and only if has the same
PY|H1 ( y)
value for all y ∈ D0 and has the same value for all y ∈ D1 .

32
For the important special case where α = 0, i.e., where
we never err when H0 is true, the previous bound gives
− D ( PY | H 0 || PY | H 1 )
β ≥2 .
Now suppose that H0 is the hypothesis that the
observation Y = E' is the legitimate cryptogram E for
the key K = k, i.e.,
PY |H 0 ( y ) = PE | K = k ( y ) , and that H1 is the hypothesis
that Y = E' is formed by the attacker according to
PY |H 1 ( y ) = PE ( y ) = ∑ PK = k ( k ) PE | K = k ( y ) ,
k
which may not be the optimum attacking strategy.
Let βk be the error probability when K = k so that
− D ( PE | K = k || PE )
βk ≥ 2 .
33
 PE ( y )
D ( PE |K = k || PE ) = − ∑ PE | K = k ( y ) log
PE | K = k ( y )
y
− ∑ D ( PE|K =k || PE )
β = ∑ PK (k )β k ≥ ∑ PK (k )2 k

k k
− ∑ PK ( k )D ( PE|K =k || PE )
≥2 k

where we have used Jensen’s inequality. But

∑ PK ( k )D ( PE|K =k || PE ) = H ( E ) − H ( E | K ) = I ( E ; K ) .
k
Moreover, β is just the probability PI of successful
impersonation, so the information-theoretic bound
becomes
PI ≥ 2 − I ( K ; E ) .
This completes the proof of Simmons’ lower bound. 34
Simmons’ proof of his bound on the probability of
deception (or impersonation) appears in
G. J. Simmons, "Authentication Theory/Coding Theory," pp. 411-431 in Advances in
Cryptology - CRYPTO '84 (Eds. G. R. Blakey and D. Chaum), Lecture Notes in
Computer Science No. 196. Heidelberg and New York: Springer, 1985.

Several simplifications of his derivation have since been

given. The most insightful one, which we have followed,
is by Maurer, cf.
U. M. Maurer, "Information Theoretic Bounds in Authentication Theory," p.12 in
Proc. IEEE Inst. Symp. Info. Th., Whistler, Canada, Sept. 17-22, 1995.
U.M. Maurer, "A Unified and Generalized Treatment of Authentication Theory, pp.
387-398 in Proc. 13th Symp. on Theoretical Aspects of Computer Science
(STACS'96), Lecture Notes in Computer Science No. 1046, New York: Springer,
1996.

Maurer based his treatment on Blahut’s information-

theoretic approach to hypothesis testing, cf.
R. E. Blahut, “Hypothesis testing and information theory”, IEEE Trans. Inform.
Theory, vol. IT-20, pp. 405-417, July 1974
35