Earlier version appears in Advances in Cryptology { CRYPTO '95. This is the full paper.

Bucket Hashing and its Application to Fast Message Authentication

Department of Computer Science, University of California, Davis, CA 95616. e-mail: [email protected]
July 18, 1997 Phillip Rogaway

Abstract. We introduce a new technique for constructing a family of universal hash functions.

At its center is a simple metaphor: to hash a string x, cast each of its words into a small number of buckets ; xor the contents of each bucket; then collect up all the buckets' contents. Used in the context of Wegman{Carter authentication, this style of hash function provides a fast approach for software message authentication.

Key words: Cryptography, Hashing, Message authentication codes, Universal Hashing.

1 Introduction
aims. The setting is that two parties, a signer S and veri er V , share a (short, random, secret) key, k. When S wants to send V a message, x, S computes for it a message authentication code (MAC), MACk (x), and S sends V the pair (x; ). On receipt of (x0; 0), veri er V checks that 0 MACVk (x ; 0) = 1. To describe the security of a message authentication scheme, an adversary E is given an oracle for MACk ( ). The adversary is declared successful if she outputs an (x ; ) such that MACVk (x ; ) = 1 but x was never asked of the MACk ( ) oracle. For a scheme to be \good," reasonable adversaries should rarely succeed. Software-efficient MACs. In the current computing environment it is often necessary to compute MACs frequently and over strings which are commonly hundreds to thousands of bytes long. Despite this, there will usually be no special-purpose hardware to help out: MAC generation and veri cation will need to be done in software on a conventional workstation or personal computer. So to reduce the impact of message authentication on the machine's overall performance, and to facilitate more pervasive use of message authentication, we need to develop faster techniques. This paper provides one such technique. Two approaches to message authentication. The fastest software MACs in common use today are exempli ed by MACk (x) = h(k k x k k), with h a (software-e cient) cryptographic hash function, such as h =MD5 22]. Such methods are described in 30]. The algorithm HMAC 3] 1
Message authentication. Message authentication is one of the most common cryptographic

represents the most re ned algorithm in this direction. Schemes like these might seem to be about as software-e cient as one might realistically hope for: after all, we are computing one of the fastest types of cryptographic primitives over a string nearly identical in length to that which we want to authenticate. But it is well-known that this reasoning is specious: in particular, Wegman and Carter 32] showed back in 1981 that we do not have to \cryptographically" transform the entire string x. In the Wegman{Carter approach communicating parties S and V share a secret key k = (h; P ) which speci es both an in nite random string P and a function h drawn randomly from a strongly universal2 family of hash functions H. (Recall that H is strongly universal2 if for all x 6= x0 , the random variable h(x) k h(x0), for h 2 H, is uniformly distributed.) To authenticate a message x, the sender transmits h(x) xor-ed with the next piece of the pad P . The thing to notice is that x is transformed rst by a non-cryptographic operation (universal hashing) and only then is it subjected to a cryptographic operation (encryption), now applied to a much shorter string. A standard cryptographic technique |the use of a pseudorandom function family, F | allows S and V to use a short string a in lieu of the in nite string P . Signer S now MACs the ith message, xi , with MAC(h;a)(xi ) = (i; Fa(i) h(xi )). As it turns out, to make a good MAC it is enough to construct something weaker than a strongly universal2 family. Carter and Wegman 10] also introduced the notion of an almost universal2 family, H. This must satisfy the weaker condition that Prh2H h(x) = h(x0)] is small for all x 6= x0. As observed by Stinson 27], an almost universal2 family can easily be turned into an almost strongly universal2 family by composing the almost universal2 family with an almost strongly universal2 one. Thus the problem of nding a fast-to-compute MAC has e ectively been reduced to nding a family of almost universal2 hash functions whose members are fast to compute. Bucket hashing. This paper provides a new almost universal2 family of hash functions. We call our hash family bucket hashing. It is distinguished by its member functions being extremely fast to compute|as few as 6 elementary machine instructions per word (independent of word size) for the version of bucket hashing we concentrate on in this paper. Putting such a family of hash functions to work in the framework described above will give rise to an e cient software MAC. A bucket hash MAC will involve signi cant overhead beyond the time which is spent bucket hashing. For one thing, the output of bucket hashing is too long to use directly; it will need to be composed with an additional layer of hashing. All the same, one can compare the instruction count mentioned above to that of MD5, which uses 36 instructions per 32-bit word 7], and see that there is potential for substantial e ciency gains even if the true cost of using bucket hashing substantially exceeds 6 instructions/word. A bucket hash MAC has advantages in addition to speed. Bucket hashing is a linear function |it is a special case of matrix multiplication over GF(2)| and this linearity yields many pleasant characteristics for a bucket hash MAC. In particular, bucket hashing is parallelizable, since each word of the hash is just the xor of certain words of the message. Bucket hashing is incremental in the sense of 4] with respect to both append and substitute operations. Finally, the only processor instructions a bucket hash needs are word-aligned load, store, and xor; thus a bucket hash MAC is essentially endian-indi erent. In a bucket hash MAC |indeed in any Wegman-Carter MAC| one is a orded the luxury of conservative (slow) cryptography even in a MAC whose software speed has been aggressively optimized. This is because one arranges that the time complexity for the MAC is dominated by the non-cryptographic work. (Recent attacks on cryptographic hash functions MD4 and MD5 may make this seem like an enormous advantage compared to the keyed-cryptography-hash approach, except 2

that constructions such as HMAC are not impacted when collisions are found in its underlying hash function.) One might worry that the linearity or simple character of bucket hashing might give rise to some \weakness" in a MAC which exploits it. But it does not. A bucket hash MAC, like any MAC which follows the Wegman{Carter paradigm, enjoys the assurance advantages of provable security. Moreover, this provable security is achieved under extremely \tight" reductions, so that an adversary who can successfully break the MAC can break the underlying cryptographic primitive (the pseudorandom function F ) with essentially identical e ciency. Previous work. The general theory of unconditional authentication was developed by Simmons; see 26] for a survey. As we have already explained, the universal-hash-and-then-encrypt paradigm is due to Wegman and Carter 32]. The idea springs from their highly in uential 10]. In Wegman{Carter authentication the size of the hash family corresponds to the number of bits of shared key|one reason to nd smaller families of universal hash functions than those of 10, 32]. Stinson does this in 27], and also gives general results on the construction of universal hash functions. We exploit some of these ideas. Subsequent improvements (rooted in coding theory) came from Bierbrauer, Johansson, Kabatianskii and Smeets 6], and Gemmell and Naor 12]. The above work concentrates on unconditionally-secure authentication. Brassard 9] rst connects the Wegman{Carter approach to the complexity-theoretic case. The complexity-theoretic notion for a secure MAC is a straightforward adaptation of the de nition of a digital signature due to Goldwasser, Micali and Rivest 14]. Their notion of an adaptive chosen message attack is equally at home for de ning an unconditionally-secure MAC. Thus we view work like ours as making statements about unconditionally-secure authentication which give rise to corresponding statements and concrete schemes in the complexity-theoretic tradition. To make this translation we regard a nite pseudorandom function (PRF) as the most appropriate tool. Bellare, Kilian and Rogaway 5] were the rst to formalize such objects, investigate their usage in the construction of e cient MACs, and suggest them as a desirable starting point for practical, provably-good constructions. Finite PRFs are a re nement of the PRF notion of Goldreich, Goldwasser and Micali 13] to take account of the xed lengths of inputs and outputs in the e cient primitives of cryptographic practice. Zobrist 33] gives a hashing technique which predates 10] and essentially coincides with one method from 10]. Arnold and Coppersmith 2] give an interesting hashing technique which allows one to map a set of keys ki into a set of corresponding values vi using a table only slightly bigger P than i vi . The proof of our main technical result is somewhat reminiscent of their analysis. Lai, Rueppel and Woolven 20], Taylor 28], and Krawczyk 18] have all been interested in computationally e cient MACs. The last two works basically follow the Wegman{Carter paradigm. In particular, Krawczyk obtains e cient message authentication codes from hash families which resemble traditional cyclic redundancy codes (CRCs), and matrix multiplication using Toeplitz matrices. Though originally intended for hardware, these techniques are fast in software, too. We recall Krawczyk's CRC-like hash in Section 2. An earlier version of this paper appeared as 23]. Subsequent work. Shoup 24] has carried out implementations and analysis of hash function families akin to polynomial evaluation. Such hash functions make good candidates for \second level hashing" when a speed-optimized hash function is applied to a long string. The techniques are also fast enough to be gainfully employed all by themselves. Halevi and Krawczyk describe a family of hash functions, MMH, which achieves extremely impressive software speeds on some modern platforms 15]. To achieve such performance one needs 3

the underlying hardware to be able to quickly multiply two 32-bit integers to form a 64-bit product. Johansson investigates how to reduce the size of the key for bucket hashing, which, in the current paper, is quite enormous 16].

2 Preliminaries
This section provides background drawn from Carter and Wegman 10, 32], Stinson 27], and Krawczyk 18]. Proofs are omitted. A family of hash functions is a nite multiset H of string-valued functions, each h 2 H having the same nonempty domain A f0; 1g and range B f0; 1gb , for some constant b.

Pr h(x) h(x0) = c h2H

De nition 1 10] A family of hash functions H = fh : A ! f0; 1gbg is -almost universal , written -AU , if for all distinct x; x0 2 A, hPr h(x) = h(x0 ) . The family of hash functions H 2H is -almost XOR universal , written -AXU , if for all distinct x; x0 2 A, and for all c 2 f0; 1gb ,
2 2

The value of = maxx6=x0 fPrh h(x) = h(x0)]g is called the collision probability. For us, the principle measures of the worth of an AU2 hash family are how small is its collision probability and how fast can one compute its functions. To make a fast MAC one may wish to \glue together" various universal hash families. The following are the basic methods for doing this. First we need a way to make the domain of a hash family bigger. Let H = fh : f0; 1ga ! f0; 1gb g. By Hm = fh : f0; 1gam ! f0; 1gbm g we denote the family of hash functions whose elements are the same as in H but where h(x1 x2 xm ), for jxi j = a, is de ned by h(x1 ) k h(x2) k k h(xm ).

Proposition 2 27] If H is -AU then Hm is -AU .

2 2

Sometimes one needs a way to make the collision probability smaller. Let H1 = fh : A ! f0; 1gb1 g and H2 = fh : A ! f0; 1gb2 g be families of hash functions. By H1 &H2 = fh : A ! f0; 1gb1 +b2 g we mean the family of hash functions whose elements are pairs of functions (h1 ; h2) 2 H1 H2 and where (h1 ; h2)(x) is de ned as h1 (x) k h2(x).

Proposition 3 If H is -AU and H is -AU then H &H is -AU . Next is a way to make the image of a hash function shorter. Let H = fh : f0; 1ga ! f0; 1gb g and H = fh : f0; 1gb ! f0; 1gcg be families of hash functions. Then by H H = fh : f0; 1ga ! f0; 1gcg we mean the family of hash function whose elements are pairs of functions (h ; h ) 2 H H
1 1 2 2 2 2 1 2 1 2 2 1 2 2 1

and where (h1; h2 )(x) is de ned as h2 (h1(x)).

1 1 2 2

Proposition 4 27] If H is -AU and H is -AU then H H is ( + )-AU . Composition can also be used to turn an AU family H whose members hash A to B , and an AXU family H whose members hash B to C , into an AXU family H H whose members hash A to C . If B = f0; 1gb for some small b, and elements of H are fast to compute on this domain, we have e ectively \promoted" H from being AU to AXU at little cost.
2 2 2 1 1 2 2 2 1 2 2 2 2 1 2 1 2 2

Proposition 5 27] Suppose H = fh : A ! Bg is -AU , and H = fh : B ! C g is -AXU . Then H H = fh : A ! C g is ( + )-AXU .

1 1 2 2 2 2 2 1 1 2 2

We end this section with a sample construction for a software-e cient AXU2 hash family, this one due to Krawczyk 18]. Let n; ` 1 be numbers and let m 2 f0; 1gn` be the string we wish to hash. We can view m as a polynomial m(x) over GF(2) of degree n` 1 (or less) by viewing the bits of m as the coe cients of xn` 1 ; : : :; x2; x; 1. We then de ne a family of hash functions K n; `] = fh : f0; 1gn` ! f0; 1g`g as follows. A random hash function h 2 K is described by a random irreducible polynomial h over GF(2) of degree `. To hash m using h we compute the degree ` 1 (or less) polynomial m(x) x` mod h(x). Viewing the coe cients of this polynomial as a string of length ` gives us the hash function h evaluated at M .

Theorem 6 18] K n; `] is n`` 1` -AXU .

2 + 2

The e ciency with which hash functions h 2 K can be computed has been studied by Shoup 24] (who also looked at related hash families). These functions are fast to compute| about 6 instructions/byte on a 32-bit machine, assuming ` = 64, and ignoring the time to \preprocess" the function h. Still, for su ciently long messages, it will be faster to use the bucket hashing technique from the following section. We comment that there are many other well-known techniques for universal hashing, such as the linear congruential hash (modulo a prime) 10], the shift register hash 31], or the Toeplitz matrix hash 18]. It seems that these and other techniques we have reviewed would not have a software speed competitive with bucket hashing (assuming the message to be hashed is su ciently long to make bucket).

3 Bucket Hashing
Let X = X1 : : :Xn be a string, partitioned into n words. To hash X using bucket hashing we will scatter the words of X into N \buckets," then XOR the contents of each bucket, and then concatenate the bucket contents. Some ways of scattering the words of X work out better than others. In this paper we analyze a particular bucket hashing scheme, which we denote by B. The scheme will depend on parameters n; N; w. Scheme B will scatter each word into three buckets.

3.1 De ning the bucket hash family B

Fix a word size w 1 and parameters n 1 and N 3. We will be hashing from domain D = f0; 1gwn to range R = f0; 1gwN . As a typical example, take w = 32, n = 1024, and N = 140. If we want to be explicit, such a family would be denoted B 32; 1024; 140]. For the scheme we describe to make sense we require that N n. 3 Each hash function h 2 B is speci ed by a length-n list of cardinality-3 subsets of f1; : : :; N g. We denote this list by h = h1 hn . The three elements of hi are written hi = fhi1; hi2; hi3 g. Choosing a random h from B w; n; N ] means choosing a random length-n list of three-element subsets of f1; : : :; N g subject to the constraint that no two of these sets are the same. That is, we insist that hi 6= hj for all i 6= j .

Let h 2 B and let X = X1 Xn be the string we want to hash, where each jXi j = w. Then h(X ) is de ned by the following algorithm. First, for each j 2 f1; : : :; N g, initialize Yj to 0w . Then, for each i 2 f1; : : :; ng and k 2 hi , replace Yk by Yk Xi. When done, set h(X ) = Y1 k Y2 k k YN . In pseudocode we have:

Yhi1 Yhi1 Xi Yhi2 Yhi2 Xi Yhi3 Yhi3 Xi return Y1 k Y2 k k YN

The computation of a h(X ) can be envisioned as follows. We have N buckets, each initially empty. The rst word of X is thrown into the three buckets speci ed by h1. The second word of X is thrown into the three buckets speci ed by h2 . And so on, with the last word of X being thrown into the three buckets speci ed by hn . Our N buckets now contain a total of 3n words. Compute the xor of the words in each of the buckets (with the xor of no words being de ned as the zero-word). The hash of X , h(X ), is the concatenation of the nal contents of the N buckets.

for j 1 to N do Yj for i 1 to n do

0w

3.2 Collision probability of the bucket hash family B

The collision probability for B w; n; N ]. is the maximum, over all distinct x; x0 2 f0; 1gnw , of the probability that h(x) = h(x0 ). Our main theorem gives an upper bound on the collision probability of B. The bound is about 3312N 6. In other words, B w; n; N ] is -AU2 for 3312N 6.

Theorem 7 Main result] Assume w 1, N 32 and n N =12. Let be the collision probability for B w; n; N ]. Then B(N ), where B(N ) = (N ) (N ), for (N ) = 1=(1 6= N )
3

and

N 3)(N 4)2 +648(N 2)(N 3)2 : (N ) = 720(N 3)(N 4)(N 5)+1944( N 3(N 1)3(N 2)3

The proof of Theorem 7 is given in Section 4. Plot of B (N ). In Figure 1 we plot B (N ) against N . Consulting the graph we see, for example, that if you hash a string down to 140 words the collision probability is about 2 31. Comments. In the applications of bucket hashing to message authentication one typically wants a collision probability of, say, 2 30 or less. As can be seen from Figure 1, getting such a small collision probability requires a fairly large value of N . Since N is the length of our hashed string (in words), large values of N are undesirable and typically require additional layers of hashing. An example of this will be illustrated in Section 5. Note that our bound shows no dependency on w or n (though there is the technical restriction N =12). Indeed it is easy to see (and the proof of Theorem 7 will show) that the that n 3 collision probability does not depend on w. In fact, it is a consequence of the proof that, when 4 n N =12, the collision probability does not depend on n, either. 3 Observe that (N ) = N =(N 36), where N = N (N 1)(N 2). By our assumption that N 32 we have that 1 (N ) 1:002. So the multiplication by (N ) can e ectively be ignored; B(N ) (N ). 6

2**-20 2**-21 2**-22 2**-23 Upper bound on collision probablity, B(N) 2**-24 2**-25 2**-26 2**-27 2**-28 2**-29 2**-30 2**-31 2**-32 2**-33 2**-34 2**-35 40 60 80 100 120 140 Number of buckets, N 160 180 200

Figure 1: A graphical representation of Theorem 7. We plot of N verses, B (N ), our bound on the collision probability of B w; n; N ]. We believe that it is possible to relax the restriction n N =12 all the way to n < N . However, 3 3 doing this would add considerable complexity to the proof, yet have relatively little practical value, since the number of buckets, N , needs to quite large in order to obtain what would usually be regarded as a suitably small collision probability. Explanation. Here is a bit of intuition for what is going on. Suppose an adversary wants to nd a pair of distinct messages x; x0 2 f0; 1gwn which are most likely to collide under a function from B. What two messages should she choose? In the proof of Theorem 7 we recast this question into the following one. An adversary will throw t triples of balls into N buckets. Each of the 3t balls will land in a random bucket, except for the following constraints: three distinct buckets are selected for the three balls of each toss; and no tosses will land in identical triples of buckets. The adversary's goal is the following: make every bucket end up with an even number of balls in it. All the adversary can do is choose how many triples of balls, t, she will disperse. The question we must answer is: what choice of t, where 1 t n, will maximize the adversary's chance to win this game? It is not hard to guess the right answer to this question: four. Here is an explanation. If the adversary tosses just one triple of balls into the buckets she can't possibly win: 3 buckets are guaranteed to have an odd number of balls. If she throws out two triples of balls she again can not win, thanks to the constraint that no two triples of balls land in identical triples of buckets. If she throws out three triples of balls she again can not win because 9 balls can't be distributed into buckets in such a way that every bucket has an even number of balls. If the adversary throws out four triples of balls then, nally, she has a chance to win. This seems like it ought to be the best thing for the adversary to do, because it would seem to become increasingly unlikely to get every bucket to have an even number of balls when more balls get tossed into the N buckets. Though 7

this intuition is a long way from being formal, four triples of balls does turn out to be the right answer. Translating back into the adversary's original goal, the adversary can do no better than to choose messages X and X 0 which di er by exactly 4 words: for X these words are, say, 0w , while for X 0 these words are, say, 1w .

3.3 The e ciency of the bucket hash family B

instruction counts for computing a function h 2 B. Though instruction counting is an extremely crude predictor of speed, an analysis like this is still a good implementation-independent way to get some feel for our method's potential e ciency. To construct a good MAC we will probably want a collision probability of 2 30 (perhaps less) and so, in view of Figure 1, we will be using a reasonably large value of N , say N 120. Thus we will be needing more buckets than can be accommodated by a typical machine's register set. There are then two natural strategies to hash the string X = X1 : : :Xn , where each Xi is a word of the machine's basic word size: Method-1 (Process words X1; : : :; Xn). We can read each Xi from memory (in sequence) and then, three times: (1) load from memory the value Yj of the appropriate bucket j ; (2) compute Xi Yj ; (3) store this back into memory, modifying Yj . Total instruction count is 10 instructions per word (4 reads, 3 writes, 3 xors). Method-2 (Fill buckets Y1; : : :; YN ). We can xor together all words that should wind up in bucket 1; then xor all words that go into bucket 2; and so forth, for each of the N buckets. We will need a total of 3n reads into X1 ; : : :; Xn , plus we will need 3n N xor operations. Depending on what we want done with the hash, we may need another N writes to put the hash value back into memory. So the total instruction count is about 6 instructions per word. Achieving the stated instruction counts requires the use of self-modifying code (\sm-code"); in e ect, we implicitly assumed that the representation of h 2 B is the piece of executable code which computes h. In implementation, this can be tricky. If we don't want to use self-modifying code (\sm-code") we will need to load from memory the bucket locations (Method-1) or word location (Method-2). This would add 3 loads per word. For Method 2, sm-code would further increase the instruction count because of the overhead needed to control the looping: it is h-dependent how many words will fall into a given bucket, so this will have to be read from memory, and loopunrolling may be di cult. Assuming an additional one instruction per word to account for this work, we have the following approximate instruction counts: Method-1, sm-code Method-1, sm-code Method-2, sm-code Method-2, sm-code

Instruction counts. To get a feel for the e ciency of bucket hashing, let us do some approximate

implementation

instrs/wd
10 13 6 10

The sm-code uses a table to specify h. Assume a machine with a word size of 32 bit. For Method-1 the needed table would typically be 3n or 12n bytes long (depending on whether one packs bucket indices into bytes or words). For Method-2 that table would typically be be 6n or 12n bytes long (depending on whether one packs word indices into double-bytes or words), plus an additional N 8

or 4N bytes long (depending on whether one packs counter-limits into bytes or words). To get a fast implementation, tables need to t into cache. Note that there is better locality of reference for Method-1 than Method-2, and this can have a substantial e ciency impact when actually coded. Implementation. A variety of bucket hashing schemes have been implemented (that is, B and methods similar to B). The observed performance of these implementations varies enormously according to the particular scheme, the parameters n and N , and the implementation. As a couple points of reference: on a typical 32-bit RISC machine (an SGI with a 150 MHz IP22 processor, 16 KByte data cache, 16 KBytes instruction cache) the most straightforward Method-1/sm implementation ran at 340 Mb/s to hash 1024 words to 140, while a Method-2/sm implementation of a bucket hash family based on the C 10; 6] graph (see Section 7) ran at 1160 Mb/s to hash 909 words to 182. Follow-on work will give detailed experimental results. Rough comparisons. Shoup estimates a cost of about 24 instructions/word (6 instructions per byte) for computing a hash function h 2 K, where K is described in Section 2 24]. Bosselaers, Govaerts and Vandewalle have implemented MD5 at a cost of 36 instructions/word on a Pentium 7] (they obtain a good degree of overlapping instruction-issue, too). In recent work, Halevi and Krawczyk estimate a cost of about 7.5 instructions per word (assuming architectural support for multiplying two 32-bit words to yield a 64-bit product) for their MMH technique 15]. We emphasize that trying to compare such numbers hides many signi cant factors, including length of hash output (worst for bucket hashing), table sizes and caching issues, and the degree of available parallelism. We have not studied these tradeo s and have no idea if bucket hashing will eventually \win out" in the choice of hash techniques for making a practical MAC. We only suggest that bucket hashing looks promising.

4 Proof of the Main Theorem

In this section we prove Theorem 7. Throughout this section x values of n and N satisfying the conditions of the theorem. Our rst two claims show how to simplify the setting. One can assume a word length of w = 1. First we argue that, without loss of generality, we can assume that the word length for B w; n; N ] is w = 1. Intuitively, this follows from the \bitwise" character of bucket hashing: when we hash X1 Xn down to Y1 YN , where jXij = jYj j = w, the `-th bit of Yi depends only on X1 `]; : : :; Xn `]. For this reason, no advantage can be gained by trying to exploit long words.

Claim 8

X;X 0 2f0;1gnw

X 6=X 0

max

H 2B w;n;N ]

Pr

H (X ) = H (X 0) = x;xmax 0 2f0;1gn
x6=x0

h2B 1;n;N ]

Pr

h(x) = h(x0) .

Proof : Let X; X 0 2 f0; 1gwn be distinct strings which maximize PrH H (X ) = H (X 0)]. Since X 6= X 0 there must be some bit position 1 ` w such that the n-bit strings x = X1 `] Xn `] 0 `] X n 0 `] are distinct. Now notice that we can treat any H 2 B w; n; N ] as a and x0 = X1 hash function h = H from B 1; n; N ], and conversely, because the description of a bucket hash hash function (a sequence of triples of indices) is insensitive to the word length w. Furthermore, H (X ) = H (X 0) implies that h(x) = h(x0 ), and so PrH H (X ) = H (X 0)] Prh h(x) = h(x0 )] . We conclude that maxX;X 0 PrH H (X ) = H (X 0)] maxx;x0 Prh h(x) = h(x0 )].

For the opposite inequality, let x; x0 2 f0; 1gn be distinct strings which maximize Prh h(x) = h(x0)]. Write x = x1 : : :xn and x0 = x01 : : :x0n, where xi and x0i are bits, for all 1 i n. De ne 0 : : :Xn 0 by setting Xi j ] = xi and X 0 j ] = x0 the wn-bit strings X = X1 : : :Xn and X 0 = X1 i i 0 for each 1 j w. Clearly Prh h(x) = h(x )] = PrH H (X ) = H (X 0)]. We conclude that } maxx6=x0 Prh h(x) = h(x0 )] maxX 6=X 0 PrH H (X ) = H (X 0)], as desired. Given what we have just shown, we henceforth assume a word length as w = 1. We will use B as shorthand for B 1; n; N ]. Exploiting linearity. For 0 t n, let 1t = 1t0n t and let 0 = 0N . For 0 < t n de ne
t

We are trying to bound , the collision probability of B, which is the maximum, over all distinct x; x0 2 f0; 1gn, of Prh2B h(x) = h(x0)]. We use Claim 8 and the structure of bucket hashing (particularly its linearity) to get the following:

= hPr h(1t ) = 0] : 2B

Claim 9 If n 4 then = t max . If n < 4 then = 0. ; ; ;::: t

=4 6 8

Proof : First observe that, for h 2 B, computing h(x) amounts to computing a product Ax over GF 2] of an N n matrix A and a column vector x. In fact, selecting a random hash function h 2 B corresponds to picking a random binary n N matrix A which has three ones in each column and no two identical columns. Writing A for the set of all such matrices we observe that

= = = = =

max x6=x0 max x6=x0 max x6=x0 max x6=0n max x6=0n

h2B

Pr h(x) = h(x0 )] Pr Ax = Ax0 ] A2A Pr A(x x0) = 0] A2A Pr Ax = 0] A2A Pr h(x) = 0] h2B

Thus we don't have to think about the probability of distinct strings colliding; it is simpler and more convenient to think about the probability that a non-zero string gets hashed to 0. Next we argue that Prh h(x) = 0] depends only on the number of ones in x (its Hamming weight), and not on the particular arrangement of zeros and ones within x. Suppose that x has t ones: we claim that PrA Ax = 0] = PrA A1t = 0]. For suppose that the non-zero positions of x = x1 xn are at locations 1 j1 < < jt n (meaning that xi = 1 if and only if and only if i 2 fj1; : : :; jtg). Then we pair each matrix A 2 A with a matrix A0 2 A by permuting the columns of A so that columns j1; : : :; jt come rst. Then for every A 2 A, Ax = A01t . Since, for any x, the associated pairing A ,! A0 is bijective, PrA Ax = 0] = PrA A1t = 0]. From Claim 8 and what we have just shown, we now know that = maxt=1;2;3;::: t . So we ask: for which t 1 is t largest? One thing is clear: it can not be any any odd-indexed 1t , for if t is odd then h(1t) 6= 0, because it is impossible to partition 3t ones into disjoint sets in such a way that there are an even number of ones in each set. In other words, Prh h(1t ) = 0] = 0 for odd t. 10

Likewise, PrA A12 = 0], because of our insistence that no two columns of A are identical. The claim now follows. } Strategy. Our plan is as follows. First we will bound 4 from above by B (N ). Then we will show that t B (N ) for all even t 6. Using Claim 9 we can then conclude that B (N ). Our analysis is made possible by using a particular Markov Chain, M . This Markov chain does not accurately describe bucket hashing. But we can correct for the inaccuracy which the chain introduces. Markov chain model. Consider for a moment an inferior form of bucket hashing: instead of B, where each hi among h = h1 hn is required to be di erent from any other, consider the the family of hash functions C , which removes that constraint. In other words, a random h = h1 : : :hn 2 C 1; n; N ] is a sequence of random triples, hi = fhi1; hi2; hi3g, where hi1; hi2; hi3 2 f1; : : :; N g are distinct. This corresponds to a random N n binary matrix C with three ones per column. While there is no natural Markov chain model for B, there is a natural Markov chain M corresponding to C . This chain keeps track of the number of buckets with an odd number of 1's. Thus the Markov chain M has (N + 1)-states, f0; 1; : : :; N g. Being in state i means that i buckets now have an odd number of ones (and N i buckets have an even number of ones). A transition in M corresponds to throwing three balls into 3 distinct buckets: after each such throw, there is a new number of buckets with an odd number of ones. So state 0 is the start state. Since three balls are tossed with each throw, there can be a non-zero transition probability from states i to j only when ji j j 3. (In fact, the only transitions that can happen are from a state i to a state j 2 fi 3; i 1; i + 1; i + 3g \ f0; : : :; N g). The probability of returning to state 0 after t steps corresponds precisely to Prh2C h(1t ) = 0]. Let N = N (N 1)(N 2). Let Pij denote the transition probability of M : the probability of of moving from state i to state j in a single step. To capture the process C we have described we need to de ne M 's transition probabilistic as follows:
8 > > > > > > > > > > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > :

Pij =

1 3(N 1)(N 2)=N (N 1)(N 2)(N 3)=N 6(N 2)=N 6(N 2)(N 3)=N (N 2)(N 3)(N 4)=N i(i 1)(i 2)=N 3i(i 1)(N i)=N 3i(N i)(N i 1)=N (N i)(N i 1)(N i 2))=N 0

if (i; j ) 2 f(0; 3); (N; N 3)g if (i; j ) 2 f(1; 2); (N 1; N 2)g if (i; j ) 2 f(1; 4); (N 1; N 4)g if (i; j ) 2 f(2; 1); (N 2; N 1)g if (i; j ) 2 f(2; 3); (N 2; N 3)g if (i; j ) 2 f(2; 5); (N 2; N 5)g if 3 i N 3 and j = i 3 if 3 i N 3 and j = i 1 if 3 i N 3 and j = i +1 if 3 i N 3 and j = i +3 otherwise

(1)

Let us give an example of how the above values are computed. Consider Pij for the case associated to 3 i N 3 and j = i + 1. In order to go from state i to state i + 1 in a single step, one ball of the three will have to land in one of the i buckets that has an odd number of balls already, while the remaining two balls must land among the N i remaining buckets. There are 3i(N i)(N i 1) ordered triples of bucket indices that will accomplish this among the N ordered triples of bucket indices. (The \3" takes care of the fact that there are 3i ways to choose the ball which lands in a bucket with an odd number of balls; after that ball is selected, the remaining two balls have to land in the other N i buckets.) The reasoning for all of the other Pij values is similar. 11

0
720 6

start state

3
210 120 378

126 216

504

1
216 48 336 336 60

6
24 504 216

360 378

120

4
360

24

300

216

7
6 126

210

300

5
60 336 336

10

720

48

Figure 2: The Markov chain M for N = 10 states. The start state is state 0. Divide the number labeling each arc i ! j by N = N (N 1)(N 2) = 720 to get the transition probability Pij . In Figure 2 we depict the Markov chain M for some the case where the number of states is N = 10. The transition probabilities are computed from Equation 1. Using M to bound 4 . We are now ready to show that 4 B (N ). Recall that B (N ) = (N ) (N ), where (N ) and (N ) are given by the formulas in the statement of Theorem 7.

Lemma 10

B(N ) .

Proof : First some notation. Let t n be a number and h1 ht be a sequence of triples of distinct elements drawn from f1; : : :; N g. Then Parity(h1 ht ) is the N -vector whose i-th component, i 2 f1; : : :; N g, is 0 if i occurs an even number of times in the multiset h1 ht , and 1 if i occurs an odd number of times. Given an N -vector of bits y = y1 yN , let NumOnes(y ) denote the number of 1-bits in y . De ne State(h1 ht ) = NumOnes (Parity(h1 ht )). Thus State(h1 : : :ht ) is the state of M one is in after hashing 1t with h = h1 ht . For an N -vector of bits, de ne State (h1 ht ) = NumOnes ( Parity(h1 ht )). Thus State (h1 : : :ht ) captures the state of M one is in after hashing 1t with h = h1 ht , given that we start in the con guration speci ed by . Let Hist(h1 h2 h3h4 ) = 0 State(h1) State(h1h2 ) State(h1h2 h3 ) State(h1h2 h3 h4 ) :

This is a list of 5 numbers, each in f0; ; N g, where Hist(h1 h2 h3 h4 ) encodes the sequence of states in M one passes through on hashing 14 according to h = h1 h2 h3 h4 . Let Distinct (h1 ht ) be true if h1 ; : : :; ht are all distinct, and false otherwise. Let Rt (\random") be the uniform distribution on h1 ; ; ht (that is, each hi is a random triple of distinct points from f1; : : :; N g). 12

Let Dt (\distinct") be the uniform distribution on distinct h1 ; : : :; ht (that is, each hi is a random triple of distinct points from f1; : : :; N g, and no two of these triples are identical). Let C (m; t) denote the probability of at least one collision in the experiment of throwing t balls, independently and at random, into m bins. We are now ready to prove the lemma.
4

= Pr State(h1 h2 h3 h4 ) = 0] D
4

= Pr State(h1 h2 h3 h4 ) = 0 j Distinct(h1 h2 h3 h4)] R

4

Pr State(h1 h2 h3 h4 ) = 0 and Distinct (h1h2 h3 h4 )] = R4 Pr Distinct(h h h h )]

R4
1 2 3 4

Pr Hist(h1 h2 h3 h4 ) 3 f03630; 03430; 03230g] R4 1 C( N ; 4) 3 (N ) Pr Hist(h1 h2 h3 h4 ) = 03630] + Pr Hist(h1 h2 h3 h4 ) = 03430] + R R Pr Hist(h1 h2 h3h4 ) = 03230] R
4 4 4

(2)

= =

(N ) P03 P36P63 P30 + P03 P34P43 P30 + P03 P32P23 P30 6 + 1 9(N 3)(N 4) 36(N 4) 6 + (N ) 1 (N 3)(NN 4)(N 5) 120 N N N N N 6( N 2)( N 3) 6 18( N 3)( N 4) (3) 1 N N N

N 3)(N 4)2+648(N 2)(N 3)2 (N ) 720(N 3)(N 4)(N 5)+1944(N 3

= B (N ) Equation 2 is justi ed by referring to Figure 2: the only length-4 routes from state 0 back to state 0 are 03630, 03430, 03230, and 03030. The last of these can only arise from non-distinct h1 ; h2; h3; h4. For the other three we simply disregard the conjunction with Distinct(h1 h2 h3h4 ) because we are giving an upper bound. Equation 3 is obtained directly from Equation 1. } Using M to bound 6 ; 8; . Assume that N is even and N 6. We will show, in this case, that t B (N ). Here is the idea. Take a random function h 2 B and look at it's last 6 maps| say h = h7 ht h1 h2h3 h4 h5h6 . Now h1; : : :; h6 are statistically correlated to h7 ; : : :; ht (for example, h1 6= h7 ), yet h1 ; : : :; h6 are not too far from being random and independent, in the sense that, for any h7 ht , a uniformly selected sequence of maps h01 h02 h03 h04 h05 h06 would have been a valid 13

continuation with probability at least 1=2. (This follows from our assumption that n N =12.) 3 Thus, up to a factor of 2, we can bound the chance of landing in state 0 on applying h to 1t by looking at the chance of landing in state 0 after applying a uniformly selected h1 : : :h6 starting from some in some arbitrary (unknown) state of the Markov chain. To formalize the above argument, let fi (t) denote the maximum, over all initial states s, of the probability that we arrive in state i in exactly t transitions, given that we start in state s. This is the same as the supremum, over all distributions on the starting state of M , of the probability that we arrive in state i in exactly t transitions, given that we start in an initial state as chosen by sampling from . We will need the following lemma about the behavior of Markov chain M .

Lemma 11 f (6) (25920N + 154080N )=N .

0 8 7 5

The proof is a tedious but straightforward calculation using the transition probabilities of M . It is relegated to Appendix B. The point isn't the speci c formula, but only that f0 (6) is less than half B(N ) for all su ciently large N .

Lemma 12 Assume 6 t n. Then

B(N ).

Proof : We use the same notation as in the proof of Lemma 10.

t

= hPr h(1t) = 0)] 2B State(h7 = h h h h Pr t 1 2 h3 h4 h5 h6 2Dt 7 = h hPr t 2Dt 7

h7 ht 2Dt
6

ht h1 h2 h3 h4 h5 h6) = 0]
State (h7

fh1 ; ;h6 g\fh7; ;ht g=; fh1 ; ;h6 g\fh7; ;ht g=;
h1 h6 2D6

h1 h6 2D6

Pr

ht h1h2 h3 h4 h5 h6 ) = 0] ht ) Parity(h1 h2 h3 h4 h5 h6 )) = 0]
ht ; de ne E ,

max

Pr

NumOnes (Parity(h7

= h Pr NumOnes ( Parity(h1 h2 h3 h4h5 h6 )) = 0] //Fix h7 1 h6 2E Pr State (h1 h2 h3 h4h5 h6 ) = 0 and Distinct(h1 h6 h7 h1 h6 2R6 = Pr Distinct(h h h h h h h h )]
h1 h6 2R6

accordingly

ht )]

Pr

State (h1 h2 h3 h4h5 h6 ) = 0]

h1 h6 2D6

2 3

4 5 6

1 6 t= N 3
//From assumption that n //De nition of f //By Lemma 12
N =12 3

2 h Pr State (h1h2 h3 h4 h5 h6 ) = 0] 1 h6 2R6 2 f0 (6) 2 (25920N 8 + 154080N 7)=N 5 B(N ) for all N 32 14

For the last inequality: it is easy to verify that this holds for su ciently large N . The crossover point was determined numerically. } We have now shown that, under the conditions of the theorem, B (N ) completes the proof.
t

for all t

1. This

5 From Universal Hash Families to Message Authentication

In this section we review the Wegman-Carter construction (and its complexity-theoretic variant). See Appendix A for formal de nitions of the security of a message authentication code (MAC) and a nite pseudorandom function (PRF). The Wegman-Carter Construction. Given a family of hash functions H = fA ! f0; 1gb g we wish to construct from it a MAC. In the scheme we denote WC H], the Signer and Veri er share a random element h 2 H, as well as an in nite random string P = P1 P2 P3 , where jPij = b. The pair (h; P ) is the key shared by the Signer and Veri er. The signer maintains a counter, cnt, which is initially 0. To generate a MAC for the message x the signer increments cnt and then computes the MAC = (cnt; Pcnt h(x)) which authenticates x. To verify a MAC = (i; s) for the message x the Veri er checks if s = Pi h(x). The following theorem says that it is impossible (regardless of time, number of queries, or amount of MACed text) to forge with probability exceeding the collision probability.

Proposition 13 32, 18] Let H be -AXU and suppose adversary E forges in the scheme WC H]
with probability . Then .
2

of the random pad P , an random index a 2 f0; 1g into a nite PRF F : f0; 1g f0; 1gl ! f0; 1gb . The Signer maintains a counter cnt 2 f0; 1gl , initially 0. (We will not distinguish between numbers and their binary encodings into l-bits.) The Signer and Veri er share a random a 2 f0; 1g and a random h 2 H. When the Signer wishes to MAC a message x, if cnt < 2l 1 then the Signer computes = (cnt; Fa(cnt) h(x)) and increments cnt. To verify a MAC = (i; s) for the message x the Veri er checks if s = Fa (i) h(x). At most 2l messages may be MACed (after that, the key a must be changed). We call the scheme just described WC H; F ]. The following result is obtained by standard techniques. Recall that de nitions are in Appendix A.
the time required to compute from this representation the hash of q strings, these strings totaling bits. Let F : f0; 1g f0; 1gl ! f0; 1gb be a nite PRF with security 0 (t; q ). Let E be an adversary which, in time t, making q queries, these queries totaling bits, forges with probability against the scheme WC H; F ]. Then + 0 (t + t; q + 1), where t = O(Th(q; ) + TH + ql + qb).

Wegman-Carter with a finite PRF. A natural complexity-theoretic variant is to use, instead

Proposition 14 Let H = fh : A ! f0; 1gbg be an -AXU family of hash functions. Let TH denote the time required to compute a representation of a random element h 2 H, and let Th (q; ) denote
2

The value of t would usually be insigni cant compared to t. Note that in Proposition 13 the forging probability is independent of the number of queries (q ) and the length of the queried messages ( ). In Proposition 14 the forging probability depends on these quantities only insofar as they are detrimental to the security of the underlying PRF. We emphasize that the Signer is stateful in the schemes WC H] and WC H; F ]. The Signer being stateful improves security (compared with using a random index) and at little practical cost. 15

Note that the Veri er is not stateful. This is possible because we are using the \standard" notion of MAC security (Appendix A), which does not credit the adversary for \replay attacks."

6 Toy Example, and Limitations on Bucket Hashing

In this section we describe a concrete MAC based on the ideas presented so far. This is only a \toy" example; doing a good job at specifying a software-optimized bucket hash MAC would involve much design, experimental, and theoretical work which we have not carried out. Still, the example helps to illustrate the strengths of bucket hashing in making a MAC, as well as the limitations. Toy Example. To keep things simple, suppose the strings we will MAC are of length at most most 4096 bytes. Assume a word size of 4 bytes (32 bits). Let F : f0; 1g f0; 1g64 ! f0; 1g64 be a nite PRF (de ned, for example, from the compression function of MD5). Here is a way for the Signer to MAC a string X whose length is at most 1024 words. Assume an even number of words. The Signer and Veri er share as a MAC key (i) a random element h1 2 B 32; 1024; 140], (ii) a random element h2 2 K 71; 64], and a (iii) a random string a 2 f0; 1g . We use the construction of Proposition 5 (slightly modi ed to account for length-variability). In the algorithm below, jX j denotes the length of X , encoded as a 2-word string. The function h1 is extended to strings of length less than 1024 words in the natural way: we stop casting words into buckets when we reach the end of the string. (This is equivalent to 0-padding the string to 1024 words.)

Algorithm TOY-MAC(X ). if cnt = 2 1 then return error = hcnt; Fa (cnt) h ( jX j : h (X ))i

64

cnt = cnt + 1

return

Let us count instructions for TOY-MAC to hash a 4096-byte message. If we bucket hash in 10 instructions per word (Section 3.3), hash using h2 2 K in 24 instruction per word ( 24]), and compute F with 600 instructions (easy to accomplish), then we will spend 10 + (142=1024) 24 + 600=1024 = 10 + 3:3 + 0:6 = 13:9 instructions/word. Notice that the \cryptographic" contribution to the above time (i.e., the time to compute F ) is very small. In a Wegman-Carter MAC one is a orded the luxury of conservative (and slow) cryptography even in an aggressively speed-optimized design. This is because one arranges that the time to compute the MAC is dominated by the non-cryptographic work. Limitations on bucket hashing. If the strings we are MACing are short then, at some point, it makes sense to switch strategies and stop using bucket hashing. In our TOY-MAC, we might hash with only h2 when the input string has length less than some constant. This is an important limitation on bucket hashing; because the output length is substantial, the technique is simply not useful until the strings to be hashed get long enough. As a consequence, any \real" MAC which employs bucket hashing would likely be a patchwork of di erent techniques for di erent message lengths. Therefore a real bucket hash MAC is unlikely to be simple to describe or implement. On the other hand, if the strings to be hashed are very long then, at some point, it makes sense to break the input into blocks and independently bucket has each block, using the construction of Proposition 2. This is because the size of the description of h 2 B grows linearly in the maximal length string which h can hash. We do not want hash functions with excessively long descriptions 16

(certainly the hash function should t in cache). This is another limitation on the bucket hashing technique, and something which will further complicate the de nition of any real bucket hash MAC. In our TOY-MAC, if we wanted a substantially better collision probability we could apply the construction of Proposition 3, but this would roughly halve the rate for bucket hashing, and perhaps other techniques might then be faster. This is a third limitation on bucket hashing: until better constructions are found, obtaining an extremely small collision probability, say 2 50 , would require an excessive number of buckets. That is, the output length of the hash function would be very long, and so the technique would only be useful for hashing extremely long messages. The last limitation we will mention is the time needed to compute a description of h. In any real MAC scheme the function h 2 B would be determined from some underlying key k with the help of a pseudorandom generator. Because the description of h is large and of a special form, computing h might take a signi cant amount of time. In most applications of fast message authentications, an one-time key pre-processing delay is not important. But if there is a limited amount of text to be MACed, or if the latency of the rst MAC must be minimized, than the time to compute the description of h could be an issue. One approach is to nd a version of bucket hashing that uses a small key (ie., a short description for h). This way the underlying pseudorandom generator (if present) is less taxed. This approach has been investigated by 16], who achieves a major reduction in the size of the description the h. Balanced against these limitations is the possibility of extremely high MAC throughput, at least for long strings.

7 Extensions and Directions

Generalizing B, we call by \bucket hashing" any scheme in which the hash function h is a given by a list h1 hn of \small" subsets of f1; : : :; N g and the hash of X = X1 Xn , where jXi j = w, is: for j 1 to N do Yj 0w for i 1 to n do

for each k 2 hi do
Yk
1 2

In the general case the distribution on h-values is arbitrary. So B is just the special case in which we use the uniform distribution on distinct triples in f1; : : :; N g. One could imagine many alternative distributions, some of which will give rise to faster-tocompute hash functions or better bounds on the collision probability. As an example, suppose h 2 H is chosen by randomly re-ordering a list h1 hn of triples which are chosen so that for all sets I f1; : : :; ng of cardinality 2 or 4, it is not the case that the multiset h2I h has an even number of each point 1; : : :; N . This new family of hash functions may have substantially smaller collision probability than B for a given n; N . The bucket hash scheme of a graph. Hash family B would have been more e cient had each word gone into two buckets instead of three. One way to specify a scheme where each word lands in two buckets is with a graph G whose N vertices comprise the N buckets and whose m edges f1; : : :; mg indicate the pairs of buckets into which a word may fall. A random hash function from the family is given by a random permutation on f1; : : :; mg. To hash a string X1 : : :Xn using , where jXij = w and n m, each word Xi is dropped into the two buckets at the endpoints of 17

return Y k Y k

Yk Xi k YN

edge (i). As before, we xor the contents of each bucket and output their concatenation in some canonical order. We call the above scheme the bucket hash of the graph G. For a graph G to be \good" we want a small number of vertices N , a large number of edges m, and such that for all k where 1 k n m, if k distinct edges are selected at random from G, then the probability that their union (with multiplicities) comprises a union of cycles is at most some tiny number . One possible choice of graphs in this regard are the (d; g )-cages (see 8]). A (d; g )-cage is a smallest d-regular graph whose shortest cycle has g edges. These graphs have been explicitly constructed for various values of (d; g ). Though (d; g )-cages are rather large (for even g they have at least (2(d 1)g=2 2)=(d 2) nodes) some (d; g )-cages may give rise to useful hash families. For example, assume d 1 is a prime power. Let C d; 6] be the (d; 6)-cage. This is the the point-line incidence graph of the projective plane of order d 1. Bucket hashing with C 10; 6] may be a good way to hash 909 words down to 182 words. Open questions. The generalized notion of bucket hashing amounts to saying that hashing is achieved for each bit position 1 : : :w by matrix multiplication with a sparse Boolean matrix H . Expressing the method in this generality raises questions like the following: for a given N , n and k, for all distributions D of binary N n matrices H having k ones per column, for which is maxx2f0;1gn f0n g HPr Hx = 0] minimized? What if we also demand that each row has a xed 2D number of ones? What if, instead of saying that there are k ones per column, we cap the density of the matrix at some value ? Answers may lead to faster bucket hash schemes.

Acknowledgments

Many thanks to the two anonymous referees for their careful reviews. Thanks also to Mihir Bellare, Don Coppersmith, Hugo Krawczyk, and David Zuckerman for their comments and suggestions.

References
1] N. Alon, O. Goldreich, J. Hastad and R. Peralta, Simple constructions of almost k-wise independent random variables, 31st Annual Symposium on Foundations of Computer Science, IEEE Computer Society, 1990, pp. 544{553. 2] R. Arnold and D. Coppersmith, An alternative to perfect hashing, IBM RC 10332 (1984). 3] M. Bellare, R. Canetti and H. Krawczyk, Keying hash functions for message authentication, Advances in Cryptology { CRYPTO '96, Lecture Notes in Computer Science, vol. 1109, Springer-Verlag, 1996, pp. 1{15. 4] M. Bellare, O. Goldreich and S. Goldwasser, Incremental cryptography: the case of hashing and signing, Advances in Cryptology { CRYPTO '94, Lecture Notes in Computer Science, vol. 839, Springer-Verlag, 1994, pp. 216{233. 5] M. Bellare, J. Kilian and P. Rogaway, The security of cipher block chaining, Advances in Cryptology { CRYPTO '94, Lecture Notes in Computer Science, vol. 839, Springer-Verlag, 1994, pp. 341{358.

18

6] J. Bierbrauer, T. Johansson, G. Kabatianskii and B. Smeets, On families of hash functions via geometric codes and concatenation, Advances in Cryptology { CRYPTO '93, Lecture Notes in Computer Science, vol. 773, Springer-Verlag, 1994, pp. 331{342. 7] A. Bosselaers, R. Govaerts and J. Vandewalle, Fast hashing on the Pentium, Advances in Cryptology { CRYPTO 96, Lecture Notes in Computer Science, vol. 1109, Springer-Verlag, 1996, pp. 298{312. 8] J. Bondy and U. Murty, Graph theory with Applications, North Holland, 1976. 9] G. Brassard, On computationally secure authentication tags requiring short secret shared keys, Advances in Cryptology { CRYPTO '82, Springer-Verlag, 1983, pp. 79{86. 10] L. Carter and M. Wegman, Universal hash functions, J. of Computer and System Sciences 18, 1979, pp. 143{154. 11] Y. Desmedt, Unconditionally secure authentication schemes and practical and theoretical consequences, Advances in Cryptology { CRYPTO '85, Lecture Notes in Computer Science, vol. 218, Springer-Verlag, 1985, pp. 42{45. 12] P. Gemmell and M. Naor, Codes for interactive authentication, Advances in Cryptology { CRYPTO '93, Lecture Notes in Computer Science, vol. 773, Springer-Verlag, 1994, pp. 355{ 367. 13] O. Goldreich, S. Goldwasser and S. Micali, How to construct random functions, Journal of the ACM, Vol. 33, No. 4, 1986, pp. 210{217. 14] S. Goldwasser, S. Micali and R. Rivest, A digital signature scheme secure against adaptive chosen-message attacks, SIAM Journal of Computing, vol. 17, no. 2 (April 1988), pp. 281{308. 15] S. Halevi and H. Krawczyk, MMH: Message authentication in software in the Gbit/second rates, Proceedings of the 4th Workshop on Fast Software Encryption, Springer-Verlag, 1997. 16] T. Johansson, Bucket hashing with small key size, Advances in Cryptology { EUROCRYPT '97, Lecture Notes in Computer Science, Springer-Verlag, 1997. 17] T. Johansson, G. Kabatianskii and B. Smeets, On the relation between A-codes and codes correcting independent errors. Advances in Cryptology { EUROCRYPT '93, Lecture Notes in Computer Science, vol. 765, Springer-Verlag, 1994, pp. 1{11. 18] H. Krawczyk, LFSR-based hashing and authentication, Advances in Cryptology { CRYPTO '94, Lecture Notes in Computer Science, vol. 839, Springer-Verlag, 1994, pp. 129{ 139. 19] M. Luby and C. Racko , How to construct pseudorandom permutations from pseudorandom functions," SIAM J. Comput, vol. 17, no. 2 (April 1988). 20] X. Lai, R. Rueppel and J. Woollven, A fast cryptographic checksum algorithm based on stream ciphers, Advances in Cryptology, Proceedings of AUSCRYPT 92, Lecture Notes in Computer Science, vol. 718, Springer-Verlag, 1992, pp. 339{348.

19

21] P. Pearson, Fast hashing of variable-length text strings, Communications of the ACM, vol. 33, no. 6 (1990), pp. 677{680. 22] R. Rivest, The MD5 message digest algorithm, IETF RFC-1321, 1992. 23] P. Rogaway, Bucket hashing and its application to fast message authentication, Advances in Cryptology { CRYPTO '95, Lecture Notes in Computer Science, vol. 963, Springer-Verlag, 1995, pp. 313{328. 24] V. Shoup, On fast and provably secure message authentication based on universal hashing, Advances in Cryptology { CRYPTO '96, Lecture Notes in Computer Science, vol. 1109, SpringerVerlag, 1996, pp. 74{85. 25] A. Siegel On universal classes of fast high performance hash functions, their time-space tradeo , and their applications, 30th Annual Symposium on Foundations of Computer Science, IEEE Computer Society, 1989, pp. 20{25. 26] G. Simmons, A survey of information authentication, in Contemporary Cryptography, The Science of Information Integrity, G. Simmons, editor, IEEE Press, New York, 1992. 27] D. Stinson, Universal hashing and authentication codes, Designs, Codes and Cryptography, vol. 4 (1994), pp. 369{380. 28] R. Taylor, An integrity check value algorithm for stream ciphers, Advances in Cryptology { CRYPTO `93, Lecture Notes in Computer Science, vol. 773, Springer-Verlag, 1994, 40{48. 29] J. Touch, Performance analysis of MD5, Proc. Sigcomm '95, ACM, 1995, pp. 77{86. 30] G. Tsudik, Message authentication with one-way hash functions, Proceedings of Infocom 92, IEEE Press, 1992. 31] U. Vazirani, E ciency considerations in using semi-random sources, Proceedings of the Nineteenth Annual ACM Symposium on Theory of Computing, ACM Press, 1987, pp. 160{168. 32] M. Wegman and L. Carter, New hash functions and their use in authentication and set equality, J. of Computer and System Sciences, vol. 22, 1981, pp. 265{279. 33] A. Zobrist, A new hashing method with applications for game playing, University of Wisconsin, Dept. of Computer Science, TR #88 (April 1970).

A De nitions of Security
codes. A MAC scheme M speci es: constants L and c, determining Messages = f0; 1g L and Tags = f0; 1gc ; a set of strings Keys; a number MAX (alternatively, MAX = 1); and a pair of functions (MAC; MACV), where MAC : MACV : Keys Messages f1; : : :; MAXg ! Tags; and Keys Messages Tags ! f0; 1g :
MAC security. We follow 14, 5] and de ne deterministic, counter-based message authentication

20

0
N
6

3
(N3)(N4)(N5) 9(N3)(N4) 120

18(N3) 36(N4)

(N1) . (N2) . (N3)

1
3(N1)(N2) 6(N2) 6(N2)(N3)

6
15 (N5) (N6) + 126 (N7) + 504

24

2
60

60 (N5) + 210

R
Figure 3: A view of the Markov chain M , where, for purposes of analysis, we have lumped together all states other than 0; 1; 2; 3; 4; 6. Divide the number labeling each arc i ! j by N = N (N 1)(N 2) to get the transition probability Pij . The rst argument to MAC and MACV will usually be written as a subscript. We demand that for any x 2 Messages, k 2 Keys, and cnt 2 f1; : : :; MAXg, MACVk (x; MACk (x; cnt)) = 1 . Let M be a message authentication scheme. A MAC oracle MACk ( ) for M behaves as follows: it answers its rst query, x1 , with MACk (x1; 1); it answers its second query, x2, with MACk (x2 ; 2); and so forth. The MAC oracle responds with the empty string to queries beyond the MAXth or to queries not in the set Messages. An adversary E for a message authentication scheme M is an algorithm equipped with a MAC oracle MACk ( ). Adversary E is said to forge on a particular execution, this execution having MAC oracle MACk ( ), if E outputs a string (x ; ) where MACVk (x ; ) = 1 yet E made no oracle query of x . When we speak of E forging with a particular probability, that probability is taken over E 's coin tosses and a random key k for the MAC oracle. Running times are measured in a standard RAM model of computation, with oracle queries counting as one step. By convention, the running time of E also includes the size of E 's description. PRF security. We follow 13, 5]. A nite pseudorandom function family (PRF) is a map F : f0; 1g f0; 1gl ! f0; 1gb . We write Fa(x) in place of F (a; x). Let Rl;b be the set of all functions mapping f0; 1gl to f0; 1gb . A distinguisher is an algorithm D with access to an oracle. We say that a PRF F is (t; q )-secure if hfor every distinguisher i h D which i runs in time t and makes F () () k D =1 Pr D =1 (t; q ) . Running times q or fewer queries to its oracle, k Pr Rl;b f0;1g are measured in a standard RAM model of computation, with oracle queries counting as one step. By convention, the running time of E also includes the size of E 's description.

21

t=1 t=2 t=3 t=4 t=5 t=6

0
N
324 6

1
1
3 42

2
N 2 +10N

3
1
N4
60 4320

4
1

6
1

R
1 1

N2 N

N4 N

N2

N3
60

54 1098N N + N3

N2 N

N 8+25680 N7 4

N 438N N + N3

15

N 25N N + N2
2

25920

8+154080 5

Figure 4: Row t, column i gives our bound on fi (t). The only value needed is f0 (6); we need that 2f0 (6) B (N ).

B Proof of Lemma 12
Here we prove Lemma 12, giving a suitable upper bound on f0 (6). We do this by direct calculation, paying attention to the states 0; 1; 2; 3; 4; 6 and \everything else." To that end, let P R= f 5 ; 7 ; 8 ; : : :; N g (i.e., \everything else") and de ne f ( t ) = R r2R fr (t). Let us write PRj P for r2R Prj . Keep in mind that R is not a state of any Markov chain we have de ned; this is just a convenient shorthand. We will establish the bounds indicated in Figure 4, where row t, column i is the upper bound we show for fi (t). To see how these bounds are computed, refer to Figure 3, which depicts the relevant transition probabilities of M . We start with the trivial bounds: f1 (1) 1, f3 (1) 1, fR (1) 1, f4 (2) 1, and f2 (6) 1. These are obvious, since each fi (t) represents a probability. Now refer to Figure 3 and calculate. Some of the mundane arithmetic is omitted. In cases such as the calculation of f6 (4), the second inequality is easily seen to hold for su ciently large N ; the crossover point was determined numerically.
f0 (2)

= f3 (1) P30 6 1 N = 6

f2 (2)

= f1 (1) P12 + f3 (1) P32 + fR (1)PR2 18(N 3) + 1 60 N 2) + 1 1 3(N 1)( N N N 3N 2 + 10N (N 12)

f1 (3)

= f2 (2) P21 + f4 (2) P41

22

3N 2 + 10N 6(N 2) + 1 24 N N N (3 N 2 + 10N )(6N 12) + 24N (N 1)(N 2) = N2 42N 3

N2

f3 (3)

= f0 (2) P03 + f2 (2) P23 + f4 (2) P43 + f6 (2) P63 6 1 + 3N 2 + 10N 6(N 2)(N 3) + 1 36(N 4) + 1 120

54N 4

N2

f0 (4)

= f3 (3) P30 54N 4 6 324N 4

N3 N N3

f2 (4)

= f1 (3) P12 + f3 (3) P32 + fR (3)PR2 42N 3 3(N 1)(N 2) + 54N 4 18(N 3) + 1 60 N2 N N2 N N 5 60 + 1098N (N 3) N N3 = f1 (3) P14 + f3 (3) P34 + fR (3) PR4 42N 3 1 + 54N 4 9(N 3)(N 4) + 1 60(N 5) + 210 N2 N2 N N 6 60 + 438N N N3 = f3 (3) P36 + fR (3) PR6 54N 4 1 + 1 15(N 5)(N 6) + 126(N 7) + 504 N2 N 15 N 2 15N 4 + 189N 3 294N 2 144N + = 15 25 N 2 + N (N 18)
N4 N2

f4 (4)

f6 (4)

N2

f3 (5)

= f0 (4) P03 + f2 (4) P23 + f4 (4) P43 + f6 (4) P63 324N 4 1 + 60 + 1098N 5 6(N 2)(N 3) + 60N + 438 N N3 4320N 8 + 25680N 7

N3

N6

N3

36(N 4) + 15N 2 + 25N 4 N N N2

120

f0 (6)

6 (5) N

N4

23

4320N 8 + 25680N 7 6 = 25920

N8

N4

N5

+ 154080N 7

This completes the proof of the lemma.

24