CAPACITY APPROACHING CODES DESIGN AND IMPLEMENTATION

SPECIAL SECTION

Fountain codes
D.J.C. MacKay

Abstract: Fountain codes are record-breaking sparse-graph codes for channels with erasures, such
as the internet, where files are transmitted in multiple small packets, each of which is either received
without error or not received. Standard file transfer protocols simply chop a file up into K packet-
sized pieces, then repeatedly transmit each packet until it is successfully received. A back channel is
required for the transmitter to find out which packets need retransmitting. In contrast, fountain
codes make packets that are random functions of the whole file. The transmitter sprays packets at
the receiver without any knowledge of which packets are received. Once the receiver has received
any N packets, where N is just slightly greater than the original file size K, the whole file can be
recovered. In the paper random linear fountain codes, LT codes, and raptor codes are reviewed.
The computational costs of the best fountain codes are astonishingly small, scaling linearly with the
file size.

1 Erasure channels forward channel is (1
f )l bits, whether or not we have

feedback. Reliable communication should be possible at
Channels with erasures are of great importance. For this rate, with the help of an appropriate forward error-
example, files sent over the internet are chopped into correcting code.
packets, and each packet is either received without error or The wastefulness of the simple retransmission protocols is
not received. Noisy channels to which good error-correcting especially evident in the case of a broadcast channel with
codes have been applied also behave like erasure channels: erasures; channels where one sender broadcasts to many
much of the time, the error-correcting code performs receivers, and each receiver receives a random fraction
perfectly; occasionally, the decoder fails, and reports that it (1
f ) of the packets. If every packet that is missed by one
has failed, so the receiver knows the whole packet has been or more receivers has to be retransmitted, those retransmis-
lost. A simple channel model describing this situation is a sions will be terribly redundant. Every receiver will have
q-ary erasure channel (Fig. 1), which has ( for all inputs in already received most of the retransmitted packets.
the input alphabet f0; 1; 2; . . . ; q
1g) a probability 1
f So, we would like to make erasure-correcting codes that
of transmitting the input without error, and probability f of require no feedback or almost no feedback. The classic
delivering the output ‘?’. The alphabet size q is 2l, where l is block codes for erasure correction are called Reed–Solomon
the number of bits in a packet. codes [1, 2]. An (N, K) Reed–Solomon code (over an
Common methods for communicating over such chan- alphabet of size q ¼ 2l ) has the ideal property that if any K
nels employ a feedback channel from receiver to sender that of the N transmitted symbols are received then the original
is used to control the retransmission of erased packets. For K source symbols can be recovered (Reed–Solomon codes
example, the receiver might send back messages that exist for Noq). However, Reed–Solomon codes have the
identify the missing packets, which are then retransmitted. disadvantage that they are practical only for small K, N,
Alternatively, the receiver might send back messages that and q: standard implementations of encoding and decoding
acknowledge each received packet; the sender keeps track of have a cost of order K(N
K) log2 N packet operations.
which packets have been acknowledged and retransmits the Furthermore, with a Reed–Solomon code, as with any
others until all packets have been acknowledged. block code, one must estimate the erasure probability f and
These simple retransmission protocols have the advan- choose the code rate R ¼ K/N before transmission. If we are
tage that they will work regardless of the erasure probability unlucky and f is larger than expected and the receiver
f, but purists who have learned their Shannon theory will receives fewer than K symbols, what are we to do? We
feel that these protocols are wasteful. If the erasure would like a simple way to extend the code on the fly to
probability f is large, the number of feedback messages create a lower-rate (N0 , K) code. For Reed–Solomon codes,
sent by the first protocol will be large. Under the second no such on-the-fly method exists.
protocol, it is likely that the receiver will end up receiving There is a better way, pioneered by Michael Luby
multiple redundant copies of some packets, and heavy use is (2002) [3, 4].
made of the feedback channel. According to Shannon, there
is no need for the feedback channel: the capacity of the
2 Fountain codes
r IEE, 2005
IEE Proceedings online no. 20050237 The encoder of a fountain code is a metaphorical fountain
doi:10.1049/ip-com:20050237 that produces an endless supply of water drops (encoded
Paper received 23rd May 2005 packets); let us say the original source file has a size of Kl
The author is with Cavendish Laboratory, University of Cambridge, bits, and each drop contains l encoded bits. Now, anyone
Cambridge, UK who wishes to receive the encoded file holds a bucket under
E-mail: [email protected] the fountain and collects drops until the number of drops in
1062 IEE Proc.-Commun., Vol. 152, No. 6, December 2005
 1− f original generator matrix
000 000
f
001 001

010 010

011 011 K

100 100

101 101

110 110
transmitted packets
111 111

Fig. 1 An erasure channel – the 8-ary erasure channel received packets

The eight possible inputs f0; 1; 2; . . . ; 7g are here shown by the binary
packets 000; 001; 010; . . . ; 111

the bucket is a little larger than K. They can then recover the
original file. K
Fountain codes are rateless in the sense that the number
of encoded packets that can be generated from the source
message is potentially limitless; and the number of encoded
packets generated can be determined on the fly. Fountain
codes are universal because they are simultaneously near-
optimal for every erasure channel. Regardless of the N
statistics of the erasure events on the channel, we can send Fig. 2 The generator matrix of a random linear code
as many encoded packets as are needed in order for the When the packets are transmitted, some are not received, shown by
decoder to recover the source data. The source data can be the grey shading of the packets and the corresponding columns in the
decoded from any set of K0 encoded packets, for K0 slightly matrix. We can realign the columns to define the generator matrix,
larger than K. Fountain codes can also have fantastically from the point of view of the receiver (bottom)
small encoding and decoding complexities.
To start with, we will study the simplest fountain codes,
which are random linear codes.
of the packet. As long as the packet size l is much bigger
than the key size (which need only be 32 bits or so), this key
3 The random linear fountain introduces only a small overhead cost. In some applications,
every packet will already have a header for other purposes,
Consider the following encoder for a file of size K packets
which the fountain code can use as its key. For brevity, let’s
s1 ; s2 ; . . . ; sK . A ‘packet’ here is the elementary unit that is
call the K–by–N matrix fragment ‘G ’ from now on.
either transmitted intact or erased by the erasure channel.
Now, as we were saying, what is the chance that the
We will assume that a packet is composed of a whole
receiver will be able to recover the entire source file without
number of bits.
error?
At each clock cycle, labelled by n, the encoder generates
If NoK, the receiver has not got enough information to
K random bits {Gkn}, and the transmitted packet tn is set to
recover the file. If N ¼ K, it is conceivable that he can
the bitwise sum, modulo 2, of the source packets for which
recover the file. If the K–by–K matrix G is invertible
Gnk is 1.
(modulo 2), the receiver can compute the inverse G
1 by
X
K Gaussian elimination, and recover
tn ¼ sk Gkn ð1Þ
k¼1 X
N
sk ¼ tn G
1
nk ð2Þ
This sum can be done by successively exclusive-or-ing the n¼1
packets together. You can think of each set of K random
bits as defining a new column in an ever growing binary So, what is the probability that a random K–by–K binary
generator matrix, as shown at the top of Fig. 2. matrix is invertible? It is the product of K probabilities, each
Now, the channel erases a bunch of the packets; a of them the probability that a new column of G is linearly
receiver, holding out his bucket, collects N packets. What is independent of the preceding columns. The first factor, is
the chance that the receiver will be able to recover the entire (1
2
K), the probability that the first column of G is not the
source file without error? Let us assume that he knows the all-zero column. The second factor is (1
2
(K
1)), the
fragment of the generator matrix G associated with his probability that the second column of G is equal neither
packets, for example, maybe G was generated by a to the all-zero column nor to the first column of G, what-
deterministic random-number generator, and the receiver ever non-zero column it was. Iterating, the probability
has an identical generator that is synchronised to the of invertibility is ð1
2
K Þð1
2
ðK
1Þ Þ      ð1
18Þ
encoder’s. Alternatively, the sender could pick a random ð1
14Þð1
12Þ, which is 0.289, for any K larger than 10.
key, kn, given which the K bits fGkn gKk¼ 1 are determined by That is not great (we would have preferred 0.999!) but it is
a pseudo-random process, and send that key in the header promisingly close to 1.

IEE Proc.-Commun., Vol. 152, No. 6, December 2005 1063

 What if N is slightly greater than K? Let N ¼ K+E, 2. If we throw three times as many balls as there are bins, is
where E is the small number of excess packets. Our question it likely that any bins will be empty? Roughly how many
now is, what is the probability that the random K–by–N balls must be thrown for it to be likely that every bin has
binary matrix G contains an invertible K–by–K matrix? Let a ball?
us call this probability 1
d, so that d is the probability that 3. Show that in order for the probability that all K
the receiver will not be able to decode the file when E excess bins have at least one ball to be 1
d, we require
packets have been received. This failure probability d is N ’K loge ðK=dÞ balls.
plotted against E for the case K ¼ 100 in Fig. 3 (it looks
identical for all K410). For any K, the probability of failure
is bounded above by Rough calculations like these are often best solved by
finding expectations instead of probabilities. Instead of
dðEÞ  2
E ð3Þ finding the probability distribution of the number of empty
This bound is shown by the thin dotted line in Fig. 3. bins, we find the expected number of empty bins. This is
easier because means add, even where random variables are
correlated.
10 0
The probability that one particular bin is empty after N
1.0
balls have been thrown is
0.9 10 −1

1 N
0.8 10 −2 1
’ e
N=K ð4Þ
K
probability of failure

0.7
10 −3
0.6 So when N ¼ K, the probability that one particular bin is
10 −4
0.5 empty is roughly 1/e, and the fraction of empty bins must be
0.4 10 −5 roughly 1/e too. If we throw a total of 3K balls, the empty
0.3 10 −6 fraction drops to 1/e3, about 5%. We have to throw a lot of
balls to make sure all the bins have a ball! For general N,
0.2 10 −7 0 5 10 15 20 the expected number of empty bins is
0.1
0 Ke
N =K ð5Þ
0 2 4 6 8 10
number of redundant packets This expected number is a small number d (which roughly
implies that the probability that all bins have a ball is (1
d))
Fig. 3 Performance of the random linear fountain
The solid line shows the probability that complete decoding is not only if
possible as a function of the number of excess packets, E. The thin K
dashed line shows the upper bound, 2
E, on the probability of error N 4K loge ð6Þ
d

5 The LT code
In summary, the number of packets required to have
probability 1
d of success is ’K þ log2 1=d. The expected The LT code retains the good performance of the random
encoding cost per packet is K/2 packet operations, since on linear fountain code, while drastically reducing the encoding
average half of the packets must be added up (a packet and decoding complexities. You can think of the LT code as
operation is the exclusive-or of two packets of size l bits). a sparse random linear fountain code, with a super-cheap
The expected decoding cost is the sum of the cost of the approximate decoding algorithm.
matrix inversion, which is about K3 binary operations, and
the cost of applying the inverse to the received packets, 5.1 Encoder
Each encoded packet tn is produced from the source file
which is about K2/2 packet operations.
While a random code is not in the technical sense a s1 ; s2 ; s3 ; . . . sK as follows:
‘perfect’ code for the erasure channel (it has only a chance 1. Randomly choose the degree dn of the packet from a
of 0.289 of recovering the file when K packets have arrived), degree distribution r(d ); the appropriate choice of r
it is almost perfect. An excess of E packets increases the depends on the source file size K, as we will discuss later.
probability of success to at least (1
d), where d ¼ 2
E. 2. Choose, uniformly at random, dn distinct input packets,
Thus, as the file size K increases, random linear fountain and set tn equal to the bitwise sum, modulo 2, of those dn
codes can get arbitrarily close to the Shannon limit. The packets.
only bad news is that their encoding and decoding costs are
quadratic and cubic in the number of packets encoded. This This encoding operation defines a graph connecting
scaling is not important if K is small (less than one encoded packets to source packets. If the mean degree d

thousand, say); but we would prefer a solution with lower is significantly smaller than K then the graph is sparse. We
computational cost. can think of the resulting code as an irregular low-density
generator-matrix code.
4 Intermission 5.2 Decoder
Decoding a sparse-graph code is especially easy in the case
Before we study better fountain codes, it will help to solve of an erasure channel. The decoder’s task is to recover s
the following exercises. Imagine that we throw balls from t ¼ sG, where G is the matrix associated with the
independently at random into K bins, where K is a large graph ( just as in the random linear fountain code, we
number such as 1000 or 10 000. assume the decoder somehow knows the pseudorandom
matrix G ).
1. After N ¼ K balls have been thrown, what fraction of the The simple way to attempt to solve this problem is by
bins do you expect have no balls in them? message passing. We can think of the decoding algorithm as
1064 IEE Proc.-Commun., Vol. 152, No. 6, December 2005
the sum–product algorithm [5, Chaps. 16, 26 and 47] if we a). We set that source bit s1 accordingly (panel b), discard
wish, but all messages are either completely uncertain or the check node, then add the value of s1 (1) to the checks to
completely certain. Uncertain messages assert that a which it is connected (panel c), disconnecting s1 from the
message packet sk could have any value, with equal graph. At the start of the second iteration (panel c), the
probability; certain messages assert that sk has a particular fourth check node is connected to a sole source bit, s2. We
value, with probability one. set s2 to t4 (0, in panel d), and add s2 to the two checks it is
This simplicity of the messages allows a simple descrip- connected to (panel e). Finally, we find that two check
tion of the decoding process. We will call the encoded nodes are both connected to s3, and they agree about the
packets tn check nodes. value of s3 (as we would hope!), which is restored in panel f.
1. Find a check node tn that is connected to only one source 5.3 Designing the degree distribution
packet sk (if there is no such check node, this decoding The probability distribution r(d) of the degree is a critical
algorithm halts at this point, and fails to recover all the part of the design: occasional encoded packets must have
source packets). high degree (i.e., d similar to K) in order to ensure that there
(a) Set sk ¼ tn. are not some source packets that are connected to no-one.
(b) Add sk to all checks tn0 that are connected to sk: Many packets must have low degree, so that the decoding
process can get started, and keep going, and so that the
tn0 : ¼ tn0 þ sk for all n0 such that Gn0 k ¼ 1: total number of addition operations involved in the
encoding and decoding is kept small. For a given degree
(c) Remove all the edges connected to the source distribution r(d), the statistics of the decoding process can
packet sk. be predicted by an appropriate version of density evolution,
2. Repeat (1) until all sk are determined. a technique first developed for low-density parity-check
codes [5, p. 566].
Before giving Luby’s choice for r(d), let us think about
This decoding process is illustrated in Fig. 4 for a toy case
the rough properties that a satisfactory r(d) must have. The
where each packet is just one bit. There are three source
encoding and decoding complexity are both going to scale
packets (shown by the upper circles) and four received
linearly with the number of edges in the graph, so the
packets (shown by the lower check symbols), which have
crucial quantity is the average degree of the packets. How
the values t1 ; t2 ; t3 ; t4 ¼ 1011 at the start of the algorithm.
small can this be? The balls-in-bins exercise helps here: think
At the first iteration, the only check node that is
of the edges that we create as the balls and the source
connected to a sole source bit is the first check node (panel
packets as the bins. In order for decoding to be successful,
every source packet must surely have at least one edge in it.
The encoder throws edges into source packets at random,
a s1 s2 s3 so the number of edges must be at least of order K loge K.
If the number of packets received is close to Shannon’s
optimal K, and decoding is possible, the average degree of
+ + + +
each packet must be at least loge K, and the encoding and
decoding complexity of an LT code will definitely be at
1 0 1 1 least K loge K. Luby showed that this bound on complexity
b 1 can indeed be achieved by a careful choice of degree
distribution.
Ideally, to avoid redundancy, we would like the received
+ + + graph to have the property that just one check node has
0 1 1
degree one at each iteration. At each iteration, when this
c
check node is processed, the degrees in the graph are
1
reduced in such a way that one new degree-one check node
appears. In expectation, this ideal behaviour is achieved by
the ideal soliton distribution,
+ + +
rð1Þ ¼ 1=K
1 1 0
1 ð7Þ
d 1 0 rðdÞ ¼ for d ¼ 2; 3; . . . ; K
dðd
1Þ
The expected degree under this distribution is roughly
+ + loge K.
1 1 This degree distribution works poorly in practice, because
e 1 0
fluctuations around the expected behaviour make it very
likely that at some point in the decoding process there will
be no degree-one check nodes; and, furthermore, a few
source nodes will receive no connections at all. A small
+ + modification fixes these problems.
1 1 The robust soliton distribution has two extra parameters,
f 1 0 1 c and d; it is designed to ensure that the expected number of
degree-one checks is about
pffiffiffiffi
S  c loge ðK=dÞ K ð8Þ
Fig. 4 Example decoding for a fountain code with K ¼ 3 source
bits and N ¼ 4 encoded bits rather than 1, throughout the decoding process. The
From [5] parameter d is a bound on the probability that the decoding

IEE Proc.-Commun., Vol. 152, No. 6, December 2005 1065

fails to run to completion after a certain number K0 of 140
packets have been received. The parameter c is a constant of delta = 0.01
order 1, if our aim is to prove Luby’s main theorem about 120 delta = 0.1
LT codes; in practice however it can be viewed as a free delta = 0.9
parameter, with a value somewhat smaller than 1 giving 100
good results. We define a positive function
8 80
> s1
>
> for d ¼ 1; 2; . . . ; ðK=SÞ
1
<K d 60
tðdÞ ¼ s logðS=dÞ for d ¼ K=S
>
> K
>
: 40
0 for d4K=S
20
ð9Þ
(see Fig. 5) then add the ideal soliton distribution r to t and 0
10 −2 10 −1
normalise to obtain the robust soliton distribution, m: a
rðdÞ þ tðdÞ
mðdÞ ¼ ð10Þ 11 000
Z
delta = 0.01
where Z ¼ Sd rðdÞ þ tðdÞ. The number of encoded
10 800 delta = 0.1
packets required at the receiving end to ensure that the
decoding can run to completion, with probability at least delta = 0.9
1
d, is K 0 ¼ KZ. 10 600

0.5
rho 10 400
tau

0.4 10 200

10 000
0.3 10 −2 10 −1
b c

Fig. 6 The number of degree-one checks S and the quantity K0

against the two parameters c and d, for K ¼ 10 000
0.2 a Number of degree-one checks S
b Quantity K0
Luby’s main theorem proves that there exists a value of c such that,
given K0 received packets, the decoding algorithm will recover the K
0.1 source packets with probability 1
d. From [5]

K packets have been received, at which point, an avalanche

0 of decoding takes place.
0 10 20 30 40 50

Fig. 5 The distributions r(d) and t(d) for the case K ¼ 10 000, 6 Raptor codes
c ¼ 0:2, d ¼ 0.05, which gives S ¼ 244, K/S ¼ 41, and Z’1:3
The distribution t is largest at d ¼ 1 and d/K ¼ S. From [5] You might think that we could not do any better than LT
codes: their encoding and decoding costs scale as K loge K,
where K is the file size. But raptor codes [6] achieve linear
Luby’s analysis [3] explains how the small-d end of t has time encoding and decoding by concatenating a weakened
the role of ensuring that the decoding process gets started, LT code with an outer code that patches the gaps in the LT
and the spike in t at d ¼ K/S is included to ensure that every code.
source packet is likely to be connected to a check at least LT codes had decoding and encoding complexity that
once. Luby’s key result is that ( for an appropriate value of scaled as loge K per packet, because the average degree of
the constant c) receiving K 0 ¼ K þ 2 loge ðS=dÞS checks the packets in the sparse graph was loge K. Raptor codes
ensures that all packets can be recovered with probability at use an LT code with average degree d
about 3. With this
least 1
d. In the illustrative Figures (Figs. 6a and b) the lower average degree, the decoder may work in the sense
allowable decoder failure probability d has been set quite that it does not get stuck, but a fraction of the source
large, because the actual failure probability is much smaller packets will not be connected to the graph and so will
than is suggested by Luby’s conservative analysis. not be recovered. What fraction? From the balls-in-bins
In practice, LT codes can be tuned so that a file of

original size K ’ 10 000 packets is recovered with an exercise, the expected fraction not recovered is f~  e
d ,
overhead of about 5%. Figure 7 shows histograms of the which for d
¼ 3 is 5%. Moreover, if K is large, the law of
actual number of packets required for a couple of settings of large numbers assures us that the fraction of packets not
the parameters, achieving mean overheads smaller than 5% recovered in any particular realisation will be very close to
and 10% respectively. Figure 8 shows the time-courses of f~. So, here is Shokrollahi’s trick: we transmit a K-packet file
three decoding runs. It is characteristic of a good LT code by first pre-coding the file into K ~ ’ K=ð1
f~Þ packets with
that very little decoding is possible until slightly more than an excellent outer code that can correct erasures if the
1066 IEE Proc.-Commun., Vol. 152, No. 6, December 2005
 K = 16

10 000 10 500 11 000 11 500 12 000

a

10 000 10 500 11 000 11 500 12 000

b

+ + + + + + + + + + + + + + + + + +

N = 18

Fig. 9 Schematic diagram of a raptor code

In this toy example, K ¼ 16 source packets (top row) are encoded by
the outer code into K ~ ¼ 20 pre-coded packets (centre row). The
10 000 10 500 11 000 11 500 12 000 details of this outer code are not given here. These packets are encoded
c into N ¼ 18 received packets (bottom row) with a weakened LT code.
Most of the received packets have degree 2 or 3. The average degree is
Fig. 7 Histograms of the actual number of packets N required in 3. The weakened LT code fails to connect some of the pre-coded
order to recover a file of size K ¼ 10 000 packets packets to any received packet – these 3 lost packets are highlighted in
a c ¼ 0.01, d ¼ 0.5 (S ¼ 10, K/S ¼ 1010, and Z’1:01)
grey. The LT code recovers the other 17 pre-coded packets, then the
b c ¼ 0.03, d ¼ 0.5 (S ¼ 30, K/S ¼ 337, and Z’1:03)
outer code is used to deduce the original 16 source packets
c c ¼ 0.1, d ¼ 0.5 (S ¼ 99, K/S ¼ 101, and Z’1:1)
From [5]

10 000
max degree 8
10 000
max degree K
8000
8000

6000
number decoded

6000

4000
4000

2000
2000

0
0 0 2000 4000 6000 8000 10 000 12 000
0 2000 4000 6000 8000 10 000 12 000
Fig. 10 The idea of a weakened LT code
Fig. 8 Practical performance of LT codes The LT degree distribution with parameters c ¼ 0.03, d ¼ 0.5 is
Three experimental decodings are shown, all for codes created with the truncated so that the maximum degree to be 8. The resulting graph has
parameters c ¼ 0.03, d ¼ 0.5 (S ¼ 30, K/S ¼ 337, and Z’1:03) and a mean degree 3. The decoder is run greedily as packets arrive. As in
file of size K ¼ 10 000. The decoder is run greedily as packets arrive. Fig. 8, the thick lines show the number of recovered packets as a
The vertical axis shows the number of packets decoded as a function of function of the number of received packets. The thin lines are the
the number of received packets. The right-hand vertical line is at a curves for the original LT code from Fig. 8. Just as the original LT
number of received packets N ¼ 11 000, i.e., an overhead of 10% code usually recovers K ¼ 10 000 packets within a number of received
packets N ¼ 11 000, the weakened LT code recovers 8000 packets
within a received number of 9250

erasure rate is exactly f~; then we transmit this slightly

enlarged file using a weak LT code that, once slightly more For our excellent outer code, we require a code that can
than K packets have been received, can recover ð1
f~ÞK ~ of correct erasures at a known rate of 5% with low decoding
the pre-coded packets, which is roughly K packets; then we complexity. Shokrollahi uses an irregular low-density
use the outer code to recover the original file (Fig. 9). parity-check code. For further information about irregular
Figure 10 shows the properties of a crudely weakened LT low-density parity-check codes, and fast encoding algo-
code. Whereas the original LT code usually recovers rithms for them, see [5, pp. 567–572] and [7, 8].
K ¼ 10 000 packets within a number of received packets
N ¼ 11 000, the weakened LT code usually recovers 8000 7 Applications
packets within a received number of 9250. Better per-
formance can be achieved by optimising the degree Fountain codes are an excellent solution in a wide variety of
distribution. situations. Here we mention two.

IEE Proc.-Commun., Vol. 152, No. 6, December 2005 1067

7.1 Storage every packet. Thus the broadcaster would have to repeat
You wish to make a back-up of a large file, but you are the entire broadcast twice in order to ensure that most
aware that your magnetic tapes and hard drives are all subscribers have received the whole movie, and most users
unreliable: catastrophic failures, in which some stored would have to wait roughly twice as long as the ideal time
packets are permanently lost within one device, occur at a before the download was complete.
rate of something like 10
3 per day. How should you store If the broadcaster uses a fountain code to encode the
your file? movie, each subscriber can recover the movie from any
A fountain code can be used to spray encoded packets all K 0 ’ K packets. So the broadcast needs to last for only, say,
over the place, on every storage device available. To recover 1.1K packets, and every house is very likely to have
the file, whose size was K packets, one simply needs to find successfully recovered the whole file.
K 0 ’ K packets from anywhere. Corrupted packets do not Another application is broadcasting data to cars. Imagine
matter; we simply skip over them and find more packets that we want to send updates to in-car navigation databases
elsewhere. by satellite. There are hundreds of thousands of vehicles,
This method of storage also has advantages in terms of and they can only receive data when they are out on the
speed of file recovery. In a hard drive, it is standard practice open road; there are no feedback channels. A standard
to store a file in successive sectors of a hard drive, to allow method for sending the data is to put it in a carousel,
rapid reading of the file; but if, as occasionally happens, a broadcasting the packets in a fixed periodic sequence. ‘Yes,
packet is lost (owing to the reading head being off track for a car may go through a tunnel, and miss out on a few
a moment, giving a burst of errors that cannot be corrected hundred packets, but it will be able to collect those missed
by the packet’s error-correcting code), a whole revolution of packets an hour later when the carousel has gone through a
the drive must be performed to bring back the packet to the full revolution (we hope); or may be the following day y’.
head for a second read. The time taken for one revolution If instead the satellite uses a fountain code, each car needs
produces an undesirable delay in the file system. If files were to receive only an amount of data equal to the original file
instead stored using the fountain principle, with the digital size (plus 5%).
drops stored in one or more consecutive sectors on the
drive, then one would never need to endure the delay of re-
reading a packet; packet loss would become less important, 8 References
and the hard drive could consequently be operated faster,
with higher noise level, and with fewer resources devoted to 1 Berlekamp, E.R.: ‘Algebraic coding theory’ (McGraw-Hill, New York,
noisy-channel coding. 1968)
2 Lin, S., and Costello, D.J. Jr.: ‘Error control coding: fundamentals and
applications’ (Prentice-Hall, Englewood Cliffs, New Jersey, 1983)
7.2 Broadcast 3 Luby, M.: ‘LT codes’. Proc. 43rd Ann. IEEE Symp. on Foundations
Imagine that ten thousand subscribers in an area wish to of Computer Science, 16–19 November 2002, pp. 271–282
4 Byers, J., Luby, M., Mitzenmacher, M., and Rege, A.: ‘A digital
receive a digital movie from a broadcaster. The broadcaster fountain approach to reliable distribution of bulk data’. Proc. ACM
can send the movie in packets over a broadcast network, for SIGCOMM’98, 2–4 September 1998
5 MacKay, D.J.C.: ‘Information theory, inference, and learning
example, by a wide-bandwidth phone line, or by satellite. algorithms’ (Cambridge University Press, 2003), Available from
Imagine that f ¼ 0.1% of the packets are lost at each house. www.inference.phy.cam.ac.uk/mackay/itila/
In a standard approach in which the file is transmitted as a 6 Shokrollahi, A.: ‘Raptor codes’. Technical report, Laboratoire
!
d’algorithmique, Ecole Polytechnique F!ed!erale de Lausanne,
plain sequence of packets with no encoding, each house Lausanne, Switzerland, 2003. Available from algo.epfl.ch/
would have to notify the broadcaster of the fK missing 7 Richardson, T., Shokrollahi, M.A., and Urbanke, R.: ‘Design of
packets, and request that they be retransmitted. And with capacity-approaching irregular low-density parity check codes’, IEEE
Trans. Inf. Theory, 2001, 47, (2), pp. 619–637
ten thousand subscribers all requesting such retransmis- 8 Richardson, T., and Urbanke, R.: ‘Efficient encoding of low-density
sions, there would be a retransmission request for almost parity-check codes’, IEEE Trans. Inf. Theory, 2001, 47, (2), pp. 638–656

1068 IEE Proc.-Commun., Vol. 152, No. 6, December 2005