Fountain Codes

Viju Mathew
May 2, 2014
1 Abstract
For this project I am planning on studying a category of algorithms known as Fountain
Codes. The algorithm deals with the transfer of data and has two primary components.
The rst component encodes and sends data, and the second component receives and de-
codes data.
The encoding process uses random number generators to randomly pick and choose bits
of the le, combine the bits together, and encrypt them. This piece randomly generates a
limitless series of encoded pieces of the original le to send to the receiver.
The receiver takes advantage of the manner in which the bits of data are packaged to
decode the data. If it receives enough pieces of the original le (the exact number is based
on the algorithm used), these pieces can always be rearranged into the original le. The
aggregate size of the pieces always has to be a little bit more than the size of the original
le. No matter the order they are received in, they can be rearranged into the original le.
To extend the fountain metaphor further, these algorithms create a fountain of encoded
bits of some le; if enough droplets are collected, regardless of which ones are collected
or missed, the original le can be created.
We will be studying a type of fountain code knows as the Luby Transform code. Speci-
cally, we will be focusing on how the data is encoded, how the data is decoded, and how
the data that is encoded is chosen.
2 Background
Fountain codes have the potential to transform how we transfer data. Thus it has powerful
applications in the elds of communication and storage.
1
2.1 Online communication
Fountain codes can work very well to streamline peer to peer system communication and
reduce overall communication in server client systems. This is because it somewhat creates
a one-way communication protocol.
The server keeps on sending the packets of the original le and the receiver. The receiver
does not send any feedback about the packets it receives until this process has completed.
If the receiver gets a packet with an error, it is simply discarded. In typical TCP commu-
nications, the receiver sends acknowledgments, so if the connection is unreliable, it would
be almost impossible for communication to occur, but this is circumvented with the use of
fountain codes.
2.2 Data storage
One way that data can be stored is using a method called systematic form which means the
data can be read without reading from a storage unit; that is, the original message symbols
are part of the coded symbols. Furthermore, bandwidth and communication load could be
another bottleneck, so the previous section about the ecacy of online communication is
especially relevant.
3 Luby Transform
3.1 Overview
The Luby Transform (LT) Code was the rst class of practical fountain codes, and it was a
fairly recent development, published in 2002. It is not the most elaborate code, but through
studying it, we should be able to develop a solid foundation for better understanding
Fountain Codes.
3.2 Encoding
Imagine we are given a message that we want to send. We begin the encoding process
by dividing the original message into n blocks of roughly equal length. These are called
source blocks and are comprised of binary bits. Then, a number 1 d n is chosen from a
distribution, which is the degree of the encoded block, the number of source blocks in that
encoded block. This process is discussed in further detail in 3.6. Then, d source blocks are
chosen randomly and then combined together. The method of combining them together
presented in this paper is XOR (exclusive or) method. If M
i
is the i
th
block, the data
portion is
m
i
1
m
i
2
... m
i
d
(1)
2
where i
1
, i
2
, ..., i
d
are the indices for the d randomly chosen source blocks of this encoded
block.
Some additional information is added as a prex to the encoded packet to aid with decod-
ing. It reports the number of blocks n in the whole message, the number of blocks d in this
packet, and the list of indices. The last thing that is attached is a bit of code that allows
the receiver to check to see if the code that they received is corrupted at all.
The gist of the encoding process is that a random number of blocks are randomly cho-
sen, combined, and then sent over to the decoder.
3.3 Decoding
The decoding component is a little bit more complex. Let B
i
be an encoded block with
degree d
i
that is received by the decoder. The goal of the decoding process is to retrieve
the source blocks that were used to construct B
i
.
There are several dierent areas that keep blocks that are at dierent stages of the
decoding process. One of these is the message queuing area. This is where all the decoded
blocks are kept. There is also a message holding area, which keeps the blocks that are
unnished with the decoding process.
If d
i
= 1, then the encoded block is made up of exactly 1 source block which means it
is fully decoded. Thus, it is moved to the message queuing area. If d
i
> 1, then each block
in the message queuing area that was used to encode B
i
is XORed with B
i
.
Each time B
i
is XORed, d
i
is reduced by one, which is eectively keeping track of the
current degree of the encoded block; that is the number of blocks that are left to XOR
with it. And B
i
becomes the block that results from this operation. If after all the blocks
in the queue have been XORed and d
i
> 1, then B
i
is added to the holding area.
And nally, if the d
i
= 1, then B
i
has successfully been decoded! This is then added
to the message queuing area. And once there are n
i
(original number of blocks) blocks
in the message queuing area, the receiver tells the transmitter that the message has been
successfully decoded.
3.4 Decoding Explanation
The decoding process relies heavily on the properties of XOR, denoted by . Several of
these properties are listed below. Let A and B be arbitrary bitstrings. 0...0 is a bitstring
of 0s. Then:
3
1. A0...0 = A
(Additive identity)
2. AA = 0...0
(Additive inverse)
3. AB = B A
(Commutative property)
4. (AB) C = A(B C)
(Associative property)
And using these properties, if we have C = AB, then we can XOR B to both sides and
then apply the commutative property to see A = B C. Similarly B = A C. Thus we
have
5. if AB = C, then A = B C and B = AC.
Now we will apply properties to the decoding process! Imagine we are given an encoded
block B
i
made up of source blocks m
i
1
...m
i
d
XORed together. If B
i
is XORed with all the
m
i
1
...m
i
d
source blocks except for some m
i
k
, where 1 k d, then the resulting block
will be m
i
k
. This will be shown below.
So we have B
i
= (m
i
1
m
i
2
... m
i
d
). And now we XOR its source blocks except
for m
i
k
.
(m
i
1
... m
i
d
) (m
i
1
... m
i
k1
m
i
k+1
... m
i
d
) (2)
Applying properties 3 and 4, we can re-order and re-group the m
i
j
s so that the same
source blocks are grouped together. Thus we get
(m
i
1
m
i
1
) ... (m
i
k1
m
i
k1
) m
i
k
(m
i
k+1
m
i
k+1
)... (m
i
d
m
i
d
) (3)
And by applying property 2, we see that all of the pairs can be reduced to 0...0 (bitstrings
of 0). And we can apply property 1 to this, resulting in
(0...0) ... (0...0) m
i
k
(0...0)... (0...0) = m
i
k
(4)
Therefore, if we are given an encoded block and XOR it with all of its source blocks except
for one, we will end up with that source block. In some sense, this is a more complex
version of property 5.
4
3.5 Examples
To make some sense of this algorithm, we will go over some examples. During these
examples, we will alter some things from the real-world implementation of the algorithm so
that it will be easier to understand. We will imagine that all the encoded blocks arrive and
then decoding begins. In the actual algorithm, the encoded blocks are continuously arriving
and the decoder is continuously decoding. And as mentioned earlier, the actual algorithm
uses a message queuing and a message holding component, and this representation does
not include those.
3.5.1 Successful decoding
Imagine the receiver receives 5 encoded blocks, each one with a single byte of information
and information about which source blocks comprised each encoded block. In a graphical
view, our information would look like:
Figure 1: This has the encoded blocks we received on the left, and the source blocks being
on the right. The source blocks that make up the encoded blocks are indicated by the lines
from the encoded blocks to the source blocks.
Filling in the information we know and converting to binary this looks like:
5
Figure 2: Data has been converted to binary. Source block 1 decoded!
As we observe the connectivity of the image, we notice that the encoded blocks 2 and 3
only depend on source block 1 and one other block. Thus we can obtain the unknown
source blocks connected to these encoded blocks by XORing them with the source block 1.
For example, to obtain source block 2, we would XOR encoded block 2 with source block
1. And so for source block 1 we obtain
1001000 0101101 = 1100101. (5)
The process is carried out for source block 3, and the results are as follows:
Figure 3: Source blocks 1, 2, and 3 decoded.
6
To obtain source block 5 we XOR encoded block 4 with source blocks 2 and 3. We know
this results in source block 5 by our work in 3.4. The equation for this one looks like
1100110 (1100101 1101100) = 1100110 0001001 = 1101111. (6)
Thus source block 5 is 1101111. The result is illustrated fully below:
Figure 4: Source block 5 decoded
And nally we can decode the last source block as follows:
Figure 5: Decoding done!
7
Thus translating from binary to ASCII we get the original message hello. Granted this
was a very convenient example, but it serves to illustrate the manner in which the XOR
function operates.
3.5.2 Unsuccessful decoding
Now we will go over some situations where the decoding process is unsuccessful. During
these examples, we are just presented with a few encoded blocks, and we will imagine that
we just have those to decode. But in actual implentations of the algorithm, encoded blocks
are continuously being sent and continuously being decoded.
The rst example we consider is one where there are no source blocks of degree 1. This of-
ten means the decoding process cannot start. This could look something like the following
diagram:
Figure 6: No encoded blocks of degree 1
In the situation described above, it is impossible for the decoding process to start because
there are no source blocks given and no ways to get a source block from XORing them
together. Note that in the real world, blocks are being sent and decoded continuously, so
if the algorithm was run for a little longer, there is a chance that an encoded block with
d = 1 sent; thus allowing the decoding to start.
Problems can also occur after the decoding process starts. One of these would be if the
connectivity of the graph does not allow the propagation eect to fully occur. This is
illustrated in the series of events below:
8
Figure 7: The encoded blocks, the source blocks, and their connections
And so now we can carry through like we did in the rst example of this section.
Figure 8: Source block 1 decoded
And then we can XOR source block 1 with encoded block 3 to propagate the information.
9
Figure 9: Source block 3 decoded
But this is as far as the propagation goes because we do not have the information to keep
going. Even though we have a lot of data, we do not know where it exactly ts in because
we are missing information about the source blocks. In this example, if we knew what any
of the remaining source blocks were, we would have been able to continue the decoding
process and ultimately nish it.
Once again, note that blocks are being sent and decoded continuously in the real-world, so
if the algorithm was run for a little longer, there is a chance that an encoded block would
be sent that would allow the decoding process to continue. But there is an important issue
to keep in mind. The encoder will be continuously sending information to the decoder, so
there will be no shortage of information, but we do not have a lot of redundant information
that will cause the process to be less ecient. Thus, we consider degree distributions.
3.6 Degree distribution
The way the encoded blocks are constructed is integral to the success of the algorithm.
The number of source blocks in each encoded block needs to be considered, which could
be envisioned as the degree distribution, that is, the way the degrees of the encoded blocks
are distributed.
3.6.1 Ideal soliton distribution
In order for the LT code work, there needs to be a few encoded blocks with a degree of
1, so the decoding process can get started. And a distribution that should guarantee this
exists, and it is called the ideal soliton distribution. It is called ideal because it displays
10
ideal behavior in terms of the number of encoded blocks needed to recover the data. It
is expected that every time a new encoded block is received, a new source block will be
decoded, thus if there are N source blocks, N encoded blocks are sucient to recover all
of the data.
The probabilities in this distribution represent the probabilities that a block will have
a certain degree. So if p(2) = 1/2, there is a 50% probability that an encoded block will
have degree 2. Thus, it is a probability distribution on the integers from 1 to N, where N
is the number of source blocks. The probability mass function is
p(1) =
1
N
, (7)
p(k) =
1
k(k 1)
, (k = 2, 3, ..., N) (8)
And visually, the ideal soliton distribution looks like:
Figure 10: Ideal soliton distribution for k = 10, 50, 100
1
However, the ideal soliton distribution is not ideal when it is used in practice. It is often
the case that when the encoded blocks are being sent, they are constructed in such a way
that leads to halts in the decoding process. This could look something like the second
example from 3.5.2 where we saw the decoding process be unable to continue. While this
hiccup in the decoding process would probably be resolved if more encoded blocks were
received, it is costly to the eciency of the algorithm. And these hiccups denitely add up,
so in practice, this algorithm is not ideal. In order to overcome this, a dierent distribution
called robust soliton distribution is used.
1
Plot taken from Fountain Codes: Gauri Joshi, Joong Bum Rhim, John Sun, Da Wang
11
3.6.2 Robust soliton distribution
The robust soliton distribution is a modifcation of the ideal soliton distribution. Then the
Robust soliton distribution, J, is represented by the two distributions I and T:
J(i) =
I(i) + T(i)

(9)
where J(i) denotes the probability that an encoded block will have degree i, =
i
(I
i
+T
i
)
denotes a normalization factor, I is the ideal soliton distribution, and T is given by:
T(i) =
R
i N
, (i = 1, ..., N/R 1), (10)
T(i) =
Rln(R/)
N
, (i = N/R), (11)
T(i) = 0, (i = N/R + 1, ..., N) (12)
where N denotes the number of source blocks in the original message, denotes the allow-
able failure probability, which is the greatest chance that the decoder fails to recover the
data from a set of N encoded blocks. R denotes the expected size of the message holding
area, which holds the received encoded blocks that have not been fully decoded.
And visually, the robust soliton distribution looks like:
Figure 11: Robust soliton distribution with k = 100, = 0.1, c = 0.1, 0.2, 0.3. k denotes
the le size, which in equations above is denoted by N.
2
The c from the plot above comes from the expected ripple size which is:
R = c ln(N/)

(N) (13)
2
Plot taken from Fountain Codes: Gauri Joshi, Joong Bum Rhim, John Sun, Da Wang
12
We see that there is are two spikes in the distribution, which isnt true for the ideal dis-
tribution. This secondary spike means that there is a higher chance that the robust soliton
distribution would create blocks with higher degrees than the ideal soliton distribution.
This generates more blocks with few source blocks and a few blocks that use nearly all of
the source blocks. And this reduces the chances that there will be pauses in the decoding
process, thus improving upon the errors of the ideal soliton distribution.
4 Discussion
To dig deeper into the relationship between the ideal and robust soliton distributions,
we use another example. Imagine that we have a le divided into 10 blocks. Using the
information about the distributions from 3.6, we can calculate the degree distributions.
For the ideal soliton distribution we have:
Degree (d) p(d)
1
1
10
2
1
2
3
1
6
4
1
12
5
1
20
6
1
30
7
1
42
8
1
56
9
1
72
10
1
90
And if we choose R = 6, = .1, we can calculate the degree distribution of the robust
soliton distribution:
Degree (d) p(d)
1
7
50
2
4
25
3
11
150
4
7
150
5
17
500
6
799
1500
7
1
210
8
1
280
9
1
360
10
1
450
Now that we have the distributions, we can proceed. Imagine that we are now trying to
decode our le made up of 10 blocks. We are at the very end of the decoding process, for we
13
just have source block 7 left to decode. But we are stuck! We dont have any information
about source block 7, so we need to receive some kind of information about it to continue
decoding. We will calculate the probability of source block 7 being in the next decoded
block sent over for each of the distributions.
First, we have to decide how each encoded block is constructed. Assume that once the
degree of the encoded block is chosen, the specic source blocks that are included in that
encoded block are chosen randomly. This means that we can use a sum of probabilites
from a binomial distribution with probability of success =
1
10
, d, the degree of each en-
coded block, trials. The summation would include all trials that have 1 successes. The
results of this are:
Trials, d Total probability
1 .1
2 .19
3 .271
4 .3439
5 .40951
6 .468
7 .5217
8 .5695
9 .6126
10 .65132
To combine these results together, we multiply the probability of an encoded block of de-
gree d being chosen with the probability that a block of degree d would contain source
block 7. Thus for an encoded block of degree 2, we get
1
2
.19 = .095 for the ideal soliton
distribution and
1
5
.19 = .038 for the robust soliton distribution.
Summing these together on all of the possible degrees, we get that the chances of source
block 7 being included in the next block under the ideal soliton distribution as .2532 and
for the robust soliton distribution as .3833. From this, we see that the robust soliton dis-
tribution would help reduce the wait time between hiccups in the decoding process. But
this example was not entirely accurate because if an encoded block were to have the same
source block encoded in it more than once, the entire block would just be 0s. This example
was to illustrate some of the benets of using the robust soliton distribution when choosing
the degrees of the blocks.
14
5 Further research
Fountain codes are a new eld of research, thus they do have some shortcomings and
weaknesses.
5.1 Verifying Data
One large issue with fountain codes is that it is dicult to verify the data that is passed
along. There is no way to compute the hash (check the validity) of an encoded chunk, even
knowing the hashes of the individual chunks. The peers cannot be completely trusted as
they could lie about the validity of certain chunks.
The entire le could be checked for validity, then from the list of invalid source chunks, the
list of invalid encoded chunks could be created, but this is unreliable, dicult, and often
too late. The sender could use a public key and sign every block, but this means there can
only be one uploader, thus losing much of the benets that come from fountain codes. But
with something called Homomorphic Hashing, these issues can begin to be sorted out.
6 Conclusion
In this paper, we looked at some of the strengths and weakness of fountain codes and
possible applications of them. We studied a specic fountain code, called the Luby Trans-
form code, in more detail, looking closely at its encoding and decoding processes. We also
studied possible degree distributions of the encoding component of the LT code - resulting
in a simple example that indicated the eectiveness of the robust soliton distribution over
the ideal soliton distribution. I hope that this paper provided some interesting information
on fountain codes and sparked your interest in them!
15