Gossiping Theory

Gossiping Models
Formal Analysis of Epidemic Protocols
Rena Bakhshi
Gossiping Models
Copyright © 2011 by Rena Bakhshi
Cover design by Jörg Endrullis
Printed and bound by Wöhrmann Print Service, Zutphen
ISBN: 978-90-8570-710-3
IPA Dissertation Series 2011-01
The work reported in this thesis has been carried out at the Vrije Univer-
siteit Amsterdam under the auspices of the research school IPA (Institute
for Programming research and Algorithmics). The research was funded
by the Vrije Universiteit Amsterdam.
VRIJE UNIVERSITEIT
Gossiping Models
Formal Analysis of Epidemic Protocols
ACADEMISCH PROEFSCHRIFT
ter verkrijging van de graad Doctor aan

de Vrije Universiteit Amsterdam,
op gezag van de rector magnificus
prof.dr. L.M. Bouter,
in het openbaar te verdedigen
ten overstaan van de promotiecommissie
van de faculteit der Exacte Wetenschappen
op donderdag 13 januari 2011 om 11.45 uur
in de aula van de universiteit,
De Boelelaan 1105
door
Rena Bakhshi
geboren te Baki, Azerbeidzjan

promotoren: prof.dr. W.J. Fokkink
prof.dr.ir. M.R. van Steen
Acknowledgements
It is a pleasure to thank those who made the completion of this thesis possible.
First and foremost I express my gratitude to my supervisor Wan Fokkink for his
guidance and patience throughout my thesis, for sharing his ideas with me, and
all the support he has provided in these four years. Moreover, I want to thank
Wan and Judi for letting me stay at their house in the beginning of my PhD,
when there was no apartment available. I’m also very grateful to my second
promotor Maarten van Steen for his constant encouragement and for being an
infinite source of inspirational ideas. I’m truly thankful that despite his busy
schedule, he always found time for our meetings and discussions.
I thank the members of the reading committee: Roberto Baldoni, Verena Wolf,
Holger Hermanns, Spyros Voulgaris, and Boudewijn Haverkort for their time,
timely response and valuable comments. I thank Holger and Wan for inviting
me to the Dagstuhl seminars, where I met a lot of interesting people in a cosy,
friendly atmosphere.
I share the credit of my work, some of it presented in this thesis, with my
co-authors Lucia Cloth, Daniela Gavidia, Ansgar Fehnker, Jörg Endrullis, Stefan
Endrullis, Jun Pang, Jaco van de Pol and Boudewijn Haverkort. I thank them
for the great work together and hope to continue our fruitful cooperation in the
future.
I had the pleasure to visit NICTA in Australia for three months, and hang
around with Ansgar Fehnker, Rob van Glabbeek, Ralf Huuck, Carroll Morgan,
Franck Cassez, Clemens Dubslaff, Jane Gillis, and all others on 5th and 6th floor.
I really enjoyed the excellent working environment as well as all lunches, coffees,
BBQs, Friday beers, and useful travel tips. I specially want to thank Ansgar, Wan
and Jane for helping to organize my visit to Sydney.
In my daily work a wonderful group of colleagues created a warm and cheer-
ful environment: Taolue Chen, Jörg Endrullis, Wan Fokkink, Daniela Gavidia,
Fatemeh Ghassemi,Clemens Grabmayer, Willem van Hage, Helle Hansen, Dimitri
Hendriks, Maxim Hendriks, Ariya Isihara, Jan Willem Klop, Cynthia Kop, Femke
van Raamsdonk, Anna Tordai, Stefan Vijzelaar, and Roel de Vrijer. Our lively
vii
lunch discussions on numerous subjects have been an important element in my
“PhD life”. I enjoyed sharing an office with Willem, Helle, Anna, Taolue, and
Cynthia. I want to also thank Caroline Waij, Fenny Bosse, Ilse Thomson, Shirley
Chedi, Caroline van der Oort, Femke van den Bosch and Elly Lammers for being
so helpful with formalities at the VU.
I took part in several IPA events, PhD courses at the VU, summer schools,
Dagstuhl seminars, and Lorentz workshops. I enjoyed very much socializing
with the people I met during these events, including Melanie Eijberts, Alexan-
dra Silva, Adam Koprowski, Lăcrămioara Aştefănoaei, Stephanie Kemper, and
Paul van Tilburg. Special thanks to Melanie for sharing with me happiness and
sorrows, showing me Amsterdam, and for the friendship since we met at the PhD
success course.
Beyond Computer Science and the VU, I’m very thankful to my friends Rox-
ana, Ulja, Jen, Fuad, Elkhan, Ali, Donnie and many others who supported me in
many respects all these years, and for being such a nice friends. Guys, you rock!
I specially thank my family Leyla, Rafik, Rasim, Nargiz, Rashid, Natasha, Igor,
Marina, and my family-in-law Karin, Manfred, Stefan, Luzi, Maria, Katrin, Robby,
Madlen, Bernd, for their love and support. Finally, to Jörg for being so loving and
understanding, for sparing time to listen to my “thoughts aloud”, for his patience,
for designing my wonderful thesis cover, and much more...
Contents
Contents I
1 Introduction 1
1.1 Requirements for Gossip Protocols . . . . . . . . . . . . . . . . . 2
1.2 Formal Analysis Techniques . . . . . . . . . . . . . . . . . . . . . 4
1.3 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Model-Based Analysis Techniques . . . . . . . . . . . . . . . . . . 6
1.5 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.6 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.7 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . 13
1.8 Origins of the Chapters . . . . . . . . . . . . . . . . . . . . . . . . 15
2 Background 17
2.1 A Generic Gossip Protocol . . . . . . . . . . . . . . . . . . . . . . 17
2.2 A Gossip Protocol for Information Dissemination . . . . . . . . . 19
3 Pairwise Exchange Model without Failures 23

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 Experimental Observations . . . . . . . . . . . . . . . . . . . . . . 24
3.4 An Analytical Model of Information Dissemination . . . . . . . . 27
3.5 Another Expression for Pdrop . . . . . . . . . . . . . . . . . . . . . 34
3.6 Optimal Size for the Exchange Buffer . . . . . . . . . . . . . . . . 36
3.8 Time Complexity of Experiments . . . . . . . . . . . . . . . . . . 40
3.9 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.10 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4 Pairwise Exchange Model in Lossy Networks 45

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
I
4.2 Our Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.3 A Model of Information Spread with Lossy Channels . . . . . . . 48
4.4 The Probability of Dropping an Item . . . . . . . . . . . . . . . . 50
4.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5 Comparison of Performance Models 59

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.2 PRISM Model Checker . . . . . . . . . . . . . . . . . . . . . . . . 61
5.3 The Basic PRISM Model . . . . . . . . . . . . . . . . . . . . . . . 61
5.4 Simulation Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.5 Modelling with Differential Equations . . . . . . . . . . . . . . . 69
5.6 Difference Equation Model . . . . . . . . . . . . . . . . . . . . . . 77
5.7 Fractions and Probabilities . . . . . . . . . . . . . . . . . . . . . . 79
5.8 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.10 Appendix: Nondeterministic PRISM Model . . . . . . . . . . . . . 83
6 Mean-field Framework 85
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.2 Gossiping Time Protocol . . . . . . . . . . . . . . . . . . . . . . . 87
6.3 Mean-field Modelling and Convergence . . . . . . . . . . . . . . . 88
6.4 A Mean-field Model for the Shuffle Protocol . . . . . . . . . . . . 95
6.5 A Mean-field Model for Basic GTP . . . . . . . . . . . . . . . . . . 105
6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
7 Automating the Mean-Field for Dynamic Networks 117

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
7.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.3 Modelling Framework . . . . . . . . . . . . . . . . . . . . . . . . 119
7.4 Mean-field Modelling . . . . . . . . . . . . . . . . . . . . . . . . . 123
7.5 Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.6 Framework for Dynamic Networks . . . . . . . . . . . . . . . . . 132
7.7 Tool Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.9 Appendix: Specification of Basic GTP Protocol . . . . . . . . . . . 140
7.10 Appendix: Specification for Mobile Case . . . . . . . . . . . . . . 141
7.11 Appendix: Specification for Dynamic Case . . . . . . . . . . . . . 143
8 Concluding Remarks 145

8.1 Analytical Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
8.2 Industrial Importance . . . . . . . . . . . . . . . . . . . . . . . . 149
9 Summary of the Thesis 151
II
9.1 Samenvatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
9.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Bibliography 153
III
Introduction
1
The emergence of the Internet as a computing platform asks for new classes of
algorithms that combine massive distributed processing and inherent decentral-
ization. Moreover, these algorithms should be able to execute in an environment
that is heterogeneous, changes almost continuously, and consists of millions of
nodes. Massive parallel computing on the Internet also demands a degree of self-
organization; the number of devices and amount of software is simply too large
to be managed by humans.
Gossip protocols (sometimes referred to as epidemic or random-walk proto-
cols) have shown to be a sensible paradigm for developing stable and reliable
communication mechanisms that scale up to massively parallel environments. In
a gossip protocol, nodes exchange data similar to the way a contagious disease
spreads. That is, a participating peer can select, according to some probabil-
ity distribution, other peers to exchange information with. For example, sensor
nodes collectively monitor temperature inside one building. Every node mea-
sures the temperature of its area, and periodically communicates with a random
node in its neighborhood to exchange local temperature value. A node then
averages all received values.
Gossip protocols have originally been proposed for database replication [75],
but more recently also for failure detection [174], and resource monitoring [173].
They are employed in wired as well as wireless environments. In a massively par-
allel setting, the gossip mechanism should be used at very high speeds, yielding
a new generation of protocols that have an unusual style of probabilistic relia-
bility guarantees, regarding scalability, performance, and stability of throughput.
Surveys [83, 85, 115] provide an introduction to the field.
When a large number of nodes interact in a connected environment, various
phenomena occur that are not explicable in terms of the behavior of any single
agent. It is necessary to understand these phenomena in order to keep the over-
all systems both stable and efficient. Distributed algorithms and protocols that
run steadily and reliably in small-scale settings, tend to lose those properties as
numbers of users, the size of the network and transaction processing rates all
1
2 Chapter 1 — Introduction
increase. Typical problems are disruptive overloads and routing changes, periods
of poor connectivity and throughput instability. Failures rise in frequency simply
because the numbers of participating components are larger [175].
Gossip protocols tend to contain several design parameters that can influence
the nonfunctional properties of these protocols, for example, performance, ro-
bustness, and fault tolerance. Values of these parameters are usually determined
empirically, without a proper understanding why the protocol performs well for
these values, and without any certainty that these values are close to optimal
or robust choices. Thorough experimental analysis in [123] has shown that the
emergent behavior of gossip protocols may vary substantially by changing only a
few design parameters.
In practice, properties of gossip protocols are usually diagnosed by emulating
such systems on a single workstation. However, in principle, owing to their often
relatively simple structure, gossip protocols lend themselves very well to formal
analysis in order to predict their behavior with high confidence.
Next section gives an overview of the different types of requirements for gos-
sip protocols. Thereafter, we discuss the available spectrum of analysis tech-
niques for probabilistic systems, and related work for gossip protocols.
1.1 Requirements for Gossip Protocols

Requirements for gossip protocols can be divided into three classes: general,
functional and nonfunctional requirements. These will be discussed in the cur-
rent section. We use terminology from [123, 176].
1.1.1 General Requirements

Gossip protocols are meant to satisfy the following general requirements:
• Simplicity: The protocol is simple and easy to deploy. For example, in a wireless
network, a node should be able to join the system without executing a complex
procedure (“plug-and-play”).
• Scalability: Each node continues to perform its operations at almost the same
rate irrespective of the network size. For example, the local knowledge (neigh-
bors list) of a node does not increase with the network size.
• Symmetry: In a large-scale network, all nodes play identical roles. Hence,
there is no single point of failure. Randomization, for example, random peer
selection, tends to fit into this requirement, because each node typically runs
the same algorithm.
1.1.2 Functional Requirements

Functional requirements describe properties of the outputs of a system and
how certain input is transformed into specific output. We classify several require-
1.1. Requirements for Gossip Protocols 3
ments on gossip protocols, and distinguish between global and local properties.
1. Global properties of the system:
• Connectivity: This can, for example, be expressed as a minimum number
of links between nodes whose removal will result in the partitioning of the
network graph. The connectivity of a graph is an important measure of
how proper it is maintained in the presence of churn, because partitioning
of a network graph creates difficulties for information dissemination.
• Convergence: One can distinguish between convergence of the system pa-
rameters to some values (for example, achieving a certain accuracy in the
estimates of the aggregate function values) and convergence of the system
structure to some particular type of graph.
2. Local properties of nodes:
• Degree distribution: The degree of a node is the number of its neighbors in
the network graph. This concept is interesting because of its relationship to
the correct system operation on a graph in the presence of node churn, its
effect on patterns of epidemic spread, and its importance in the distribution
of resource usage of nodes.
• Clustering coefficient: The clustering coefficient of a node expresses the ra-
tio of the number of links between the node’s neighbors to the number of all
possible links between them. Intuitively, it shows how many neighbors of a
node are neighbors among themselves. Analysis of this property is interest-
ing because a high clustering coefficient affects information dissemination,
as the number of redundant messages increases. It also affects the self-
healing capacity by increasing the number of links to the node neighbor-
hood from the other nodes, thus decreasing the probability of partitioning.
• Shortest path length: The shortest path length between two nodes is the
minimum number of edges that must be traversed to go from one node
to the other. The average path length is the average of all shortest path
lengths between any two nodes in a graph. The shortest and average path
length give information on the time and communication costs to reach a
node.
1.1.3 Nonfunctional Requirements

Nonfunctional requirements regard the quality (for example, performance,
maintainability, fault tolerance) and economics (for example, timing, cost) of
system behavior. The following high-level nonfunctional requirements can be
identified for gossip protocols:
Time complexity: The number of time units it takes (at worst or on average) for
a gossip protocol to “infect” every node in the network, for example, for data
delivery to all nodes, or for computation of an aggregation function output.
Message complexity: The total number of gossip messages (at worst or on aver-
age) exchanged over the network during an execution.
Information dissemination: For a given time span, there is a a high probability
that a piece of information is shared with all processes.
Robustness: The ability of a gossip protocol to maintain correct system operation
in the face of massive node crashes and node churn.
Graceful degradation: Large numbers of node or link failures in the system may
affect its operation. However, performance, functionality and reliability of
gossip protocols should not drop rapidly as the number of failures increases.
Elasticity: Robustness of the overall system behavior in the face of largely vary-
ing node capabilities in terms of memory, bandwidth and connectivity.
Self-organization: The nodes should be able to organize themselves in unpre-
dictable circumstances without external interventions. For example, in gos-
sip protocols a network graph forms overlays that are adaptable to network
and environmental changes.
Specifically in wireless networks, nodes communicate through error-prone ra-
dio channels and typically also have limited computational capabilities. Special
design issues then include energy usage, mobility, transmission power, mem-
ory usage and latency. Network properties such as message reliability and node
reachability may in that case influence the behavior of the protocol.
Next, we give an overview of the different approaches that can be taken to
formally analyze gossip protocols.
1.2 Formal Analysis Techniques

When analyzing a system we generally aim to obtain a better understanding
of, and gain further confidence in the system’s behavior, to detect possible errors,
and to improve its design. A complication in the analysis of gossip protocols
is that they are meant to work in very large networks, and for ad hoc wireless
networks even with lossy channels. Properties of such protocols are generally
diagnosed by emulating such networks.
The formal specification of systems helps to make explicit the underlying as-
sumptions (like the synchronization primitives), which tend to remain hidden
in implementations or simulation exercises. Also such a specification can be an-
alyzed using (semi-)automated formal verification techniques. There is a rich
history of the use of such techniques for verifying a variety of desirable proper-
ties for a wide range of systems. The efficiency of a particular formal analysis
technique depends on the system under study.
Gossip protocols in general contain several design parameters that heavily
influence their behavior, such as, the number of protocol cycles, message for-
warding strategies, message delays, or cache usage and size. Formal analysis
techniques can help in the search for optimal values of such parameters.
Rigorous formal analysis techniques for gossip protocols have so far hardly
been applied in large-scale settings.The main scope of the thesis is to investigate
1.3. Experimental Evaluation 5
which formal analysis techniques can, in principle, be employed efficiently in

the analysis of gossip protocols, for wired as well as wireless systems. In the
remainder of this section, we will provide an overview of the existing analysis
methods, their use and limitations, as well as a comparison of related work on
the formal verification of gossip protocols.
Figure 1.1: Spectrum of validation
Figure 1.1 depicts the spectrum of analysis techniques, ranging from exper-
imental work with a real system implementation up to rigorous mathematical
analysis. Real system statistics and simulation techniques are based on experi-
ments performed on the system and on collecting data statistics either from the
running system by monitoring it at real time or from a discrete-event simulation
of the system. Usually, these approaches are used to study the behavior of a par-
ticular implementation (instance) of the system. The other approaches require a
formal modelling of the system. Typically, they are used to verify specific proper-
ties in a more general context. Although these methods often require particular
assumptions to be made, their advantage is that they can be used before a system
is being implemented, and that in principle they are not costly (in comparison to
full-scale experiments on a real system).
Of course, there are pros and cons of the use of experimental and model-
based formal analysis techniques for gossip protocols. We will discuss them later
in the chapter, outlining the challenges.
1.3 Experimental Evaluation

In practice, properties of gossip protocols are often diagnosed by emulation,
and by performing simulations (see, for example, [94]). Monte Carlo simulation
is a widely used statistical sampling method. An implementation of a gossip
protocol is executed over and over, each time using a different set of random
values, drawn from some probability distribution, to make random choices in
the protocol. At the end, a Monte Carlo simulation produces distributions of
possible outcome values. Commonly used discrete-event simulators in the area of
communication systems are ns-2 [53, 143], OPNET [153] and GloMoSim [182,
183]; but often a customized simulator is built in for example Java or MATLAB.
Experimentation is the major source of gaining first insight. The reason is
that in reality performance of gossip protocols depends on many factors: charac-
teristics of the network, certain distributions (capacity, node degree, etc), usage
scenarios, user models, incentives, etc.
However, different simulators can produce vastly different results, even for
simple systems (see, for example, [56]). The reason is that simulators employ
different models for the medium access control and physical layers. Results of
simulators often say as much about the simulated system as about the particular
lower-level implementation of the simulator. Also the employed random number
generators have an (unpredictable) impact; for instance, [5, 172] question the
credibility of this type of simulations. Moreover, different simulation analyses of
gossip protocols make different assumptions about the underlying model, which
makes comparison of results difficult. For example, [56] and [58] both evaluate
the flooding protocol, but sending and receiving is perfectly synchronized in [56],
while [58] assumes a random waiting period between sending and receiving.
Few attempts have been made to implement systems that use one of the ex-
isting gossip protocols, that is, Astrolabe [173], Tribler [158], ARRG [79] and
HEAP [89]. Notably, in [79], it is also shown that a gossip protocol that behaves
well in a simulated environment, may not behave well when emulated or truly
implemented, especially in an environment where nodes can crash. The reason
is again that the simulations are based on assumption that do not always hold in
realistic environment.
1.4 Model-Based Analysis Techniques

The formal specification of a system helps to obtain not only a better (more
modular) description, but also a clear understanding and an abstract view of the
system. Formal analysis techniques, often referred to as formal verification, are
supported by (semi-)automated tools. They can detect errors in the design that
are not so easily found using emulation or testing, and can be used to establish
the correctness of a design. The most effective way to apply formal methods is
actually during the design of a system, rather than after-the-fact, as is, unfortu-
nately, often the case.
Formal models need to be realistic. An experimental evaluation can help to
obtain a first insight in the behavior of a system and to identify which character-
istics need to be included into the model.
There are two main approaches to formal verification. The first approach
involves a rigorous mathematical analysis of the properties of the system, us-
ing results from, for instance, calculus and probability theory. Such an analysis
can be supported semi-automatically by means of MATLAB or a theorem prover.
MATLAB is an easy to use mathematical tool, but introduces imprecision due to
1.4. Model-Based Analysis Techniques 7
numerical approximations. On the other hand, theorem prover requires a lot of

effort from the user but supports general mathematical reasoning about the sys-
tem. Important theorem proving tools are Isabelle/HOL [151], PVS [155] and
Coq [42].
The second approach is model checking, which consists of a systematic and
fully automatic exploration of the state space of the system specification. The
explorative nature of the approach in principle requires that the state space is
finite. However, recent work also addresses symbolic model checking techniques
for infinite-state models [162].
1.4.1 Rigorous Mathematical Analysis

Rigorous analysis techniques for gossip protocols are built on sound mathe-
matical foundations and draw inspiration from the mathematical theory of epi-
demics [17, 83]. These analysis techniques are in general used to verify specific
properties of a gossip protocol. Therefore, such a study is usually done on a sim-
plified system model of the actual protocol: one has to decide which characteris-
tics of the protocol should be studied (see Section 1.1), and which parameters of
the protocol should be modelled in order to study these characteristics.
Gossip protocols are intrinsically probabilistic. For instance, a node may ran-
domly select a “gossip” partner or a data item for the exchange with another
node. Or it may be the case that when a node receives a message, then with some
probability p it forwards the message to all (or some) of its neighbors, while with
probability 1 − p the message is purged. A key property is that if the probability
p is sufficiently large, and the network sufficiently dense, then the probability
of successful information spread remains close to 1, while the number of sent
messages is relatively small compared to flooding.
Thus, the mathematical foundations underlying the modelling and verifica-
tion of gossip protocols can be found in probability theory, combinatorics, mathe-
matical physics, optimization theory, spectral theory, statistical mechanics, graph
theory and so on.
Mathematical analysis can be combined with simulations to validate the re-
sults and understand the system behavior. A strong point of mathematical anal-
ysis is that often it scales well with respect to the size of a network. However, it
requires a lot of effort, it can only be used to analyze a limited class of properties,
and the assumptions that are invariably made to simplify the analysis often affect
the accuracy of the results [49]. For example, the analyses in [125, 130, 131] rely
on the assumption of full knowledge of group membership, ignoring its practical
infeasibility.
Hand-crafted Markov chains

Markov chains can be used for modelling a variety of aspects of gossip proto-
cols. Markov chains allow to capture the evolution of gossip-based systems; each
state of the Markov chain describes one state of the system. However, a state of
the Markov chain does not represent a state of the whole system, but only a state
of the system with the subset of parameters that are modelled. From one state
of the Markov chain to another, there are probabilistic transitions correspond-
ing to the probabilistic evolution of the system. There are two types of Markov
chains, distinguished by the transitions occurring at any time (continuous-time
Markov chain) or only for defined steps (discrete-time Markov chain). By ana-
lyzing the different possible sequences (and their probabilities), it is possible to
obtain a global insight in the operation of the protocol. The Markov chain de-
scribing the system evolution converges to a useful distribution over all possible
system states, from which interesting protocol properties can be concluded. The
interested reader is referred to [105, 161].
The examples of the use of Markov chains for gossip and related protocols in-
clude studies on gossip-based membership management [3, 34, 46, 100], gossip-
based distributed aggregation [50, 51, 78, 129, 149], gossip-based information
dissemination [65, 72, 82] and network topology change [91]. Other methods
of probability theory has further been applied to analyze gossip-based informa-
tion dissemination [44, 60, 61, 92, 125, 141] and gossip-based resource location
[130, 131].
Deterministic Approximation
For large-scale networks, analysis can be performed by measuring a set of
nodes in a possible state over time instead of modelling every node individually.
Then, the evolution of such a system can be approximated by a deterministic
equation.
Epidemical models fit into this framework, for example, see [6, 17, 164, 166,
179]. In a deterministic model of epidemics, individuals in the population are
split into different groups, each of which represents a specific state of the epi-
demic. The transition rates from one group to another are expressed as deriva-
tives, and the model is described as a system of differential equations.
1.4.2 Model Checking

Model checking is an exhaustive state space exploration technique that is
used to validate formally specified system requirements with respect to a formal
system description [64]. Such a system is verified for a fixed configuration; so in
most cases, no general system correctness can be obtained.
Using some high-level formal modelling language, an underlying state space
can be automatically derived, be it implicitly or symbolically. The system re-
quirements are specified using some logical language, like LTL, CTL or exten-
sions thereof [116]. Well-known and widely applied model-checking tools are
SPIN [114], CADP [86], Uppaal [38] (for timed systems), and PRISM [113] (for
1.4. Model-Based Analysis Techniques 9
probabilistic systems). The system specification language can, for example, be

based on process algebra, automata or Petri nets.
Model checking suffers from the so-called state space explosion problem,
meaning that the state space of a specified system grows exponentially with
respect to its number of components. The main challenge for model checking
lies in modelling large-scale dynamic systems. To overcome the state explosion
problem and to speed up the verification process, various state space reduction
techniques have been proposed. For instance, combinations of symbolic verifi-
cation techniques with explicit state space exploration (symbolic model check-
ing), verification of properties on a smaller abstract model of the system under
scrutiny, possibly obtained after bisimulation reduction, parallelization of verifi-
cation algorithms, partial exploration of the state space (bounded model check-
ing, on-the-fly model checking, partial order reduction), and efficient state repre-
sentation (BCG and SVC formats), have been proposed to make model checking
practically feasible.
Initial model checking approaches used as underlying mathematical model a
finite-state automaton, that is, a model with neither explicit time nor probabili-
ties. Recently, model-checking techniques have been proposed for system models
including both time and probabilities, possibly in combination with nondeter-
minism. In view of the probabilistic features in gossip protocols, we focus on
model checking of probabilistic models.
Probabilistic and Stochastic Model Checking

In probabilistic and stochastic model checking, the underlying system model
is not represented by an automaton, but instead as a stochastic process of some
sort, mostly a finite-state Markov chain (discrete- or continuous-time). Often,
these Markov chains are extended with state labels and transition labels, so-
called action names. These Markov chains are mostly specified using some high-
level formalism, like stochastic process algebra [63] or stochastic Petri nets [13,
35].
Gossip protocols may require models that, in addition to pure probabilistic
choices, also allow for nondeterministic choices. That is, it is possible in a given
state of a system to non deterministically move to another state with some prob-
ability. Here, Markov decision processes can be applied [160]. The key idea to a
Markov decision process is to allow a set of probability distributions in each state
instead of a single distribution as in Markov chains. The choice between these
distributions is made externally and non deterministically, either by a scheduler
that decides which sequential subprocess takes the next step (as in, for example,
concurrent Markov chains), or by an adversary that influences or affects the sys-
tem. Probabilistic choices are internal to the process and made according to the
selected distribution.
The system requirements of interest are again specified through logical ex-
pressions over the paths that can be taken through the model. For that pur-
pose, the logics are extended to include a notion of time and probability. Promi-
nent examples of such logics are CSL [15, 16], CSRL [104] and asCSL [14] for
continuous-time models, and pCTL [102] for discrete-time models.
Where traditional model checking algorithms lean heavily on determining
reachability of certain states or state groups (or the nonreachability), in proba-
bilistic and stochastic model checking also the time until some states are reached
(or avoided) plays a major role. Furthermore, reachability of states is expressed
in terms of a probability (no mutually exclusive yes or no) and a timebound.
As an example, certain states might be highly probably reached for short time
periods, but not for longer time periods. Stochastic model checking relies on
algorithms for reachability analysis, as well as on numerical algorithms for de-
termining long-term and transient behavior in Markov chains. Although such
algorithms are well understood, their implementation requires care, especially if
very large models are to be addressed.
Stochastic and probabilistic model checking has been applied in a wide va-
riety of case studies, ranging from workstation cluster availability [106] to the
evaluation of power-saving methods [152] and the analysis of wireless (sensor)
networks [144].
Gossip protocols with their probabilistic and asynchronous behavior fit well
to the model classes supported by the known model checking techniques. How-
ever, the key to successfully verifying gossip-based systems lies in coping with
their scale. Some of the optimization techniques to deal with large networks for
general modelling techniques have been adapted to probabilistic model check-
ing, in particular: abstraction [55, 128, 137, 139], distribution [39, 40, 98] and
Markovian bisimulation [127].
Approximate and Statistical Probabilistic Model Checking

An alternative approach to cope with state explosion for probabilistic systems
is found in approximate probabilistic model checking. The main idea of this
approach is to apply Monte Carlo sampling techniques [87, 101]; the resulting
probabilities are accurate only with respect to some accuracy criterion.
Approximate probabilistic model checking [98, 109] is an approximation
method for the logic restricted to time-bounded safety properties (“positive”
LTL). Monte Carlo model checking [97] is based on a randomized algorithm for
probabilistic model checking of safety properties for general LTL model checking;
Monte Carlo model checking uses the optimal approximation algorithm of [68].
In so-called statistical probabilistic methods (for example, [180]), statisti-
cal hypothesis testing is used instead of randomized approximation schemes.
The approach of [181] describes a model-independent procedure for verifying
properties of discrete-event systems based on Monte Carlo simulation and sta-
tistical hypothesis testing. The procedure uses a refinement technique to build
statistical tests for the satisfaction probability of CSL formulas. The statistical
method of [165] concentrates on model checking of black-box probabilistic sys-
1.5. Challenges 11
tems against specifications given in a sub-logic of CSL.

Similar to the idea of approximation-based probabilistic model checking, [84]
combines probabilistic model checking with Monte Carlo simulations for the per-
formance analysis of probabilistic broadcast protocols in a wireless network. In
particular, this study shows results for reliability and reachability properties un-
der different assumptions, such as message collision, lossy channels and unreli-
able timing, and their impact on the results.
A case study in [73] presents the modelling of a sensor network using ap-
proximate probabilistic model checking. Another case study [55] presents the
results of an analysis of the MAC protocol for sensor networks using approximate
probabilistic model checking. eXtended Reactive Modules (XRM) [74] have been
proposed for modelling wireless sensor networks to generate RM models suitable
for PRISM and approximate probabilistic model checking.
Approximate probabilistic model checking could serve as a good compromise
between probabilistic model checking and simulation. They do not provide an ex-
haustive search to verify a given property, and as a result they do not suffer from
the state explosion problem that much. Still in practice they can provide rather
accurate probabilistic estimates. But approximate probabilistic model checking
is still coming off age, and needs to be further developed.
1.5 Challenges
The formal analysis of gossip protocols is a rather unexplored research field,
with many challenges and open problems. A more insightful and systematic
methodology should be developed that targets gossip protocols. The assumptions
made for simplifying such an analysis should be restricted as much as possible,
as otherwise the analysis itself becomes unrealistic.
Markov chains give a precise mathematical description, but the analysis is
time-consuming and can be used only for a restricted class of properties. It
would be worthwhile to use theorem proving tools in order to support such a
mathematical analysis, like in [108].
Probabilistic model checking techniques are convenient to use, as they are
based on verification algorithms. But formally modelling a gossip protocol still
requires considerable effort, and can introduce mistakes by itself (if the model
deviates from the actual protocol). Also the verification algorithms are very much
under development, and probabilistic model checking is, even more than stan-
dard model checking, suffering from the state explosion problem. This compli-
cates the analysis of gossip protocols considerably, as they are supposed to be
applied in large-scale networks. The use of optimization techniques, like ab-
straction and distributed verification, will form important ingredients for model
checking to become practically of interest for the evaluation of gossip protocols.
1.6 Objectives
We outlined and published the challenges from Section 1.5 in [18], following
the Lorentz Center workshop in December 2006 devoted to gossip-based com-
puter networking. All these challenges and problems related to them are covered
in this thesis. In particular, we address the problem of analyzing gossip protocols
in peer-to-peer and ad hoc networks. Moreover, we aim at developing analytical
frameworks for these protocols. We now formulate the objectives (in a cursive
typeface) and summarize how they are addressed in the next chapters.
Is there a level of abstraction for modelling a gossip protocol to allow for op-
timization of the protocol parameters? Can the performance of the protocol be
explained through the model? Can the model clarify the relationship between
the protocol parameters? To address all these questions, we develop the formal
model of a gossip protocol based on the pairwise interaction between two gossip-
ing nodes. Although this model fulfills all the above criteria, it accurately predicts
the behavior of the protocol only for a certain set of network configurations, due
to the assumptions made for simplifying the modelling task.
Thus, the next question we address in this thesis is the following: is it possible
to revise our model relaxing the assumptions made previously? We start with our
analytical model and extend it with a component based on statistical analysis.
The resulting model scales well compared to the models produced by traditional
probabilistic verification techniques, and accurately predicts the performance of
the protocol in very large networks. Moreover, we are able to observe the impact
of different network configurations on the protocol without relying on a pure
experimental approach.
Does it make sense to use traditional probabilistic verification techniques for such
a large-scale application? To increase the network size that can be modelled in
commonly used probabilistic model checker, we use our abstract analytical model
as the basis for the formal protocol specification. Gossip protocols exhibit non-
deterministic behavior. This brings us to the question of whether the resolution
of the nondeterminism can significantly affect a chosen property of the protocol
under study. In general, we are interested in exploring the impact of modelling
choices that are underspecified in a protocol. To that end, we build and com-
pare models produced by different modelling frameworks, including traditional
epidemics-like equational models and probabilistic model checking.
In order to fight the state space explosion problem, analysis of a discrete system,
such as a system executing a gossip protocol, can make use of a continuous
approximation. Notably, the mean-field method approximates a distribution of
nodes in the set of possible states over discrete time for the case that the number
of interacting nodes tends to infinity. We demonstrate that the mean-field method
is suitable for gossip protocols by developing a modelling framework.
The mean-field method is hardly used for communication networks because a
mean-field model tends to abstract away the underlying topology, and computing
of the transition probabilities matrix is a time-consuming, error-prone procedure.
1.7. Organization of the Thesis 13
Therefore, we extend our mean-field framework to allow for the performance eval-
uation of dynamic gossip networks. We identify a class of stochastic systems of
nodes for which the construction of the mean-field model can be automated, and
develop a tool that performs automatic computation depending on the user-input
specification.
All analytical results presented in this thesis are supported by large-scale
Monte Carlo simulations using PeerSim [122], and self-implemented Java-based
and Python-based simulators.
1.7 Organization of the Thesis

The thesis is organized in eight chapters, and Figure 1.2 depicts relations
between them.
Ch. 6 & 7
N →∞
Ch. 5 Ch. 3
P (00|00) P (01|01) Ch. 5
dx (0, 0) (0, 1)
dt
)
10
0.3
P (11|01)
1|
P (01|11)
simulation (average)
(0
model
)
0.25
01
P
replicas (% of nodes)
0|
0.2
(1
P
0.15
0.1
P (11|10)
0.05
(1, 0) (1, 1)
0
0 200 400 600 800 1000 P (10|11)
rounds P (10|10) P (01|01)
Ch. 4
P (00|00) P (01|01)
P (00|01)
(0, 0) (0, 1)
)
10
P (11|01)
1|
P (01|11)
(0
⇒
)
A B
01
P
0|
(1
P
P (11|10)
(1, 0) (1, 1)
P (10|11)
P (10|10) P (01|01)
Figure 1.2: Pictorial representation of the thesis structure
Chapter 2: Background on Gossip Protocols. The next chapter introduces a

family of gossip protocols, analyzed in this thesis. It first surveys the application
domain of gossip protocols, and compares them, giving examples of common
measures of interest for various applications of gossip. This survey is followed
by the description of the shuffle protocol, a gossip push-pull protocol for shar-
ing data in a distributed network. Two properties of this protocol are specified:
how fast an observed datum is replicated through a network, and how fast the
observed datum covers the network.
Chapter 3: Pairwise Exchange Model without Failures. This chapter ana-

lyzes the shuffle protocol. It introduces a novel approach of analyzing gossip
protocols, modelling the protocol behavior at an abstract level as pairwise node
interactions. The analysis relies on the assumption of no communication fail-
ures. With this model, we compute the optimal value of the protocol parameters
to obtain fast replication.
Chapter 4: Pairwise Exchange Model in Lossy Networks. This chapter ex-

plores the impact of lossy communication channels on the shuffle protocol. It
challenges the simplifying assumptions made for the formal model from the pre-
vious chapter, namely uniform distribution of data in the network. We present
an extended model, one component of which is based on statistical analysis.
Chapter 5: Comparison of Performance Models. This chapter presents and

compares different models of the shuffle protocol, produced in different tradi-
tional modelling frameworks. Using the pairwise interaction model, we build
several formal models with the probabilistic model checker PRISM, and directly
with differential equations. The chapter explores the impact of modelling choices
made for the parts of the protocol that are underspecified.
Chapter 6: Mean-field Framework. This chapter demonstrates that the exist-

ing mean-field method for very large networks (N → ∞) is suitable for gossip
protocols. The chapter introduces a mean-field framework for the evaluation of
gossip protocols, accompanied by two case studies, the shuffle protocol and the
gossip clock synchronization protocol GTP. The mean-field results for the shuf-
fle protocol are compared with the differential equation model developed in the
previous chapter.
Chapter 7: Automating the Mean-Field for Dynamic Networks. This chap-

ter focuses on the automation of mean-field analysis, and demonstrates how a
network with pairwise communication between nodes can be modelled and ana-
lyzed. This chapter extends the results of the previous chapter with the possibility
to encode physical node positions and to model dynamic networks. It determines
a class of stochastic systems of nodes for which the construction of the mean-field
model can be automated.
Finally, Chapter 8: Concluding Remarks summarizes the main results of this

thesis, and discusses future work.
A general introduction into the field is provided by Chapter 1. Background
information, including a description of the shuffle protocol used as the main case
1.8. Origins of the Chapters 15
study in this thesis, is presented in Chapter 2. Together with Chapters 1 and 2,

every other chapter can be read separately, and includes its own related work,
conclusions and future work. The research questions, formulated earlier, are
addressed in Chapters 3–7, respectively.
1.8 Origins of the Chapters

Chapter 1 is based on a joint invited publication with François Bonnet, Wan
Fokkink, and Boudewijn Haverkort. It appeared in [18] in the aftermath of the
Lorentz Center “Gossip-based Computer Networking” Workshop, held in Leiden
in December 2006.
In Chapter 2, a survey of gossip protocols and the shuffle protocol description
are composed of parts of [21, 25]. Section 2.2 of the same chapter is based
on [94].
Chapter 3 is mostly based on the journal paper [28], which is an extended
version of the conference paper [29]. Section 3.5 appeared as [26] in the context
of a card dealing procedure.
Chapter 4 is currently submitted for publication.
Chapter 5 is a composition of two papers, the journal paper [28] with Daniela
Gavidia, Wan Fokkink and Maarten van Steen, and the conference paper [25]
with Ansgar Fehnker.
Chapter 6 contains the material of joint work with Lucia Cloth, Wan Fokkink,
and Boudewijn Haverkort, published first as a preliminary paper [20] in the post-
proceedings of the Lorentz Center “Two Decades of Probabilistic Verification –
Reflections and Perspectives” Workshop in Leiden in November 2007. It later ap-
peared as an extended conference version [19], and, finally, as an invited journal
version [21].
Chapter 7 appeared as a conference paper [23].
Since most of the material presented in the thesis is the results of collabo-
rations, as described above, I use “we” instead of “I” throughout the thesis to
acknowledge this fact. The publications that are not a part of this thesis include
[24, 27, 30].
Background on Gossip
Protocols 2
In this chapter, we give background on gossip protocols at an algorithmic
level. We start with a short introduction on gossip applications, summarizing
measures of interest for each application. We also present a background on the
shuffle protocol, originally introduced in [94] for the dissemination of informa-
tion in a wireless environment. This protocol is used for the case studies in the
next chapters. Gossip protocols are often used for information dissemination. We
choose the shuffle protocol as a prime example of this application.
2.1 A Generic Gossip Protocol

In gossip protocols, the information exchange between nodes can be imple-
mented as one of the following policies: only the node that initiates a gossip
sends local state data to its partner (push), a node-initiator requests state data
from its gossip partner (pull), or both nodes send their state information to each
other (push-pull). In this thesis, we mainly consider push-pull gossip protocols.
Figure 2.1 illustrates the skeleton
of a generic push-pull gossip protocol. while true do while true do
Each node has a local state s and ex- wait (∆t time units)
ecutes two different threads, an active p ← RandomPeer();
and a passive one. The active thread prepare(s);
send s to p; sp ← receive(·);
periodically initiates a state exchange prepare(s);
with a random peer p by sending it a sp ← receive(·); send s to sender(sp);
message containing the local state s, s ← Update(s, sp ); s ← Update(s, sp );
after which it waits for a response. The
passive thread waits for a message sent (a) active thread (b) passive thread
by an initiator and replies to it with its Figure 2.1: A gossip protocol
local state. The random peer selection
17
18 Chapter 2 — Background
is based on the set of neighbours as determined by a membership protocol (e.g.,

[123]).
For a pair of nodes A and B, where A is the active node and B is the passive
one, we describe the protocol from the point of view of each participating node.
In particular, node A picks a neighbor B at random (method RandomPeer())
after a not necessarily constant (discrete) time span of length ∆t, and initiates a
state exchange (gossip) with it. It does so by sending (part of) its local state s
to B, and waits for B’s response. Upon receipt of the response, node A updates
its local state (according to the method Update(s, sp )). In response to being
contacted by A, node B sends (part of) its local state to A and updates its local
state accordingly (method Update(s, sp )).
Applications of Gossip
The method Update is protocol specific. It updates the local state of a node
based on the previous local state, and the state information received from the
random gossip partner. In gossip-based information dissemination protocols (as
in, e.g., distributed news service protocols [94, 120]), a finite list of data items
(e.g., news items), called the cache, composes the local state of a node. The
generic operation prepare(s) in Figure 2.1 is replaced by an operation s ←
RandomItems(). The method Update merges the list of old items with the list
of received items. The measures of interest of these protocols include the num-
ber of copies of a data item in the network after some time, and the amount of
time needed for the data to spread in the network.
In gossip-based membership management protocols, a finite set of peer ad-
dresses, called the partial view, comprises the local state of a node. The method
Update (as in [3, 177]) creates a new state through a sample of the union of the
old and the received views. The performance metrics of these protocols include
a distribution of the partial view size, and the number of nodes reached in the
presence of node failures.
In probabilistic broadcasting (e.g., [184]), the state of a node is a flag that
records whether the node is infected. The method Update sets the state to in-
fected if the received state is infected. The performance of these protocols can
be measured by the time until all nodes have been infected.
In gossip-based distributed aggregation (e.g., [121]), the state of a node is
a numeric value, which can be any parameter of the environment, such as a
temperature or the current load. All values at nodes contribute to an aggregate
value, computed using some aggregation function, for instance, average, sum,
etc. The method Update simply returns the result of the aggregation function.
For these protocols, a general measure of interest is the convergence of results of
the aggregation function, but other measures depend on the aggregation function
chosen.
We refer to [132] for a survey on gossip applications.
2.2. A Gossip Protocol for Information Dissemination 19
2.2 A Gossip Protocol for Information Dissemination

We use the shuffle protocol, introduced in [94], as the main case study in
the thesis. The shuffle protocol is, at heart, a simple push-pull gossip protocol
which can be used also in wired networks. The protocol disseminates data items
of general interest throughout a network of small devices.
The network consists of a collection of nodes, each of which contributes a lim-
ited amount of storage space (which we will refer to as the node’s cache) to store
data items. Nodes executing the shuffle protocol initiate a shuffle periodically. In
order to execute the protocol, the initiating node needs to contact a gossip part-
ner and shuffle data items from its cache with the partner. Such a random partner
is delivered by an underlying layer that keeps track of the neighborhood mem-
bership. In a wired environment, this service could be provided by, for instance,
a peer sampling service [123] running at each node. For wireless environments,
the neighborhood is determined by the radio connectivity between nodes.
Items can be published by any user of the system and are propagated through
the network. An item is a piece of information, and for each item several copies
may exist in the network. As items are gossiped between neighboring nodes,
replication may occur when a node has available storage space to keep a copy of
an item it just gossiped to a neighbor.
The behavior of the protocol is probabilistic in nature. A node chooses a
random partner to communicate with, and it exchanges a random subset of data
items from its cache. Additionally, the shuffle protocol exhibits nondeterministic
behavior in that the order in which the nodes initiate a gossip is not determined.
2.2.1 Protocol Assumptions

All nodes have a common agreement on the frequency of gossiping. However,
there is no agreement on when to gossip. In terms of storage space, we assume
that all nodes dedicate the same amount of storage space to keep items locally,
and that all items are of the same size. Therefore, we say that each node has a
cache of fixed size c. When shuffling, each node sends a fixed number s of the c
items in the cache.
2.2.2 Protocol Description

Every node maintains a finite list of data items in a cache. The shuffle protocol
is executed at nodes initiating a contact and their contact partners. Both nodes
exchange a random subset or the whole set of items from their local caches in
one contact. Each node agrees to keep the items received from its gossip partner.
Taking into account the size of a cache, a node might have to discard some items
from the cache in favor of the items received during an exchange. The items to
be discarded are picked from the ones that have been sent to the peer. Since the
20 Chapter 2 — Background
while true do while true do

wait (∆t time units)
B := randomPeer()
sA := itemToSend(cA );
send sA to B; sA := receive(·);
sB := itemToSend(cB );
sB := receive(·); send sB to sender(sA );
cA := itemKeep(cA \(sA \sB ), sB \cA ); cB := itemKeep(cB \(sB \sA , sA \cB );
(a) initiating node (b) contacted node

Figure 2.2: The shuffle protocol algorithm
peer, in its turn, has agreed to store these items in its cache, discarding items
does not lead to the loss of information in the network.
The shuffle algorithm is illustrated in Figure 2.2. The caches of nodes A and
B are denoted by cA and cB , respectively. An exchange message (or buffer) sent
from A to B is denoted as sA , and a message from B to A is denoted as sB . The
size of the caches cA and cB is c, the size of the exchanged messages sA and sB
is s, and the total number of different items in the network is n.
A node A periodically initiates a shuffle with a random neighbor B. It sends B
a message sA , containing a copy of s random items from its cache cA , and waits
for a response from B. Upon receipt of sA from A, node B sends message sB
with s random items from its cache cB to A. Node A and node B then eliminate
all received items that are already in their caches from sB and sA , respectively.
A adds the remaining received items (sB \cA ) to the cache. It adds them by
replacing items in its cache cA that were sent to, but not received from B. This
is the set sA \sB , and its elements may be replaced randomly, while elements in
cA \(sA \sB ) will be kept. This ensures that not only the total number of items
will not exceed c, but also that the items sent from B to A do not get lost. In a
similar fashion node B adds the items in sA \cB to its cache, by replacing items
in sB that were not in the received message sA . Figure 2.2 depicts pseudo-code
for the algorithm in the style of the skeleton in Figure 2.1. Note that the code
for the node that initiates the contact (i.e. Figure 2.2(a)) differs from the code
of the contacted node (i.e. Figure 2.2(b)).
We refer to [94] for a more detailed description. Figure 2.3 gives an example
of two nodes A and B executing the shuffle protocol and gossiping with each
other with protocol parameters n = 8, c = 5 and s = 3.
2.2.3 Properties
We study the performance of the protocol focusing on two properties that can
be observed in large deployments: i) the number of replicas of an item in the
network, and ii) the coverage achieved by an item over time.
2.2. A Gossip Protocol for Information Dissemination 21
1 2
A B A B
a b a b
b b b c b b b c
c c d d c d c d
e g h f e h g f
g h g h
cA sA sB cB cA sB sA cB
3 4
A B A B
a b a b
b b b c b b b c
c d c d d d c d
e h g f e h g f
g h h g
cA\(sA\sB) sB\cA sA\cB cB\(sB\sA) c’A sB\cA sA\cB c’B
Figure 2.3: Illustration of the shuffle algorithm with n = 8, c = 5 and s = 3.

(1) Selecting random data. (2) Exchanging data. (3) Determining which data to
keep. (4) Merging cache with received data. Black items are the ones that nodes
decide to discard.
Replication This property is defined as the number of nodes that hold a copy
of an observed item d in their local cache at a given time. With every shuffle,
a copy of the item can be created (or discarded), if it is in the cache of the
gossiping node. Owing to the limited storage space c at each node and random
choice of the exchanged items, the number of copies of all data items constantly
fluctuates in the network, with all items having the same chance to be replicated
or discarded. Thus, in a network with n different items, the average fraction of
nodes that hold a replica of an item converges to nc .
Coverage This property is defined as the fraction of nodes that have held an
observed item d since it was introduced into the network until a given time. Due
to the periodic nature of the protocol, copies of an item continually move through
the network. Over time, the fraction of nodes discovering item d grows, eventu-
ally converging to 1. The speed of convergence is influenced by the parameters
of the protocol: n, c and s.
Pairwise Exchange Model
without Failures 3
As pointed out in the introduction, gossip protocols usually contain design
parameters that influence the nonfunctional properties of these protocols such
as performance. The relationship between the values of these parameters and
protocol performance can be inferred through extensive large-scale Monte Carlo
simulations for a wide range of system configurations.
In this chapter, we develop an analytical model of information dissemination
for the shuffle protocol, assuming no communication failures. We define two
properties of the protocol to validate our model: (i) how fast an item is repli-
cated through a network, and (ii) how fast the item covers the network. With
this model we determine the optimal size of the exchange buffer to obtain fast
replication. Our results are confirmed by large-scale simulation experiments.
3.1 Introduction
We develop an analytical model of the shuffle protocol, described in the pre-
vious chapter. Recall that nodes executing the protocol periodically contact each
other, and exchange data items. Concisely, a node initiates a contact with a
random neighbor, pulls a random subset of items from the contacted node, si-
multaneously pushing its own random subset of items.
The main focus of this study is a thorough probabilistic analysis of information
dissemination in a large-scale network using the shuffle protocol. Our modelling
framework for a gossip protocol differs from the ones presented by others in that
it does not follow the traditional modelling using the mathematical theory of
epidemics [83]. Instead, the behavior of the protocol is modelled at an abstract
level as pairwise node interactions. When two neighboring nodes interact with
each other (gossip), they may undergo a state transition (exchange items) with
a certain probability. The transition probabilities depend on the probability that
23
24 Chapter 3 — Pairwise Exchange Model without Failures
a given item in a node’s local storage has been replaced by another item after the
exchange.
The analytical expressions for these probabilities are derived. One of the
expressions depends not only on the number of items, message size, and local
storage size, but also on the number of items both gossiping nodes have in com-
mon, in particular, how many of such items the contacted node receives during
the exchange. The expression is complex because it incorporates the expected
value of the number of such items, and a connection between the parameters is
not obvious. We also determined a close approximation that is expressed by a
much simpler formula, as well as a correction factor for this approximation al-
lowing for precise error estimations. Thus we obtain a better understanding of
the emergent behavior of the protocol and how parameter settings influence its
nonfunctional behavior.
We investigated two properties characterizing the protocol, namely, the num-
ber of nodes that have ‘seen’ a given item over time (coverage), and the number
of replicas of this item in the network at a certain moment in time (replica-
tion). Using the values of the transition probabilities, we determined the optimal
number of items to exchange per gossip, for a fast convergence of coverage and
replication. All our modelling and analysis results are confirmed by large-scale
simulations, in which simulations based on our analytical models are compared
with running the actual protocol. We believe that the developed model is an ac-
curate, realistic formal model that can be used to optimally design and fine-tune
a given gossip protocol. In this sense, our main contribution is demonstrating the
feasibility of a model-driven approach to developing real-world gossip protocols.
3.2 Assumptions
As mentioned earlier, we use the shuffle protocol (described in Chapter 2) for
our case study. For our analysis in this chapter, we assume that:
• nodes do not join or leave the network, and do not fail;
• nodes communicate over perfect, error-free channels.
The gossip exchange can then be seen as an atomic procedure. Thus, on the
implementation level, once a node initiates an exchange with another node, this
pair of nodes cannot become involved in another exchange until the current ex-
change is finished.
3.3 Experimental Observations

This section focuses on an important aspect that we have observed during our
extensive simulation study of the shuffle protocol: the tendency of items to repli-
cate to even levels. In other words, a newly published item generates replicas
in the network over time until its number of replicas reaches a level comparable
to the number of replicas of other items. This comes as a consequence of the
3.3. Experimental Observations 25
random selection of items to gossip and to keep in local storage. By applying

random selection, no item is favored over the others resulting in a fair division of
the storage space. That is, once the system has reached equilibrium, each item
in the network will have, on average, the same number of replicas ( Nn·c ).
Figure 3.1 shows a set of experiments where the distribution of replicas for
all the items in the system is tracked over several gossip rounds. In all cases,
the network consists of N = 2500 nodes with a cache size of c = 100 and each
node sends s = 50 items when it gossips. The number of different items that are
inserted in the network is n = 500. Each graph shows three curves corresponding
to the following initial scenarios:
1. Nodes are arranged in a 50 × 50 grid. The insertion of items to be gossiped
occurs simultaneously at round 0. All nodes start with an empty cache,
except for 500 randomly chosen nodes. A unique item is placed in the cache
of each of these 500 nodes. As a result, at round 0 the network contains 500
different items and each of these items has a single replica.
2. Nodes are arranged in a 50 × 50 grid. The insertion of items to be gossiped
occurs at randomly chosen times before round 100. At round 100, all 500
items will be present in the network. However, items that were inserted
earlier will have more replicas due to having been shuffled for a longer time.
3. Topology with higher density at the center. The insertion of items to be gos-
siped occurs at randomly chosen times before round 100. In this topology,
nodes near the center have more neighbors to gossip with.
As can be seen from the graphs, the different initial conditions result in dif-
ferent replication patterns early on. However, note how as time progresses the
replication patterns become more and more similar. By round 350, it is clear that
items have on average Nn·c = 500 replicas, regardless of the initial conditions
of the experiment. This convergence to an equilibrium where the storage space
(N · c) is evenly divided between the number of items present (n) is a result of
the repeated execution of the protocol.
In the shuffle protocol, there is no loss of information during the gossip ex-
change. We assume that the shuffle operation between two nodes is atomic and
since nodes swap items, no item can possibly disappear once inserted into the
network. For this reason, once the system has reached equilibrium in terms of
replication of items and assuming that there is a path between any two nodes in
the network, the replicas will continue to be shuffled, moving from one node to
another in a random fashion. Under these conditions, it is reasonable to assume
that the probability of finding any given item in a node’s cache is the same for all
items in the networks. To be more specific, based on our observations we make
the assumption that items are uniformly distributed throughout the network and
that any item can be found in a node’s cache with probability nc . This is the start-
ing point for our probabilistic analysis. The observation of uniform distribution
is not entirely unexpected. Similar observations about uniform distribution after
repeated shuffling have been made regarding the shuffling of decks of cards in
[8, 37].
round 100 round 150

16 16
1. grid, insert at round 0 1. grid, insert at r
14 2. grid, insert at random 14 2. grid, insert at r
3. dense center, insert at random 3. dense center, insert at r
12 12
number of items
10 10
8 8
6 1 6 1
4
3 4
2 2
0 0
0 200 400 600 800 1000 200 300 400 500
number of replicas number of replic
round 200 round 25

16 16
12 12
number of items
10 10
8 1 2 8
6 3 6
4 4
2 2
0 0
200 300 400 500 600 700 800 0 200 400 60
number of replicas number of rep
round 300 round 350

16 16
12 12
number of items
10 10
8 8
6 6
4 2 4
2 2
0 0
200 300 400 500 600 700 800 200 300 400 500
number of replicas number of replic
3.4. An Analytical Model of Information Dissemination 27
3.4 An Analytical Model of Information Dissemination

We analyze dissemination of a generic item d in a network in which the nodes
execute the shuffling protocol.
3.4.1 Probabilities of State Transitions

This section presents a model of the shuf-
fle protocol that captures the presence or ab- P (00|00) P (01|01)
sence of a generic item d after shuffling of two
nodes A and B. There are four possible states (0, 0) (0, 1)
of the caches of A and B before the shuffle:
)
both hold d, either A’s or B’s cache holds d,
10
P (11|01)
1|
P (01|11)
(0
or neither cache holds d.
)
01
P
We use the notation P (a2 b2 |a1 b1 ) for the
0|
(1
probability that from state a1 b1 after a shuffle
P
we get to state a2 b2 , with ai , bi ∈ {0, 1}. The P (11|10)
indices a1 , a2 and b1 , b2 indicate the presence (1, 0) (1, 1)
(if equal to 1) or the absence (if equal to 0) P (10|11)
of a generic item d in the cache of an initiator P (10|10) P (01|01)
A and the contacted node B, respectively. For Figure 3.2: Symbolic represen-
example, P (01|10) means that node A had d tation for caches of gossiping
before the shuffle, which then moved to the nodes.
cache of B, afterwards. Due to the symmetry
of information exchange between nodes A and B in the shuffle protocol,
P (a2 b2 |a1 b1 ) = P (b2 a2 |b1 a1 ).
Figure 3.2 depicts all possible outcomes for the caches of gossiping nodes as
a state transition diagram. If before the exchange A and B do not have d (a1 b1 =
00), then clearly after the exchange A and B still do not have d (a2 b2 = 00).
Otherwise, if A or B has d (a1 = 1 ∨ b1 = 1), the shuffle protocol guarantees that
after the exchange A or B still has d (a2 = 1 ∨ b2 = 1). Therefore, the state (0, 0)
has a self-transition, and no other outgoing or incoming transitions.
We now determine values for all probabilities P (a2 b2 |a1 b1 ). They are ex-
pressed in terms of probabilities Pselect and Pdrop . The probability Pselect expresses
the chance of an item to be selected by a node from its local cache when en-
gaged in an exchange. The probability Pdrop represents a probability that an item
which can be overwritten (meaning it is in the exchange buffer of its node, but
not of the other node in the shuffle) is indeed overwritten by an item received by
its node in the shuffle. Due to the symmetry of the protocol, these probabilities
are the same for both initiating and contacted nodes. In Section 3.4.2, we will
calculate Pselect and Pdrop . We write P¬select for 1 − Pselect and P¬drop for 1 − Pdrop .
State (0, 0). Before shuffling, neither node A nor node B have d in their cache.
a2 b2 = 00: neither node A nor node B have item d after a shuffle because neither
of them had it in the caches before the shuffle:
P (00|00) = 1
a2 b2 ∈ {01, 10, 11}: cannot occur, because none of the nodes have item d.
State (0, 1). Before shuffling, a copy of d is only in the cache of node B.
a2 b2 = 01: node A does not have d because node B had d but did not select it
(to send) and, thus, B did not overwrite d, i.e., the probability is
P (01|01) = P¬select
a2 b2 = 10: only node A has d because node B selected d and dropped it; that is,
the probability is
P (10|01) = Pselect · Pdrop
a2 b2 = 11: both nodes A and B have a copy of d because node B selected d and
kept it; that is,
P (11|01) = Pselect · P¬drop
a2 b2 = 00: cannot occur as completely discarding d is not possible in the pro-

tocol; that is, if either nodes send an item, its partner keeps this copy as
well, and if an item is not among the selected for a shuffle, the item is not
replaced by another one (see Section 2.2.2).
State (1, 0). Before shuffling, d is only in the cache of node A. Due to the sym-
metry of nodes A and B, this scenario is symmetric to the previous one with
P (a2 b2 |10) = P (b2 a2 |01).
State (1, 1). Before shuffling, d is in the cache of node A as well as in the cache
of node B.
a2 b2 = 01: only node B has d because node A selected d and dropped it and
node B did not select d; that is,
P (01|11) = Pselect · Pdrop · P¬select
a2 b2 = 10: this outcome is symmetric to the previous one:
P (10|11) = P¬select · Pselect · Pdrop
a2 b2 = 11: after the shuffle both nodes A and B have d, because:

∗ nodes A and B had d but both did not select it, i.e., P¬select · P¬select ;
∗ both nodes A and B selected d (thus, both kept it),i.e., Pselect · Pselect ;
∗ node A selected d and kept it and node B did not select d: Pselect · P¬drop ·
P¬select ;
∗ symmetric case with the previous one: P¬select · Pselect · P¬drop .
Thus,
P (11|11) = P¬select · P¬select + Pselect · Pselect + 2 · Pselect · P¬select · P¬drop
a2 b2 = 00: cannot occur, discarding of an item is not permitted by the protocol

(see Section 2.2.2).
3.4.2 Probabilities of Selecting and Dropping an item

The following analysis assumes that all node caches are full (that is, the net-
work is already running for a while). Moreover, we assume a uniform distribu-
tion of items over the network; this assumption is supported by experiments in
Section 3.3 as well as in [94, 123].
Consider nodes A and B engaged in a shuffle, and let B receive the exchange
buffer SA from A. Let k be the number of duplicates (see Figure 3.3), i.e., the
items of an intersection of the node cache CB and the exchange buffer of its
gossip partner SA (i.e., SA ∩CB ). Recall from Section 3.2 that CA and CB contain
the same number of items for all A and B, and likewise for SA and SB ; we use
c and s for these values. The total number of different items in the network is
denoted as n.
The probability of selecting an item d in the cache is the number of selected
items (i.e., s) divided by the total number of items in the cache (i.e., c):
s
Pselect = .
c
Thus, the probability that an item d in the cache is not selected is:
c−s
P¬select = 1 − Pselect = .
c
n
CB
SA k CB
SA
sb SB
Figure 3.3: k items in SA ∩ CB Figure 3.4: sb items in SA ∩ SB

Consider Figs. 3.3 and 3.4. The shuffle protocol demands that all items in
SA are kept in CB after the shuffle. This implies that: a) all items in SA \CB
will overwrite items in SB ⊆ CB , and b) all items in SA ∩ CB are kept in CB .
Thus, the probability that an item from SB will be overwritten is determined by
the probability that an item from SA is in CB , but not in SB . Namely, the items
in SB \SA provide a space in the cache for items from SA \CB . We would like
to express the probability Pdrop of a selected item d in SB \SA (or SA \SB ) to be
overwritten by another item in CB (or CA ). Due to symmetry, this probability is
the same for A and B; therefore, we calculate only the probability that an item
in SB \SA is dropped from CB . The expected value of this probability depends
on how many duplicates a node receives from its gossip partner a :
 s
 X |S ∩C |=k

 (PdropA B · P|SA ∩CB |=k ) if s + c 6 n


k=0
E[Pdrop ] = Xs

 |S ∩C |=k

 (PdropA B · P|SA ∩CB |=k ) otherwise

k=(s+c)−n
where P|SA ∩CB |=k is the probability of having exactly k items in SA ∩ CB , and
|S ∩C |=k
PdropA B is the probability that an item in SB \SA is dropped from CB given k
duplicates in SA ∩ CB . The case distinction is because if s + c > n, then clearly
(s + c) − n items in SA ∩ CB .
there are at least
From the ns possible sets SA , we compute how many have k items in com-
mon with CB . Firstly, there are kc ways to choose k such items in CB . Secondly,

there are n−c
s−k ways to choose the remaining s − k items outside CB . So in total,
c
n−c
k · s−k possible sets SA have k items in common with CB . Hence, under the
assumption of a uniform distribution of the data items over the caches of the
nodes, b
n−c
c s−k
P|SA ∩CB |=k = n
.
k s
|S ∩C |=k
The expected value of PdropA B isc :

 X k

 |S ∩S |=b s

 PdropA B · P|SA ∩SB |=bs if s + k 6 c

|SA ∩CB |=k sb=0
E[Pdrop ]= k

 X |S ∩S |=b s

 PdropA B · P|SA ∩SB |=bs otherwise


sb=(s+k)−c
a The other case is presented for the sake of completeness.

b Here we use a generalization of the usual definition of binomial coefficients to negative integers.

That is, for all m and l ≥ 0, m l
= (−1)l −m+l−1
l
. See also [111].
c The other case is presented for the sake of completeness.
where sb is the number of items in SA ∩ SB (see Figure 3.4). The case distinction
is because if s + k > c (with k the number of items in SA ∩ CB ), then clearly
there are at least (s + k) − c items in SA ∩ SB .
Among the s items in SB , there are sb items also in SA , and thus only the s − sb
|S ∩S |=b s
items in SB \SA can be dropped from CB . PdropA B is the probability that an
item in SB \SA is dropped from CB , given sb items in SA ∩ SB :
(
|SA ∩SB |=b
s 0 if s = sb
Pdrop = s−k
s−bs otherwise
P|SA ∩SB |=bs is the probability of having exactly sb items in SA ∩ SB :

c−s
s k−bs
P|SA ∩SB |=bs = c
.
sb k
The intuition behind this expected value of P|SA ∩SB |=bs is similar to the one of

P|SA ∩CB |=k . From the kc possible sets SA ,we compute how many have sb items
in common with SB . That is, there are ssb ways to choose sb items in SB , and
c−s

k−bs ways to choose the remaining k − s b items outside SB .
Let us assume s + c ≤ n and s + k ≤ c. Then, substituting in the expression
for E[Pdrop ] in case s + c ≤ n, and noting that in the summand k = s the factor
|S ∩S |=s
PdropA B is equal to zero, we get:
s−1 n−c
k c−s

X c s−k
X s−k s k−b
E[Pdrop ] = n
c
s
k s
s − sb sb k
k=0 sb=0
s−1 k c−s
s
n − c X (n − c) − 1 X k−b
s sb
= n
(3.1)
s
(s − k) − 1 s − sb
k=0 sb=0
The probability of keeping an item d in SB \SA ⊆ CB can be expressed as P¬drop =

1 − Pdrop .
3.4.3 Simplification of Pdrop

In order to gain a clearer insight into the emergent behavior of the gossip
protocol we make an effort to simplify the formula for the probability Pdrop of an
item in SB \SA to be dropped from CB after a shuffle. Therefore, we re-examine
the relationships between the k duplicates received from a neighbor, the sb items
|S ∩C |=k
of the overlap SA ∩ SB , and Pdrop . Let’s estimate PdropA B by considering
each item from SA separately, and calculating the probability that the item is a
duplicate (i.e., is also in CB ). The probability of an item from SA to be a duplicate
(also present in CB ) is nc . In view of the uniform distribution of items over the
network, the items in a node’s cache are a random sample from the universe of n
data items; so all items in SA have the same chance to be a duplicate. Thus, the
expected number k of items in SA ∩ CB can be estimated by s · nc . The expected
number sb of items in SA ∩ SB can be estimated by k · sc , because only the k items
in SA ∩ CB may end up in SA ∩ CB ; sc captures the probability that an item
from CB is also selected to be in SB . It follows that the probability of an item in
SB \SA to be dropped from CB after the shuffle is
s−k s − s · nc n−c
Pdrop = = = .
s − sb s − s · nc · sc n−s
The complementary probability of keeping an item is

n−c c−s
P¬drop = 1 − = .
n−s n−s
These estimates are valid for general s ≤ c ≤ n.
Substituting the expressions for Pselect and the simplified Pdrop into the formu-
las for the transition probabilities in Figure 3.2, we obtain:
c−s s c−s n−c

P (01|01) = P (10|10) = c P (01|11) = P (10|11) = c c n−s
s n−c
P (10|01) = P (01|10) = c n−s P (11|11) = 1 − 2 sc c−s n−c
c n−s
s c−s
P (11|01) = P (11|10) = c n−s
1 1
c=250 n=700 c=500 n=1400
1e-20 n=1100 n=2200
n=1500 1e-50 n=3000
1e-40
1e-60 1e-100
relative error
1e-80
1 1e-150 1
1e-100
1e-05 1e-200 1e-05
1e-120
1e-140 1e-250
1e-10 1e-10
1e-160
1e-300 1e-15
1e-180 1e-15
1 3 5 1 3 5
1e-200 0
30 60 90 120 60 120 180
exchange buffer size exchange buffer size
Figure 3.5: The relative error of the difference of the accurate Pdrop and its ap-
proximation, for c = 250 (left) and c = 500 (right) and different values of n.
In order to verify the accuracy of the proposed simplification for E[Pdrop ], we

compare the simplification and the accurate formula (3.1) for different values
of n. We plot the difference of the accurate Pdrop and the simplification, for
cache sizes c = 250 and c = 500 (Figure 3.5). These figures show that the
simplification gives a close approximation of the accurate formula for Pdrop (note
the log y-scale).
3.4.4 Correction Factor

n−c
We now investigate how closely the simplified formula of Pdrop , that is n−s
(here referred to as S(n, c, s)) approximates formula (3.1) (here referred to as
E(n, c, s)). We compared the difference between these two formulas using a
Java package based on common fractions, which provides lossless calculation
[95]. We observe that the inverse of the difference of the inverse values of both
formulas, i.e.,
−1
ec,s (n) = E(n, c, s)−1 − S(n, c, s)−1 ,
exhibits a certain pattern for different values of n, c and s.

For s = 1 and arbitrary values of n and c, E(n, c, 1) = n−c n , whereas S(n, c, 1) =
n−c n−c
n−1 . This leads us to investigate the correction factor θ as in E(n, c, s) = (n−s)+θ .
For s = 1 clearly the factor θ = 1. Yet, for s > 1 the situation is more complicated.
We therefore calculate the first, the second and other (forward) differencesd over
n.
For s = 1, the result of the first difference of the function ec,1 (n) is 1. How-
ever, for s = 2 the first difference is, e.g., if c = 4:
e4,2 (7) − e4,2 (6) = 3.5, e4,2 (8) − e4,2 (7) = 4, e4,2 (9) − e4,2 (8) = 4.5
and so on. Thus, we observe that the second difference of ec,2 (n) is 12 . By cal-
culating higher differences for s > 2, we conclude that the s-th difference of the
function ec,s (n) is always 1s .
Moreover, at the point n = 0 the first, . . . , s-th differences of the function ec,s
exhibit a pattern similar to the Pascal triangle [163]. That is, for d ≥ 1 the d-th
difference is
1
(∆d ec,s )(0) = s−1

s· d

(assuming ab = 0, whenever b > a). Knowing the initial difference at point
n = 0, we were able to use the Newton forward difference equation [1] to derive
the following formula for n > 0:
n−c
E[Pdrop ] = 1 , (3.2)
(n − s) + γ
where
s−1
X n n s−1
X
d s 1
γ = s−1
= · n−d
(3.3)
s· d
(n − s) + 1 (s−1)−d
d=0 d=0
d A forward difference of discrete function f : Z → Z is a function ∆f : Z → Z with ∆f (n) =
f (n + 1) − f (n) (cf. [1]).

In this equation the sum is finite because due to the observation that the s-th
difference is constant 1s , all higher differences are 0.
Thus, the correction factor is θ = γ1 . Extensive experiments with Mathemat-
n−c
ica and MATLAB indicate that and formula (3.1) indeed coincide
(n − s) + γ1
numerically. We can also see from Figure 3.5 that the correction factor is small.
In the next section, we will prove these two formulas are equal to each other. The
methodology applied in the current section will be further discussed in Chapter 8.
3.5 Another Expression for Pdrop

There is another expression for Pdrop . According to the assumption, SA is a
uniform selection from CA , which, in turn, is a uniform selection from the entire
population of n items. SA is chosen uniformly at random from CA , and SB is
chosen from CB . Using this, we revise the probabilities in Section 3.4.2. The
probability P|SA ∩SB |=bs of having exactly sb items in SB ∩ SA is:
s
n−s

sb s−b
P|SA ∩SB |=bs = n
s ,
s
and the probability of dropping the item is:

s
X |S ∩SB |=b
s
E[Pdrop ] = (PdropA · P|SA ∩SB |=bs ).
sb=0
Recall the shuffle protocol. After the exchange, node A uses random items from
SA to fill its cache (CA \SA ) ∪ SB up to c items. That is, exactly |(CA \SA ) ∩ SB |
items are randomly chosen from SA \SB . Basically, the items in SA \SB (that are
also in CA ) are replaced by the items in SB \CA . Since the numbers of items in
SB \SA and in SA \SB are equal, a one-to-one correspondence can be arranged
by random selection of pairs from the sets. If an item in SB \SA is also in CA , the
pair item from SA \SB is kept, and dropped, otherwise. Thus, the probability of
an item in SA \SB to be dropped depends only on the probability that a pair item
in SB \SA is not in CA . In view of the uniform selection, every item in SA \SB
has a fair chance of being replaced by an item in SB \CA . Moreover, an item in
SB \SA has a chance to be any item in CA \SA . Hence, the probability that the
n−c
item is not in CA is n−s .
|S ∩S |=b
s
Thus, the probability PdropA B of an item in SA \SB to be dropped from
CA , given sb items in SB ∩ SA , is:
(
|S ∩S |=b
s 0 if s = sb
PdropA B = n−c
n−s otherwise
3.5. Another Expression for Pdrop 35
Putting the pieces together we obtain a formula E[Pdrop ] for every s ≤ c ≤ n:

s−1 s
n−s
s !
X sb s−b n−c n−c X s n−s
E[Pdrop ] = n
s · = −1
sb=0 s
n−s (n − s) ns sb=0
sb s − sb
!
e n−c 1
= · 1 − n (3.4)
n−s s
Formulas (3.1) and (3.4) are exact solutions of the counting problem, consti-
tuting a combinatorial proof of their equality. Formula (3.2) has been derived by
extensive numerical experiments. Its equality to the formula (3.4) (and transi-
tively, to (3.1)) can be proven algebraically by applying the well-known Gosper
algorithm [96, 157].
Proposition 3.1. For n > 0, s > 0 and s ≤ n, the following equality holds:
s−1 n

X s n
d
s−1
= −1 (3.5)
d
n−s s
d=0
Proof. We first modify the fraction of the binomials as follows:

n
n
n

d d s−1−d d
s−1 = s−1
1
= · s−1 (3.6)
d d−1 · d(s−1−d)
d d−1
Instead of computing the sum on the left hand side of (3.5), Gosper’s algo-
rithm [96, 157] allows us to get a result of the sum by solving the first-order
linear recurrence equation
qd · f (d + 1) − rd−1 · f (d) = pd (3.7)
where f (d) is an unknown rational function of d, and qd , rd , pd are polynomials

in d.
(nd)
Let ad = s−1 . The summand ad in (3.5) is a hypergeometric term. Thus, we
( d )
apply Gosper’s algorithm to see if the sum on the left hand side of (3.5) can be
expressed as a hypergeometric term. That is, given the hypergeometric term ad ,
rs−1 · f (s)
to find a hypergeometric term Zs = · as , such that:
ps
s−1
X
ad = Z s − Z 0
d=0
e Vandermonde’s
Pk m n−m n
identity states that i=0 j k−j
= k
holds for binomial coefficients.
ad+1 n−d ad+1 pd+1 qd

The term ratio = , also defined by Gosper as = , is
ad s−1−d ad pd rd
rational in d as expected. To find a nonzero rational solution f (d) of the first-
order recurrence (3.7), we choose qd = −(n − d), rd = −(s − d − 1), pd = 1. The
equation
−(n − d) · f (d + 1) + (s − d) · f (d) = 1
1
has a nonzero solution f (d) = − . We substitute the solution f (d), the
n−s
summand as , and the polynomials rs and ps into the expression for Zs :

(3.6) 1 n s s n s
Zs = s−1 = n − s s , and Z0 = n − s .

n − s s s−1
Hence,
s−1 n

X s n
d
s−1
= ZS − Z0 = −1
d
n−s s
d=0
Through simple algebraic manipulations, we finally obtain the following equal-

ity:
!
n−c n−c 1
s = · 1 − n (3.8)
(n − s) + s−1 n
n−s s
X
d
s−1

d=0 d
3.6 Optimal Size for the Exchange Buffer

We study the optimal value for fast convergence of replication and coverage
with respect to an item d. Since d is introduced at only one node in the network,
one needs to maximize the chance that an item is duplicated. That is, the proba-
bilities P (11|01) and P (11|10) should be maximized (then P (01|11) and P (10|11)
are maximized as well, intuitively because for each duplicated item in a shuffle,
another item must be dropped).
We give a rigorous argument for this claim in the case of a fully connected
network. Let α be a replication of the distinguished item at a given moment, and
0 ≤ α ≤ 1. In the long run, α converges to nc . Suppose that a shuffle takes place.
The chance that after the shuffle there is one more copy of the distinguished item
in the system is
(P (11|01) + P (11|10)) · α · (1 − α) = 2 · P (11|01) · α · (1 − α)

Here α · (1 − α) expresses the chance that exactly one of the nodes in the shuffle
contains a copy of the item. Likewise, the chance that after the shuffle a copy of
the item has been removed from the system is
n−c
(P (01|11) + P (10|11)) · α2 = 2 · P (01|11) · α2 = 2 · · P (11|01) · α2
c
So after a shuffle, the change in the number of copies of the item in the system
is on average
n−c n
2 · P (11|01) · α · (1 − α) − 2 · · P (11|01) · α2 = 2 · P (11|01) · α · (1 − · α)
c c
c
So as long as α < n, maximizing P (11|01) maximizes replication of the item.
90
85
optimal exchange buffer size
80
75
70
65
60
55
50
0 250 500 750 1000
items (n)
Figure 3.6: Optimal value of exchange buffer size, depending on n.
These probabilities both equal sc n−s

c−s
; we compute when the s-derivative of
this formula is zero. This yields the equation
s2 − 2ns + nc = 0
Taking into the account that s ≤ n, the only solution of this equation is
p
s = n − n(n − c)
We conclude that this is the optimal value for s to obtain fast convergence of
replication (see Figure 3.6). This will also be confirmed by the experiments and
analysis in the following sections.

In order to test the validity of the analytical model of information spread
based on the shuffle protocol presented in the previous section, we follow an
experimental approach. This section presents the comparison of properties ob-

served while running the shuffle protocol in a large-scale deployment with sim-
ulations of the model under the same conditions. These experiments show that
the analytical model indeed captures information spread of the shuffle protocol.
Note that a simulation of the analytical model is much more efficient (in compu-
tation time and memory consumption) than a simulation of the implementation
of the shuffle protocol, see Section 3.8.
The experiments simulate the case where a new item d is introduced at one
node in a network, in which all caches are full and uniformly populated by n =
500 items. The protocol has been implemented in an event-based simulator that
takes as input the topology of the network to determine which pairs of nodes
can gossip. The experiments are performed on a network of N = 2500 nodes,
arranged in a square grid topology (50×50), where each node can communicate
only with its four immediate neighbors (to the North, South, East and West). This
configuration of nodes is arbitrary, except for the requirement of a large number
of nodes for the observation of emergent behavior. Our aim is to validate the
correctness of our analytical model, not to test the endless possibilities of network
configurations. The model and the shuffle protocol do not make any assumptions
about the network. The network configuration is provided by the simulation
environment and can easily be changed into something different, e.g., another
network topology. For this reason, the large grid has been chosen for testing,
although other configurations could have been possible. In the experiments that
follow, after each gossip round, the following measurements are performed:
(i) the total number of occurrences of d in the network (replication), and
(ii) how many nodes in total have seen d (coverage).
Simulations with the shuffle protocol Each node in the network has a cache
size of c = 100, and sends s items when gossiping. In each round, every node
randomly selects one of its neighbors and shuffles. In order to make a fair com-
parison with the simulations with the model, the nodes gossip for 1000 rounds
before initiating the measurements of the properties. After this start up period
of 1000 rounds, items are replicated and the replicas fill the caches of all nodes
fulfilling the uniform distribution requirement of the model. At round 1000, item
d is inserted into the network at a random location. From that moment on its
replication and coverage are recorded.
Simulations with the model For the simulations with the model, n, c and s
are only system parameters. Instead of maintaining a cache, each node in the
network maintains only a variable that represents whether it holds item d or not
(state 1 or 0, respectively). Nodes update their state in pairs according to the
transition probabilities introduced before (Figure 3.2). This mimics an actual
exchange of items between a pair of nodes according to the shuffle protocol.
While in the protocol this results in both nodes updating the contents of their
caches, in a simulation using the analytical model updating the state of a node
refers to updating only one variable: whether the node is in possession of the
item d or not. To sum up, we use transition probabilities to update the state of
one variable. Since we do not need a startup time for the simulations with the
model, at round 0 we set the state of a random node to 1 (while all the other
have state 0) and track the state of the nodes for the remainder of the simulation.
600 1
500
coverage (% of nodes)
0.8
number of replicas
400
0.6
300
0.4
200
s = 95 0.2 s = 95
100
s = 50 s = 50
s = 10 s = 10
0 0
0 200 400 600 800 1000 1200 1400 0 200 400 600 800 1000 1200 1400
rounds rounds
0.001 0.0004
s=10 (diff) s=10 (diff)
0.0001
replication diff (shuffle-model)
0.0002
coverage diff (shuffle-model)
1e-05 0
0.001 s=50 (diff) 0.0006 s=50 (diff)
0.0004
0.0001
0.0002
1e-05 0
0.001 s=95 (diff) 0.0004 s=95 (diff)
0.0001 0.0002
1e-05 0
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
rounds rounds
Figure 3.7: Grid 50 × 50, range 1 with n = 500, c = 100. Top: Replication (left)
and coverage (right) of item d for the shuffle. Bottom: difference between the
shuffle and the model for replication (left) and coverage (right).
Figure 3.7 shows the behavior of both the shuffle protocol in terms of repli-
cation and coverage of d (upper row of Figure 3.7, the left and the right graphs,
respectively), and the difference between the results of the protocol and the ana-
lytical model (lower row of Figure 3.7), for various values of s. Each curve in the
top graphs represents the average and standard deviation calculated over 100
runs. The experiments with the model calculate Pdrop using the simplified for-
n−c
mula n−s described in Section 3.4.3. It can be seen very clearly from the lower
graphs that the difference between the results obtained from the shuffle and the
model is very small. Note that measure unit of y-axis is the fraction of network.
We note that in all cases, the network converges to a situation in which there
500
are 500 copies of d, meaning that replication is 2500 = 0.2; this agrees with the
c 100
fact that n = 500 = 0.2. Moreover, our experiments show that replication and
coveragep display the fastest convergence

√ when s = 50; this agrees with the fact
that n − n(n − c) = 500 − 500 · 400 ≈ 50 (cf. Section 3.6).
The characteristics of the replication and coverage are influenced by the
topology of the network. It is the topology of the network that will dictate the
likelihood of any two nodes to gossip. In order to model the properties of the
protocol, it is crucial that we know the probabilities of a node in a given state (0
or 1) to interact with another node in a given state. For this reason, we choose
to model the properties for a fully-connected network, where a node can gossip
with any other node in the network. This topology allows us to easily calculate
the probability of a node in state 1 to interact with a node in state 0 or to interact
with another node in state 1. Knowing this, we concentrate on applying the state
transition probabilities that we calculated in Section 3.4. The aim of this section
is to provide an example of how the transition probabilities can be used to model
the properties of dissemination for a specific topology.
3.8 Time Complexity of Experiments

In this section we discuss some topics/issues that arose during the develop-
ment and testing of the model for the shuffle protocol.
During the experimental phase of this work, we observed a remarkable dis-
parity between the time required to run an experiment with the shuffle protocol
or with the model. In both cases, the experiment was the same in terms of prop-
erties being measured and parameters used (cache size c, exchange buffer size s,
number of different items n and network size N ).
The difference in execution times can be traced back to the two different
algorithms executed at each simulated node. From a node’s point of view, the
shuffle protocol requires the selection of items to send to the gossip partner and
the selection of items to keep for the next round. The first operation can be done
in linear time, O(c). The second operation requires checking if the incoming
items are already in the cache and removing entries from the cache if space is
needed. These steps can be done in O(c · log c) time. With the second operation
dominating the execution time, we can estimate the time complexity for one
round of the shuffle to be O(c · log c).
Now, let us look at the time complexity of the model. Unlike the protocol
implementation, the model requires very little state, namely the value of the
parameters and a variable to indicate the presence or absence of item d. At
each round, each simulated node and its gossip partner only have to determine
their current state (a1 , b1 ) and transition to a new state (a2 , b2 ) according to the
appropriate transition probabilities. The transition probabilities themselves do
not change during the simulation (since they depend solely on the parameters
c, s and n), so they are precomputed at the initialization phase. As a result, the
execution of the model for each node has a constant time complexity, O(1).
The defining factor in the execution time of the simulations with the shuffle
3.9. Related Work 41
protocol is the size of the cache. With a cache size of 100 for all of our exper-
iments, the execution times for our simulations with the shuffle protocol and
the model differed by approximately two orders of magnitude. Considering that
large networks are needed to clearly observe emergent behavior and that this be-
havior evolves over many rounds, the value of having a model becomes evident.
Being simply parameters in the model, the cache size and exchange buffer size
have no effect on the memory requirements and execution time of the simula-
tion, thus freeing computational resources to experiment with larger networks
and different topologies.
3.9 Related Work

Two areas of research are most relevant to the model, presented in this chap-
ter: rigorous analysis of gossip (and related) protocols, and results from math-
ematical theory of epidemics [17, 69]. The results from epidemics are often
used in the analysis of gossip protocols [83] (e.g., the traditional gossip paper
by Demers et al. [75]). Thus, the scope of the overview is restricted to the most
relevant publications from the area of (analysis of) gossip protocols.
Several works have focused on gossip-based membership management pro-
tocols. Allavena et al. [3] proposed a gossip-based membership management
protocol and analyzed the evolution of the number of links between two nodes
executing the protocol. The states of the associated Markov chain are the number
of links between pairs of nodes. From the designed Markov chain they calculated
the expected time until a network partition occurs. This case study also includes
a model of the system under churn. A goal of that paper is to show the effect of
mixing both pull and push approaches.
Eugster et al. [82] presented a lightweight probabilistic broadcast algorithm,
and analyzed the evolution of processes that gossip one message. The states
of the associated Markov chain are the number of processes that propagate one
gossip message. From the designed Markov chain, the authors computed the
distribution of the gossiping nodes. Their analysis has shown that the expected
number of rounds to propagate the message to the entire system does not depend
on the out-degree of nodes. These results are based on the analysis assumption
that the individual out-degrees are uniform. However, this simplification has
shown to be valid only for small systems (cf. [123]).
Bonnet [46] studied the evolution of the in-degree distribution of nodes exe-
cuting the Cyclon protocol [177]. The states of the associated Markov chain are
the fraction of nodes with a specific in-degree distribution. From the designed
Markov chain the author determined the distribution to which the protocol con-
verges.
There are a number of theoretical results on gossip protocols, targeted to
distributed aggregation. In these protocols, a set of data is distributed over the
nodes of a network and the nodes compute an aggregate of the data set. Kempe
et al. [129] proposed a push-only gossip-based aggregation protocol for a fully

connected network. In this paper, the authors used Gaussian mixture modelling
[76, 145]. A performance of the protocol has been measured by how quickly
a data item originating with a node diffuses through a network (for uniform
gossip). Each node locally maintains an aggregation vector vt,i . A state of the
associated Markov chain is the fraction of the vector node i sends to other node.
From the designed Markov chain, the authors studied the convergence rate. In
addition, the authors showed that the diffusion speed for flooding corresponds to
the mixing time of a random walk on the network. Validation of the theoretical
results with practical experiments is left as future work.
The protocol [129] has been further tailored by Boyd et al. [50] to work on an
arbitrarily connected network. In their analysis, the Markov chain is defined by a
weighted random walk on the graph. Every time step, a pair of nodes (connected
by an edge) communicates with a transition probability, and sets their values
equal to the average of their current values. A state of the associated Markov
chain is a vector of values at the end of the time step. The authors considered
the optimization of the neighbor selection probabilities for each node, to find
the fastest-mixing Markov chain (for fast convergence of the algorithm) on the
graph.
Jelasity et al. [121] proposed a push-pull solution for aggregation in large
dynamic networks, supported by a performance analysis of the protocol. A state
of the system is represented by a vector, the elements of which correspond to
the values at the nodes, a target value of the protocol calculated from the vector
elements, and a measure of homogeneity characterizing the quality of local ap-
proximations. The vector evolves at every step of the system according to some
distribution. In the analysis, the authors considered different strategies (e.g.,
neighbor selection) to optimize the protocol implementation, and calculated the
expected values for the above mentioned protocol parameters.
Deb et al. [72] studied the adaptation of random network coding to gossip
protocols. The authors analyzed the expected time and message complexity of
two gossip protocols for message transmission with pure push and pure pull com-
munication models.
Busnel et al. [54] have studied a gossip-based membership protocol, where
nodes exchange a part of their neighborhood lists like in the shuffle protocol.
The authors have shown that the system executing the protocol, converges to a
configuration, where caches of all nodes have uniformly distributed items (links).
In [61] and its sequel [60], the speed of the convergence of push-pull gossip
has been calculated for network graphs with respect to their connectivity.
3.10 Conclusions
This chapter has demonstrated that it is possible to model a gossip protocol
through a rigorous probabilistic analysis of the state transitions of a pair of nodes
3.10. Conclusions 43
engaged in the gossip. It has been shown, through an extensive simulation study,
that the dissemination of a data item can be faithfully reproduced by the model.
Having an accurate model of node interactions, the following results have been
produced:
• After finding precise expressions for the probabilities involved in the model,
we provide a simplified version of the transition probabilities. These sim-
plified, yet accurate, expressions can be easily computed, allowing us to
simulate the dissemination of an item without the complexity of executing
the actual shuffle protocol. These simulations use very little state (only
some parameters and variables, as opposed to maintaining a cache) and
can be executed in a fraction of the time required to run the protocol.
• The model reveals relationships between the parameters of the system.
Armed with this knowledge, we successfully optimized one of the parame-
ters (the size of the exchange buffer) to obtain the fastest convergence of
the observed properties.
While gossip protocols are easy to understand, even for a simple push-pull pro-
tocol, the interactions between nodes are unexpectedly complex. Understanding
these interactions provides insight into the mechanics behind the emergent be-
havior observed in gossip protocols. We believe that understanding the mechan-
ics of gossip is the key to optimizing (and even shaping) the emergent properties
that make gossip appealing as communication paradigm for distributed systems.
Next Steps
In this chapter, we have considered a simple push-pull gossip protocol that
was originally proposed for wireless environments. After successfully analyzing
this protocol at an abstract level (pairwise node interactions) independent of net-
work topology or the characteristics of the environment itself, we are interested
in the following possibilities for future study:
• We would like to investigate how our transition probabilities can be applied
to model the properties of the protocol. As a first step, we can model the
properties for a fully-connected network.
• In order to have a more realistic representation of a wireless setting, we
would like to incorporate the presence of lossy links in our network. The
characteristics of the lossy links would model an underlying MAC layer. In
other words, we would like to relax the assumption of atomic transactions
for the shuffle protocol. This would result in new possible transitions in the
state transition diagram (Figure 3.2) due to loss of messages.
We will cover both items in the next chapters.
Pairwise Exchange Model in
Lossy Networks 4
Previous studies of gossip-based information dissemination protocols, includ-
ing the one presented in Chapter 3, have relied on the assumption that nodes
interact with each other through perfect, lossless communication channels. In
this chapter, we drop this simplifying assumption and explore the impact of lossy
channels on a gossip protocol with a push-pull information exchange. To analyze
the protocol in the presence of message loss, we introduce into the formal model
a component based on statistical data from Monte-Carlo simulations. We success-
fully validate the obtained model against simulations. As opposed to the models
produced by traditional probabilistic verification methods, our analytical model
scales well with the network size, and captures the performance of the proto-
col in very large networks. Our results show that the presence of message loss,
coupled with specific topology configurations, significantly impact the expected
behavior of the protocol.
4.1 Introduction
We now consider ad hoc networks, where nodes are continually communicat-
ing with each other for information exchange over an unreliable medium. On the
one hand, gossip protocols are considerably more robust compared to determin-
istic protocols due to randomness and the distributed nature, also in the presence
of failures and data loss. On the other hand, since the work of Demers et al. [75],
gossip protocols often follow bidirectional data exchange (push-pull) for better
performance. As observed in [79], the performance of these protocols is usually
evaluated under the assumption of a perfect, error-free communication medium.
Thus, data exchange operations between nodes are previously considered to be
atomic operations, e.g., [28, 52, 78, 94, 120, 140, 167, 177]. That is, if a node
initiates a gossip with another node, both nodes base their local decisions upon
each other’s data, and in some protocols even guarantee the preservation of each
45
46 Chapter 4 — Pairwise Exchange Model in Lossy Networks
other’s data. In practice, however, implementing data exchange as an atomic op-

eration is hard to achieve assuming communication over unreliable media [170].
This chapter investigates the impact of a realistic environment with lossy com-
munication channels on a push-pull-based gossip protocol. We take the analytical
model of the shuffle protocol introduced in Chapter 3 as the starting point of our
study. This model is revised under the assumption that the shuffle operation is
not atomic. The messages sent by a node to its gossip partner get lost with a
certain probability, due to the unreliability of the communication channels.
We present the analytical model of a gossip protocol in the presence of mes-
sage loss. An exchange of items between nodes is modelled as a state transition.
We consider every state transition and compute the corresponding probabilities.
Like in the previous pairwise model, an important part of state transition ma-
trix is the probability Pdrop that an observed item in a node’s local storage has
been replaced by another item after the gossip. Since it is not feasible to cover
all possible network topologies in one formula, the computation of this proba-
bility is based on the statistical data that can be obtained from the Monte Carlo
simulations of the protocol. To be more precise, we deconstruct the expression
for Pdrop into its main components, and identify the ones that depend on specific
scenario configurations, i.e., message loss and topology combinations. Such a
component is the probability of an observed item found in one of the caches to
be also found in the other. Instead of finding analytical expression for every spe-
cific configuration, which would result in the set of infinitely many expressions,
we use statistical data sampling to obtain the value for this probability, thereby
isolating the modelling framework from the scenario-specific components.
Furthermore, we present in more details the following discoveries:
• Introducing message losses in the network model affects the emergent be-
havior of the protocol in a specific manner. With the introduction of lossy
communication channels, the correlation between cache contents of neigh-
boring nodes increases, and the degree of the correlation depends on the
network topology.
• The smaller the radio range of nodes, the stronger the effect of message loss
on the emergent behavior of the protocol. For fully connected networks,
message loss does not have an impact on the distribution of data over the
network, which remains uniform (as observed when there are no losses).
Related Work
So far, there has not been much work that studied the performance of gossip
protocols in the presence of message loss. There is previous work on a simple
gossip-based membership protocol with nonatomic protocol actions in the pres-
ence of message loss [100]. To overcome the problem that a message from a
peer can get lost, the authors proposed a push-based gossip protocol, wherein
only the node initiating a gossip sends a message to its random peer, and imme-
diately removes the sent data from its cache. Moreover, a node sends only two
4.2. Our Assumptions 47
ids (e.g., IP addresses and ports) of itself and a random one from its local cache.
In the shuffle protocol, however, each gossiping pair exchanges a random subset
of the data items. The authors proved the correctness of this simple push-based
protocol modelling it by graph transformations, similar to the approach in [67].
The protocol performance is analyzed assuming up to 1% of the message loss.
The protocol properties analyzed include the expected number of neighbours a
node has and the uniformity of the neighborhood lists. The paper has presented
pure theoretical findings and did not include experimental evaluation.
Another work [84] combined model checking and Monte Carlo simulations
for the performance analysis of a very simple probabilistic broadcast in the pres-
ence of message loss. The scope and the framework of that paper differs from
ours; it aims at studying how different modelling choices impact the performance
of the probabilistic system. The authors simply compare the results of modelling
for a small network of nine nodes (producing the state space of the model) to
the results of Monte Carlo simulations for a network of 1000 nodes.
A survey on consensus-based distributed algorithms [77] summarizes an ob-
servation regarding the effects of intermittent or lossy links for probabilistic con-
sensus algorithms. The authors conclude that the topology directly affects the
convergence rate, and in the connected network, convergence of consensus al-
gorithms is not affected by lossy or intermittent links, and convergence speeds
degrade gracefully. The paper suggests that the same observation is applicable to
gossip protocols, but no analysis is carried out to support the claim.
4.2 Our Assumptions

We consider the shuffle protocol for our case study, and refer to Chapter 2 for
the description. In contrast to the previous model, we now assume that commu-
nication channels are lossy. That is, every message sent has a positive probability
to be lost due to a disturbance of the communication medium, for example, in a
wireless network. Moreover, message losses occur with the same probability and
are independent. We do not assume that the shuffle procedure is atomic. When
a node has sent a message, it does not know whether the message is delivered
successfully.
A B A B A B
(a) (b) (c)
Figure 4.1: Scenarios of communication with lossy channels: (a) loss of the
request message, (b) loss of the reply message, (c) communication without loss.
There are three general cases in pairwise communication with respect to mes-
sage delivery, as depicted in Figure 4.1: (a) node A initiates a gossip with B by
sending a message, but the message is lost, (b) node A successfully initiates a
gossip with B, but a message from B is lost on its way to A, and (c) a gossiping
pair successfully receives messages from each other.
4.3 A Model of Information Spread with Lossy Channels

We analyze the spread of a data item d in a network of the nodes executing
the shuffle protocol. In this analysis, we assume that the shuffle operation is
nonatomic and communication channels are prone to losses.
We consider our analytical model of the shuffle protocol from Chapter 3, that
captures the presence or absence of a generic item d before and after the shuffle
between two nodes A and B. The model has four possible states of the caches of
A and B before and after the shuffle: when both hold d, either of the caches holds
the item d, and neither cache holds d. These correspond to the states (0, 0), (0, 1),
(1, 0), and (1, 1) in the state transition diagram, shown as Figure 4.2. Unlike in
Chapter 3, in our current model, the state (0, 0) is no longer isolated, since there
is a possibility to remove the only copy of d from the cache of the node B, in the
scenario shown in Figure 4.1(b).
Transitions from one state to another are
labelled by the respective transition probabil- P (00|00) P (01|01)
ities P (a2 b2 |a1 b1 ), where a1 b1 is the state be- P (00|01)
fore a shuffle, and a2 b2 is the state after the (0, 0) (0, 1)
shuffle, with ai , bi ∈ {0, 1}. The bits a1 , a2
)
and b1 , b2 indicate the presence (if equal to 1)

10
P (11|01)
1|
P (01|11)
(0
or the absence (if equal to 0) of a generic item

)
01
P
d in the cache of an initiator A and the con-

0|
(1
tacted node B, respectively. Note that these

P
transition probabilities differ from the ones P (11|10)

calculated for the pairwise interaction model (1, 0) (1, 1)
in Chapter 3. P (10|11)
We express all transition probabilities P (10|10) P (01|01)
P (a2 b2 |a1 b1 ) of the state diagram in terms of Figure 4.2: State diagram for
three probabilities: 1) the probability Pselect caches of gossiping nodes.
of an item to be selected by a node from its
cache for an exchange, 2) the probability Pdrop that an item is replaced by an-
other one, received by its node in the shuffle, and 3) the probability Ploss that
a message is not delivered to a node due to channel loss. We write P¬select for
1 − Pselect , P¬drop for 1 − Pdrop , and P¬loss for 1 − Ploss .
State (0, 0). Before shuffling, neither node A nor node B have d in their cache.
a2 b2 = 00: neither node A nor node B have item d after a shuffle because neither
of them had it in the caches before the shuffle: P (00|00) = 1.
a2 b2 ∈ {01, 10, 11}: cannot occur, because none of the nodes have item d.
4.3. A Model of Information Spread with Lossy Channels 49
State (1, 0). Before shuffling, d is only in the cache of node A.

a2 b2 = 01: this outcome occurs when both request from A and reply of B are
successful and node A selects and drops d; that is, the probability is
2
P (01|10) = (P¬loss ) · Pselect · Pdrop
a2 b2 = 10: node B does not have d because: (a) A did not select d or (b) A
selected d, but the request message got lost on the way to B.
P (10|10) = P¬select + Ploss · Pselect
a2 b2 = 11: both nodes A and B have a copy of d because: (a) either both nodes
received the gossip messages and A selected d and kept it, or (b) A selected
d and the reply message from B got lost; that is,
2
P (11|10) = (P¬loss ) · Pselect · P¬drop + P¬loss · Ploss · Pselect
a2 b2 = 00: cannot occur as A would only drop d if it received a reply, which
implies that B would keep d.
State (0, 1). Before shuffling, a copy of d is only in the cache of node B.
a2 b2 = 01: only B has d because the message from A got lost or, B received the
message, and: (a) it did not select d or (b) B selected d, kept it, and reply
got lost; i.e. the probability is
P (01|01) = P¬loss (P¬select + Ploss · Pselect · P¬drop ) + Ploss
a2 b2 = 10: both request and reply messages were successfully delivered and B
selected and dropped d, which amount to the probability
2
P (10|01) = (P¬loss ) · Pselect · Pdrop
a2 b2 = 11: both messages were delivered successfully, and B selected and kept
d; that is, P (11|01) = (P¬loss )2 · Pselect · P¬drop .
a2 b2 = 00: neither of nodes have d because B selected and dropped d, but node
A did not receive the reply message. P (00|01) = P¬loss · Ploss · Pselect · Pdrop .
State (1, 1). Before shuffling, d is in the cache of node A as well as in the cache
of node B.
a2 b2 = 01: only node B has d because both messages were successfully received,
A selected and dropped d while B did not select d.
2
P (01|11) = (P¬loss ) · Pselect · Pdrop · P¬select
a2 b2 = 10: only node A has d because node B selected d, dropped it, and node
A did not select d and either (a) did not receive a message from B, or (b)
received the message from B: P (10|11) = P¬loss · P¬select · Pselect · Pdrop .
a2 b2 = 11: after the shuffle both nodes A and B have d, because:
a) the message from A did not arrive at B, i.e. Ploss ;
b) the message from A successfully received by B, but the reply message got
lost, and:
⋆ A did not select d while B (i) did not select d as well or (ii) selected and
kept d. P¬loss · P¬select · Ploss · (P¬select + Pselect · P¬drop )
⋆ node A selected d: P¬loss · Pselect · Ploss
c) both nodes received each other messages, and:
⋆ A did not select d while B (i) also did not select it or (ii) selected and
kept it: (P¬loss )2 · P¬select · (P¬select + Pselect · P¬drop )
⋆ (i) A selected and kept d while B did not select d or (ii) both nodes
2
selected d. (P¬loss ) · Pselect · (P¬select · P¬drop + Pselect )
Hence, P (11|11) = 1 − (2 − Ploss · (3 − Ploss )) · Pselect · P¬select · Pdrop .
a2 b2 = 00: discarding of an item by both nodes cannot occur.
Hence, we obtain the resulting expressions for the transition probabilities in
Figure 4.2, summarized in Figure 4.3.
P (00|00) = 1 P (10|10) = 1 − P¬loss · Pselect

P (10|01) = (P¬loss )2 · Pselect · Pdrop P (01|10) = (P¬loss )2 · Pselect · Pdrop
P (11|01) = (P¬loss )2 · Pselect · P¬drop P (01|11) = (P¬loss )2 · Pselect · P¬select · Pdrop
P (00|01) = P¬loss · Ploss · Pselect · Pdrop P (10|11) = P¬loss · P¬select · Pselect · Pdrop
P (01|01) = P¬loss (P¬select + Ploss · Pselect · P¬drop ) + Ploss
P (11|10) = Pselect · P¬loss · (1 − P¬loss · Pdrop )
P (11|11) = 1 − (2 − Ploss · (3 − Ploss )) · Pselect · P¬select · Pdrop
Figure 4.3: Revised transition probabilities
Again, our analytical model of the shuffle protocol uses the probabilities
Pselect , Pdrop and Ploss as building blocks. While Ploss is an input parameter that
expresses the reliability of the communication channels, Pselect and Pdrop are de-
rived based on the behavior of the protocol. Pselect describes the random selection
of items from the cache; an item has sc chance of being selected, regardless of
message loss. The calculation of Pdrop is complex, and we discuss it in the follow-
ing section.
4.4 The Probability of Dropping an Item

In this section we analyze how message loss affects one of the building blocks
of our model: Pdrop . In our earlier work (assuming perfect communication, see
Chapter 3), we had already found an expression for Pdrop , relying on the assump-
tion of uniform distribution of items. Here, we revisit Pdrop and derive a general
formula without making any assumptions about the distribution of items. Later
on, we will explore how message loss affects the distribution of items, and will
determine that it is the coupling of message loss and the topology of the network
that affects Pdrop . Having isolated the component of Pdrop that is affected by the
message loss/topology coupling, we will propose the use of statistical data to
calculate that specific part of the Pdrop expression.
4.4. The Probability of Dropping an Item 51
When a node selects an item to be sent to a neighbor, there is a probability

Pdrop that the item will be dropped from the node’s cache after the gossip ex-
change. The selected item may be dropped only when there is a need to create
space for an item received from a gossip partner. Therefore, the probability Pdrop
depends on the relationship between a) the new items received from the gossip
partner (for which the node needs space, the shaded area in Figure 4.4(b), re-
ferred to as A1 ), and b) the items that the node has selected to send to the gossip
partner and is allowed to discard (the shaded area in Figure 4.4(c), referred to as
A2 ). In order to find an expression for Pdrop , we need to calculate the probability
of an item being in A1 and the probability of an item being in A2 , which we will
denote as P (A1 ) and P (A2 ), respectively.
c = CA ∩ CB
b A1 = SA \ CB A2 = SB \ SA
CB CB CB
CA cb SA SA
SB SB
(a) (b) (c)
Figure 4.4: a) Items in common for caches of both nodes A and B; b) Items
sent by A that are not in the cache of B; c) Items sent by B that are not in the
exchange buffer of A.
Given two nodes, A and B, that engage in a gossip exchange, we can repre-
sent their caches as sets CA and CB , and the sets of items they exchange with
each other by SA and SB , respectively. The sets CA and CB may have common
elements (items), see Figure 4.4(a). We define Pinx as the probability of an item
found in one of the caches to be also found in the other, i.e., Pr(d ∈ CB | d ∈ CA )
for any item d, one of n items in the network. Knowing Pinx , we can formulate
P (A1 ) as Pselect · (1 − Pinx ) and P (A2 ) as Pselect · (1 − Pselect · Pinx ). The probability
Pdrop can then be expressed as:
P (A1 ) 1 − Pinx
Pdrop = = (4.1)
P (A2 ) 1 − Pselect · Pinx
In order to calculate a value for the probability of dropping an item, Pdrop , it is
necessary to have a value for the probability Pinx . For the case of CA and CB
being random samples of a population of n items, that is, when the system is
in the stable state, we can analytically deduce that Pinx = nc . In other words,
under the assumption that items are uniformly distributed over the network, we
can find an expression for Pinx . In our earlier work modelling the shuffle pro-
tocol (i.e.,Section 5.5 under the assumption of perfect communication channels
we discovered that repeated execution of the protocol results in a uniform dis-
tribution of items. In the next section, we look at the effect that lossy channels
have on the distribution of items.
4.4.1 Uniform Distribution of Items: Does it Still Hold?

As already mentioned in Section 4.3, the calculation of Pdrop in the presence
of message loss requires us to speculate about the contents of the gossip partner’s
cache. Assuming that the items in the caches of both gossip partners are random
samples of the totality of items in the network, we can easily estimate Pinx as
c n−c
n and, therefore, have an analytical expression for Pdrop ≈ n−s . In this section,
we verify whether the uniform distribution of items, observed under no message
loss, is still a valid assumption.
The Importance of Randomness

In order to understand the effect of message loss on the theoretical level,
consider one node in the network, executing the shuffle protocol, with a neigh-
borhood of b nodes.
c = CA ∩ CB
b c = CA ∩ CB
b
A B
CA b
c CB −−−−−−−−−−→
CA c ∪ SA
b CB
Figure 4.5: Correlation of the caches increases due to message loss

We examine the scenario, depicted in Figure 4.1(b), where a reply message
from a node to the initiating node is lost due to channel failure. When two nodes
A and B gossip their local items to each other, the probability that the message
from B to A is lost, is P¬loss · Ploss .
If the message of B is not delivered to A, items b c ∪ SA will be common
to cache CA and cache CB after the shuffle since B, unaware of the failure,
purges the sent items SB , and A, which received no reply, keeps its SA items
(see Figure 4.5). Over time neighboring nodes have a growing intersection of
items in their cache. If a node has a small neighborhood, the random sample
becomes more biased towards the collection of items left in the neighborhood.
Limited by the access to the storage space of the neighborhood, a set of neighbors
maintains only a subset of all items. For the fully connected graph, since it is the
entire collection of the items every node has access to, the uniform distribution
assumption remains. A larger neighborhood reduces the probability of repeated
communication between two nodes, i.e. ( 1b )i to contact each other i times, and
due to the increased communication range, there is a faster exchange between
distant areas of the network.
Experimental Observations
To support our theoretical discovery that message loss affects the uniform
distribution of items observed under no message loss, we conduct simulation
4.4. The Probability of Dropping an Item 53
experiments. The setup of the experiments is the following: each simulation

experiment has a startup period of 1000 rounds during which N = 2500 nodes
gossip n = 500 different items under no message loss. We use three different
network topologies: a fully connected network, a square grid with each internal
node having 4 neighbors and a network where every node has 4 randomly chosen
neighbors (outdegree 4). By the end of the startup period, the items have been
replicated, achieving uniform distribution. That is, every item has a probability
c 100
n = 500 = 0.2 of being present in a given node’s cache. After the startup period
has finished, the communication channel is set up to fail with a probability Ploss .
Under this new scenario, we record, for a given item x, the fraction of shuffles
between two nodes that both have x, over the total number of shuffles where
at least one of the nodes has x. With this information, we calculate, per round,
the probabilities P (11), P (10) and P (01), which represent the probability that
a gossip exchange involves both nodes having item x, only the initiator having
item x, or only the gossip partner having item x, respectively.
Figure 4.6 shows the average values of P (11), P (10) and P (01) over 1000
rounds for different values of Ploss using the three topologies mentioned ear-
lier. As expected, for the case of no message loss (Ploss = 0) the probabilities
suggest a uniform distribution of items in the network. We can analytically de-
duce their expected value, which matches the experimental results, with P (11)
as nc · nc = 0.04 and P (10) = P (01) as nc · (1 − nc ) = 0.16. However, as the
probability of message loss increases, we observe some changes in P (11), P (10)
and P (01). For the case of the fully connected network, the probabilities remain
stable, suggesting that the uniform distribution of items remains unaffected by
message loss. With the other topologies, however, as message loss increases the
number of gossip interactions between nodes that have item x also rises. Since
item x is representative of all items in the network, the graphs suggest that, as a
consequence of message loss, gossip partners have more items in common than
they would have if items were uniformly distributed. In other words, due to mes-
sage loss, a node is more likely to have items in common with a neighbor than
with another random node in the network. Hence, the topology of the under-
lying network now plays a role in defining the probability of dropping an item
after a gossip exchange, Pdrop .
4.4.2 Calculating Pdrop Based on Statistical Data

By now, we have established that the occurrence of message loss affects the
distribution of items in a network of gossiping nodes. While a network operating
under no message loss converges to a uniform distribution of items (i.e., each
node holds a random sample of items), when message loss occurs, the likelihood
of a node having the same items as its gossip partners increases. That is, the
sample held by a node is highly correlated to the samples held by the node’s
neighbors. As a result, under message loss, replicas of an item are more likely to
be found within the same neighborhood. It follows that the structure of neigh-
P(01) P(10)
0.3 0.3
fully connected fully connected
0.25 grid range 1 0.25 grid range 1
0.2 outdegree 4 0.2 outdegree 4
0.15 0.15
0.1 0.1
0.05 0.05
0 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Ploss Ploss
(a) (b)
P(11)
0.3
fully connected
0.25 grid range 1
0.2 outdegree 4
0.15
0.1
0.05
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Ploss
(c)
Figure 4.6: Probability that a gossip exchange occurs between a) a node that
does not have item x and a node that has the item, b) a node that has item x and
one that does not, and c) two nodes have item x, for different topologies.
borhoods plays an important role in determining the distribution of items, which,

in turn, determines the probability of an item found in a node’s cache to be also
found in the gossip partner’s cache, Pinx . Our calculation of Pdrop depends on
finding a value for Pinx .
For the case of a fully connected topology, where a node’s neighborhood is
the whole network, we can analytically derive Pinx . Having established earlier in
this section that uniform distribution of items is maintained in a fully connected
network (regardless of message loss), it follows that every node has the same
probability nc of having a given item. Therefore, for a node that has item x, the
chance of its gossip partner also having item x is nc . That is, Pinx is on average
c
n . For other topologies, the analysis will undoubtedly be more complex. We opt
for obtaining Pinx from statistical data collected from experiments. With Pinx ,
we can proceed to calculate Pdrop , obtaining the final building block of the model
needed for validation.
We can calculate Pinx from the probabilities P (11), P (10) and P (01) mea-
sured experimentally in the previous section:
P (11)
Pinx =
P (10) + P (11)
The left graph of Figure 4.7 shows the values for Pinx calculated for each ex-
periment using a different Ploss . Based on these values, we compute Pdrop using
(4.1), as seen in right graph of Figure 4.7. As expected, the calculated val-
ues show that, in the face of message loss, different topologies yield different
probabilities of dropping an item, with Pdrop dropping more harshly in the more
clustered topologies, which suffer more from neighboring nodes having similar
items as message loss increases.
0.5 0.9
fully connected 0.88
0.45 grid range 1 0.86
outdegree 4 0.84
0.4
0.82
0.35
Pdrop
0.8
Pinx
0.3 0.78
0.76
0.25 0.74 fully connected
0.2 0.72 grid range 1
0.7 outdegree 4
0.15 0.68
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Ploss Ploss
Figure 4.7: Probabilities Pinx (left) and Pdrop (right), for different topologies.

The experiments in this section simulate the case where a new item d is in-
troduced by one node in the network in which all caches are full and uniformly
populated by n = 500 items. Each experiment takes as input the topology of the
network to determine which pairs of nodes can gossip. In all cases, we use a net-
work of N = 2500 nodes arranged in either of the three topologies mentioned in
the previous section (a fully connected network, a square grid or a graph where
every node has 4 randomly chosen neighbors). In the experiments that follow,
after each gossip round, we measure the total number of occurrences of d in the
network (replication), and how many nodes in total have seen d (coverage).
Simulations with the shuffle protocol Each node in the network has a cache
size of c = 100, and sends s = 50 items when gossiping. In each round, every
node randomly selects one of its neighbors and shuffles. In order to make a fair
comparison with the simulations with the model, we let the nodes gossip for
1000 rounds with Ploss = 0, to ensure that none of the n = 500 initial items is
600 2500
number of nodes reached

500 2000
number of replicas
400
1500
300
1000
200
100 Ploss = 0.2 500 Ploss = 0.2

Ploss = 0.4 Ploss = 0.4
0 0
0 500 1000 1500 2000 0 500 1000 1500 2000
rounds rounds
100 Ploss=0.2 (diff) Ploss=0.2 (diff)
80 400
std. dev. std. dev.
60 300

40 200
20 100
400
80 std. dev std. dev
60 300
40 200
20 100
80 std. dev. 400 std. dev.
60 300
40 200
20 100
0 500 1000 1500 2000 0 500 1000 1500 2000

rounds rounds
Figure 4.8: Grid, range 1: Top: Replication (left) and coverage (right) of item
d for the shuffle. Bottom: difference between the shuffle and the model for
replication (left) and coverage (right).
lost in the startup period before initiating the measurements of the properties.
After this startup period of 1000 rounds, items are replicated and the replicas fill
the caches of all nodes. At round 1000, Ploss is set to the desired value and item
d is inserted into the network at a random location. From that moment on, we
track its replication and coverage.
Simulations with the model For the simulations with the model, n, c and s
are system parameters set to 500, 100 and 50, respectively. Instead of maintain-
ing a cache, each node in the network only maintains a variable that represents
whether it holds item d or not (state 1 or 0, respectively). Nodes update their
state in pairs according to the transition probabilities introduced before, see Fig-
ure 4.2. This mimics an actual exchange of items between a pair of nodes ac-
cording to the shuffle protocol. While in the protocol this results in both nodes
updating the contents of their caches, in a simulation using the analytical model
updating the state of a node refers to updating only one variable: whether the
node is in possession of item d or not. To sum up, we use transition probabilities
600 2500
number of nodes reached

500 2000
number of replicas
400
1500
300
1000
200
100 Ploss = 0.2 500 Ploss = 0.2

0 0
0 200 400 600 800 1000 0 200 400 600 800 1000
rounds rounds
100 Ploss=0.2 (diff) 500 Ploss=0.2 (diff)
60 300

40 200
20 100
100 Ploss=0.4 (diff) 500 Ploss=0.4 (diff)
80 std. dev 400 std. dev
60 300
40 200
20 100
Ploss=0.6 (diff) 500 Ploss=0.6 (diff)
80 300
60
40 200
20 100
0 200 400 600 800 1000 0 200 400 600 800 1000
rounds rounds
Figure 4.9: Topology with outdegree 4: Top: Replication (left) and coverage
(right) of item d for the shuffle. Bottom: difference between the shuffle and the
model for replication (left) and coverage (right).
to update the state of one variable. Since we do not need a startup time for the
simulations with the model, at round 0 we set the state of a random node to 1
(while all the others have state 0) and track the state of the nodes for the re-
mainder of the simulation. Pdrop is calculated using equation (4.1), as described
in Section 4.4.2.
Figs 4.8 and 4.9 show the behavior of the shuffle protocol (top row) and how
it compares to the analytical model (bottom row) in terms of replication (left)
and coverage (right) of d, for different values of Ploss . For each value of Ploss , 100
simulation runs (for both the shuffle protocol and its model) were executed. Due
to message loss, it is possible for item d to disappear after being introduced into
the network (usually in the first few rounds). As Ploss increases, this situation
is more likely. In the graphs, we only take into account the successful runs, i.e.,
where the item did not disappear, but spread.
The top rows of Figs 4.8 and 4.9 show the average and standard deviation of
the successful runs with the shuffle protocol. We compare this data with the av-
erage of the successful runs of the model and present the difference (in absolute
value) in the bottom rows of Figs. 4.8 and 4.9. We include the standard devia-
tion for the shuffle in the bottom row for comparison. It can be observed very
clearly that the difference between the results obtained from the shuffle and the
model fall well within the standard deviation of the shuffle results, confirming
the ability of the model in reproducing the average behavior of the protocol.
We note that in all successful runs (despite message loss), the network con-
verges to a situation in which there are roughly 500 copies of d, indicating that
after repeated execution of the protocol d receives a fair share of the storage
space in the network; 2500 · 100 cache slots divided between 500 items. Also, as
expected, due to random gossiping item d is eventually seen by all nodes in the
network, when coverage reaches 100%.
4.6 Conclusions
In this chapter, we have presented the analysis of a gossip-based information
dissemination protocol in the presence of transient communication failures. For
our analytical framework, we introduced a hybrid method to compute Pdrop that
combines both rigorous modelling of the protocol and statistical data sampling
from large-scale Monte Carlo simulations for different network topologies. To
be more precise, we derive analytical expressions for components of our frame-
work that are invariant with respect to the scenarios we want to model. Having
decomposed our model into its basic components, we identify the ones that are
affected by specific scenario configurations (in this case, message loss and topol-
ogy combinations). Finding analytical expressions for these components (in this
work, only Pinx ) would require modelling each specific scenario configuration,
reducing the applicability of the framework to those very specific configurations.
Instead, we isolate the framework from the scenario-specific component and use
statistical data sampling to obtain a value for it.
We opt for this combined approach since it allows us to build a framework
that captures the behavior of the gossip protocol without requiring us to incor-
porate scenario-specific elements into the model. The challenge in this approach
lies in being able to decompose the model into its minimal components, in such
a way that the ones which are dependent on particular scenarios (for which ex-
pressions that encompass all scenarios cannot be derived) can be isolated and
computed separately. In effect, we strive for a golden mean between high-level
models such as for epidemics showing only the emergent behavior and the low-
level models of the protocol that depend on particular implementation settings.
With respect to gossip-based dissemination, our study revealed that the uni-
form distribution of data has been a valid simplifying assumption for networks
with perfect, error-free communication media. However, for networks with lossy
communication channels, this assumption is valid only if every node can gossip
with any other node in the network, but not for other types of network graphs.
Comparison of Performance
Models 5
In Chapter 3, we developed an analytical model of information dissemination
for the shuffle protocol. With this model we now analyze how fast an item is
replicated through a network, and how fast the item covers the network. We
determine formulas that capture the dissemination of an item in a fully connected
network.
We also build several formal models using the probabilistic model checker
PRISM. Despite of the well-known state space explosion problem, we were able
to analyze a network of up to 15 nodes. In addition, we implement two types
of equations in MATLAB, a discrete one and its continuous alternative, as well as
the algorithm itself in the peer-to-peer network simulator PeerSim.
By comparing different modelling frameworks, we further explore the im-
pact of modelling choices, such as different scheduling policies and the notion
of rounds. The evaluation of distributed protocols, especially gossip protocols, is
difficult and a comparison of different evaluation techniques is greatly desired,
since the evaluation techniques vary a lot and are based on different assumptions.
The comparison of different results allowed us to discover hidden assumptions,
which clarifies the interpretation of the obtained results.
5.1 Introduction
With increasing use of distributed algorithms in applications such as dis-
tributed data bases or wireless sensor networks comes the need for methods to
analyze their behavior before deployment in a specific application. Most alterna-
tives to testing the distributed algorithm in a testbed rely to some extent on an
analysis of a model of the system; be it analysis by simulation or be it analysis by
model checking.
We compare different models used to analyze a distributed algorithm for shar-
ing data in a distributed network. The algorithm examined in this chapter is the
59
60 Chapter 5 — Comparison of Performance Models
shuffle algorithm, at the heart of the shuffle protocol, described in Chapter 2.

So far, we analyzed the spread of a distinguished data item, in a distributed
storage space maintained by the algorithm. An exchange of data between nodes
occurs pairwise; a node selects one node to communicate. If none of the two
nodes has the data item, neither will have it after they exchange data. If either
or both possess the data item, at least one of the nodes will possess it after the
exchange. The probability with which nodes possess the item after communica-
tion depends on the number of different data items, exchange message size and
local storage size. Moreover, this probability depends on whether both or only
one of the nodes possess the item before communication.
We now focus on two properties of the protocol: i) replication, i.e. the number
of nodes that hold the data item currently, and ii) coverage, i.e. the number of
nodes that have seen the data item in the past. We adopt the notion of rounds,
often used in the evaluation of gossip protocols (e.g. [94, 123, 177]), although it
is not a part of these protocols. In each round every node initiates one exchange
of data. This means that it might also be selected by another node as partner
in an information exchange. Note that the order in which nodes initiate data
exchange is arbitrary. This raises a natural question whether the order in which
nodes initiate data exchange can affect the coverage property.
We build formal models for the probabilistic model checker PRISM, imple-
menting different scheduling policies and notion of rounds. The first PRISM
model is nondeterministic and covers all possible schedulers, giving best- and
worst-case probabilities. The other PRISM models employ a random scheduler,
a round-robin scheduler, an “eager” scheduler that gives preference to nodes
that possess the item, and finally a “reluctant” scheduler that gives preference to
nodes without the item. The latter two PRISM models are used to determine if
implementable schedulers exist that get close to the best- and worst-case bounds.
All these models are based on transition probabilities from Chapter 3.
A distributed network often lacks the ability to enforce that all nodes progress
at an even pace. However, it is not uncommon (especially in an implementation
for simulation) to assume that communication occurs in rounds throughout the
network. Therefore, we additionally investigate the effect of omitting the notion
of round by assuming that all nodes progress with an even chance.
We then derive analytical expressions for replication and coverage as differ-
ential equations also based on the transition probabilities from Chapter 3, used
as the rates with which nodes exchange data items. These results are compared
with the other results of this chapter. To that end, we implemented two equations
of coverage in MATLAB – a discrete version as a difference equation and its con-
tinuous alternative in the form of a differential equation. Moreover, we compare
results from an implementation of the shuffle protocol running on a peer-to-peer
network simulator PeerSim [122]. Even for small networks, all models produced
different results, although these results are based on rigorous formalizations of
the algorithm (in the case of PeerSim, on an actual implementation). The com-
parison of the results of different formalisms allowed us to discover hidden as-
5.2. PRISM Model Checker 61
sumptions in these alternative modelling frameworks, which helps to clarify the

interpretation of the obtained results.
The network size in each model (meaning PRISM model, equation and imple-
mentation) is restricted to the least upper bound (supremum) that the respective
modelling framework could handle. Despite the well-known state space explo-
sion problem, we were able to specify a network of 10 nodes (for a round-based
PRISM model) and of 15 nodes (for a PRISM model without rounds) with PRISM.
However, this is rather restrictive when compared to the equational expressions
with a few thousand nodes. The results for the shuffle algorithm using the dif-
ferential and difference equations were obtained under the assumption that the
network is completely connected. The simulation and the PRISM models use the
same assumption, even though both would work without this assumption.
5.2 PRISM Model Checker

For formal modelling and analysis of the shuffle protocol we first use the
probabilistic model checker PRISM (version 3.2), developed at the University
of Oxford [112]. PRISM supports three types of probabilistic models: Markov
decision processes (MDP), discrete-time Markov chains (DTMC), and continuous-
time Markov chains (CTMC). In this chapter, we consider MDPs and DTMCs for
modelling.
The PRISM input language is a state-based language for describing probabilis-
tic state transition systems. In a PRISM model, a system is regarded as a set of
“communicating” modules. Each module consists of a set of guarded commands.
A command is composed of the guard (a predicate on the state variables) and a
probabilistic update relation (the probabilistic assignment to variables). Modules
can synchronize on shared variables or on common actions labels.
Once specified, the PRISM model checker constructs an internal representa-
tion of the system model – a Markov-style transition system for the composition
of specified modules – which can then be analyzed exhaustively relative to a
specified property using a suite of numerical algorithms. The specification lan-
guage is PCTL, a probabilistic extension of CTL, expressive enough to describe
many performance style properties. PRISM computes the probabilities of satis-
faction for DTMCs, and the best- and worst-case probabilities of satisfaction for
MDPs. The maximal and minimal probabilities for an MDP refer to the worst-
and best-case resolutions of the nondeterministic choice.
5.3 The Basic PRISM Model

All PRISM models described in this section share a similar structure. A node
initiates an exchange of data items with a random partner exactly once per
round, but may be contacted not at all, once, or multiple times. In addition to
the random partner selection, the shuffle protocol exhibits nondeterministic be-
havior in that the order in which the nodes initiate a gossip is not determined. In
Chapter 3, the performance of the protocol is measured according to the spread
of the data item d in the network. Thus, it is sufficient to use a boolean to indicate
the presence (as true) or the absence (as false) of d in the cache of node.
For each node i in the network we introduce one global boolean variable ni
(whether the node currently possesses data item d) and one module nodei. This
module has a local variable senti, which records whether a node has initiated
an exchange in the current round. A node i initiating the contact with a node j is
modelled as a local transition of module nodei. This transition selects the node
j with probability 1/(N − 1), where N is the number of nodes in the network. It
then sets the variable senti to true, and updates the state variables ni and nj,
using the probabilities of pairwise interaction model, as calculated in Section 3.4.
Thus, the entire communication process, including the exchange of data items, is
modelled by the probability that the participating nodes acquire the distinguished
data item d. Figure 5.11 (in the appendix) contains an example PRISM model for
3 nodes. Defining the variables ni and nj as global enables the initiating node
to update the state atomically, removes the need for synchronizing transitions
between modules, and thus simplifies the PRISM model considerably.
Once all modules have initiated a gossip in a round, they perform one syn-
chronous transition on the labelled action done to mark the end of the round.
This transition will reset the local variables senti back to false. Overall, each
round in the PRISM model has N + 1 transitions: for each of the N node one
transition to specify the initiated contact, and one transition to mark the end of
the round.
5.3.1 Coverage Property in PRISM

The coverage property, analyzed in Section 5.5.3, is expressed as the fraction
of nodes that have “seen” item d in the past. This property cannot be expressed
directly in PRISM; this tool allows for probabilistic specifications, i.e. it allows to
compute the probability that any given node has “seen” a node in the past. Since
the network is completely connected and all nodes are symmetric, it suffices to
compute the probability for one node. The probability that node i had data item
d in the past k rounds is expressed as follows:a
P=?[(true) U<=k*(N+1) (ni)] (5.1)
Note that (N + 1) is the number of steps in a round.

We now describe different PRISM models for networks of several nodes. All
models assume that the communication channels are reliable, provided by an
underlying layer that maintains the neighborhood information.
a For MDP models it is necessary to specify whether to compute the upper or lower bound prob-
ability. PRISM provides the PCTL operators Pmax=? and Pmin=? for this purpose. For simplicity we
only use P=? throughout this chapter.
5.3. The Basic PRISM Model 63
5.3.2 All Possible Schedules

The basic PRISM model as described above does not specify in which order
the nodes initiate communication. We first consider the nondeterministic PRISM
model which allows all schedulers for communication. The nondeterministic
PRISM model of the algorithm is specified as an MDP. PRISM then allows to
compute the minimum (worst-case) and maximum (best-case) probabilities of a
node to see item d over all possible schedulers, until round k.
0.30
0.25
0.20
Probability
0.15
0.10
MDP max
0.05 MDP min
Worst case t=9
Worst case t=10
0.00
1 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19 20 21
round 1 round 2
Figure 5.1: Worst- and best-case bounds for MDP model for the initial two
rounds, and schedules realizing the worst-case bounds at t = 9 and t = 10
Example 5.1. We consider a network with N = 10 nodes, cache size c = 100,

number of different data items n = 500, and message size s = 50. It is assumed
that one node, node 1, has the item d initially. Figure 5.1 depicts the maximal and
minimal coverage in detail for the first two rounds, i.e. we are checking
P=?[(true)U<=t (ni)]
rather than (5.1), for a node i other than node 1.

The results show that during the first two rounds the worst- and best-case prob-
abilities diverge. Note, that this is the worst-case probability for any distinguished
node, and not the average probability over all nodes. For example, right before the
second last step in round 1 (t = 8), the worst-case probability of node i will still be
0. This assumes that the other eight are scheduled first, then, at t = 9, node 1, and
finally, at t = 10, node i. All other nodes will have a positive probability at t = 8,
since they initiated communication before. The average coverage for these nodes
at this point is therefore strictly greater than 0, while the worst-case probability of
node i is still 0. There exists no schedule that realizes the lower bound of 0 for all
nodes simultaneously.
Furthermore, the worst-case bound in Figure 5.1 is not realized by a single worst-
case schedule, but by a combination of schedules. The previous paragraph discussed
the worst-case schedule for node i leading up to the second last step in round 1. The
worst-case probability for node i at the next step (t = 10), when all nodes but one
initiated communication, is realized by a schedule in which node i is the first to
initiate communication, followed by the other nodes, and then followed by the node
1 at t = 10 as the last node in this round. Figure 5.1 depicts the results for the
two different schedules in comparison to the worst-case bound for the MDP PRISM
model. Note also that in the second round, when these schedules are repeated in
round-robin fashion, neither of them realizes the worst-case bound.
Obviously, similar observations hold for the best-case bound, namely that it
is the best-case probability for any individual node, which will not be realized by
any single schedule. To that end, we presented the details for the steps within a
round, but in the remainder of this chapter we consider only the probabilities at
the end of each round.
5.3.3 Fair Random Scheduler

Rather than exploring all nondeterministic choices in PRISM model, we now
instantiate them with uniform distribution. In PRISM, this is simply achieved
by modelling the system as a DTMC (rather than as a MDP), which effectively
implements a fair random scheduler. On the level of PRISM modules, it is done
by using keyword probabilistic instead of nondeterministic.
In Figure 5.2, we can see that the curve of DTMC PRISM model represents
the average results and, as such, lies roughly in the middle of the area between
the upper and the lower bounds obtained from the MDP PRISM model.
5.3.4 Round-Robin Scheduler

The following PRISM model implements a round-robin scheduler to select the
next node that initiates communication. The nodes are arranged from node 1 to
node N , and node i + 1 can initiate communication only when node i has done
so. An exception to this rule is node 1 which can initiate the first communication
in each round.
The round-robin scheduler does not reflect a scheduler that would be imple-
mented by an actual network, since it requires a central scheduler with a global
0.8
0.6
Probability
0.4
0.2 Round-robin
MDP max
MDP min
DTMC
DTMC wo rounds
0
0 5 10 15 20 25 30
Rounds
Figure 5.2: Coverage for MDP and fair DTMC PRISM model.
view of the network. This would defeat the purpose of the distributed database
that executes a distributed shuffling algorithm. However, a round-robin sched-
uler would be suitable for any simulator that checks and selects the next node
in each round in the same order. In the context of the shuffle protocol, a round-
robin scheduler is nothing but a loop iterating over an array of node IDs. Analysis
of models that use a round-robin scheduler enables to estimate the range of cov-
erage which simulators employing a deterministic round-robin scheduler may
give.
Example 5.2. As before, we consider a network with ten nodes, cache size c =
100, number of different data items n = 500, and the size of the message s = 50.
Node 1 has the distinguished data item d initially. Figure 5.2 shows the results
for the nondeterministic, the fair random, and the round-robin scheduler. For all
these PRISM models the probability that a node has “seen” the item d in the past
will converge towards 1. Also as to be expected, the coverage for the fair random
scheduler and the round-robin scheduler lie in-between the upper and lower bounds
obtained from the nondeterministic PRISM model.
For the round-robin scheduler, coverage depends on when node 1 and node i are
scheduled. Selecting nodes in a predetermined order destroys the symmetry between
the nodes. The case when node 1 is scheduled first yields a higher coverage for the
other nodes compared to the round-robin scheduler that schedules this node last.
Also, if node i precedes in the round-robin order node 1, it has a higher coverage
than when it succeeds it.
Another interesting observation is that the average coverage over all round-
robin schedulers is not equal to the result of coverage of the fair scheduler. The
average coverage for round-robin schedulers is slightly improved with respect to
a fair random scheduler.
5.3.5 PRISM Model without Rounds

All previous PRISM models assumed that communication is organized in
rounds, and that each node initiates communication exactly once in each round.
This assumption is motivated by the assumption that all nodes progress at ap-
proximately the same speed. However, round-based PRISM models do require
either that all nodes have global knowledge of whether the other nodes initiated
exchange in the current round, or a strict underlying time synchronization al-
gorithm. In general, this cannot be assumed for the shuffle protocol. It is not
only difficult to accomplish in large distributed networks, it would also defeat
the purpose of a distributed system.
As an alternative we built a PRISM model without rounds. It selects the
next nodes independent of how often they have been selected before, with a
uniform distribution over all nodes. The model is similar to the fair DTMC PRISM
model. However, the model neither needs variable senti to record whether it
has initiated communication in this round, nor the synchronizing transition to
mark the end of a round.
Figure 5.2 also shows the result for this PRISM model. To compare it with the
round-based PRISM models we defined a “round” to be N steps of communicat-
ing nodes. It is one step less than for the other PRISM models, since the model
omits the transition at the end of a round. The results show that the coverage
in a PRISM model without rounds is smaller. However, they are clearly not only
within the bounds given by the MDP, but also within the range of the round-robin
schedulers. Even with a pure random scheduler, without rounds, it is guaranteed
that the coverage will converge to 1.
Note that this model without rounds does not capture more intricate timing
behavior such as clock drift or jitterb . To capture this would require to keep local
timing information. While possible, it does, even for networks of very moderate
size, quickly result in state spaces that are larger than the model checker can
handle.
b A clock jitter is a deviation of the locally measured time of the event with respect to the perfect
global time.
5.3.6 Unfair Schedulers

The previous results of this section show that the coverage for the fair ran-
dom and the round-robin schedulers are not close to the best- and worst-case
probabilities of the MDP PRISM model. It was already discussed in subsection
5.3.2 that the worst- and best-case bounds are for individual nodes only, and that
a single probabilistic scheduler that realizes these bounds for all nodes simulta-
neously may not exist. These bounds provide conservative outer bounds on the
coverage. This raises the question how conservative these bounds are.
To address this question, we studied the behavior of unfair schedulers that
have a global knowledge of the network. These include implementable sched-
ulers, and thus provide an inner approximation on the bounds on the coverage.
The first class are “eager” schedulers that give precedence to nodes that hold
the item. Nodes that do not hold the item are blocked from initiating communica-
tion as long as there is a node with an item that has not initiated communication,
yet. The other class are “reluctant” schedulers that give precedence to nodes that
do not have d in their cache. Both resulting PRISM models remain nondeter-
ministic. Thus, the upper and lower bounds for these classes will constrain the
coverage for any implementable scheduler in this class. The lower bound for the
“eager” schedulers thus provides an inner approximation on the upper bound for
1
0.745
0.9
0.74
0.735
0.8
0.73
7
0.7
0.9325
0.6
Probability
0.9275
0.5 20
0.4
0.3
MDP max
Eager max
0.2
Eager min
MDP min
0.1 Reluctant max
Reluctant min
0
0 5 10 15 20 25 30
Rounds
Figure 5.3: Coverage for unfair scheduler models, and results for MDP (gray) for
comparison. Inset figures show the maximal distance between the results.
MDP PRISM model, while the upper bound for the “reluctant” schedulers gives
an inner approximation on the lower bound. Note, that there might exist other
implementable schedulers that improve these bounds.
As shown in Figure 5.3, the respective upper and lower bounds for the “eager”
schedulers are numerically close to each other. Even though they do not coincide,
they cannot be distinguished in this figure. The same holds for the bounds on
the “reluctant” schedulers. Figure 5.3 also shows that the MDP bounds and the
inner approximation almost coincide as well. The difference is less than 0.005
in probability. This suggests that although the outer bounds might be somewhat
conservative, there exist implementable schedulers that approach these bounds
closely.
5.4 Simulation Model

We compare the result for coverage from the PRISM models with coverage
observed while running the shuffle protocol in a simulated environment. We
deploy the shuffle protocol in a round-based fashion, using the event-based sim-
ulator PeerSim [122]. The round-based mechanism of PeerSim allows collecting
the results of shuffling from cache at the end of each round. The experiments
simulate the case when a new item is introduced at one node in a network, while
all caches are full and uniformly populated with 500 items. As we know by now
(see Section 3.3), it can be achieved by letting the nodes gossip for 1000 rounds
before inserting the distinguished data item.
At the start, the simulator creates a randomly ordered list of nodes, provided
by a configuration file. Along with the network size and the number of items,
other parameters such as network topology and exchange buffer size are also
provided as input to the simulator. The network topology determines which
pairs of nodes can gossip. The experiments are performed on a fully connected
network of ten nodes. At the beginning of every gossip round, a scheduler in
the PeerSim simulator randomly shuffles an order of the nodes in the list. (This
effectively prevents the emergence of round-robin schedules.) Within a round of
the protocol simulation, all nodes proceed in a lock-step fashion, executing the
protocol in the order decided by the scheduler.
For the experiments we assumed a cache size of c = 100. In every round each
node randomly selects one of its neighbors and exchanges s = 50 items during
the interaction. After each gossip round, we measure how many nodes in total
have seen the distinguished item d at the end of each round.
Note that the simulator measures only whether a node holds d at the end
of the round, and ignores if the node acquired the item but lost it later in the
same round. The same holds for the equations that will be discussed later in
this chapter. In order to ensure that the results are comparable with the PRISM
results, we use in the remainder of the chapter, instead of (5.1), the following
5.5. Modelling with Differential Equations 69
0.8
0.6
Probability
0.4
0.2
Simulations (avg with std)
MDP max
MDP min
DTMC
0
0 5 10 15 20 25 30
Rounds
Figure 5.4: Comparison of results for PRISM models (gray), and with the results
of simulation with standard deviation.
PCTL property
P=?[(true) U<=k*(N+1) (ni & sent)] (5.2)
The boolean sent is the conjunction of the variables senti, and takes the value
true at the end of the round.
The simulation results in Figure 5.4 are the average of 1000 independent
runs. They show that the coverage obtained by simulation is within the MDP
bounds and close to the DTMC results. Given the uncertainty inherent within
simulation results, there is a close match with the PRISM model. They also show
that the MDP bounds are within the standard deviation of the simulation results.
5.5 Modelling with Differential Equations

We exploit the analytical model of information dissemination (from Chap-
ter 3) to perform a mathematical analysis of replication and coverage with regard
to the shuffle protocol. For the particular case of a network with full connectiv-
ity, we can find explicit expressions for the dissemination of a generic item d in
terms of the probabilities presented in Section 3.4. We construct two differential
equations that capture replication and coverage of item d from a round-based
perspective. The advantage of this approach is that we can determine the long-
term behavior of the system as a function of the parameters.
The characteristics of the replication and coverage are influenced by the
topology of the network. It is the topology of the network that will dictate the
likelihood of any two nodes to gossip. In order to model the properties of the
protocol, it is crucial that we know the probabilities of a node in a given state (0
or 1) to interact with another node in a given state. For this reason, we choose
to model the properties for a fully-connected network, where a node can gossip
with any other node in the network. This topology allows us to easily calculate
the probability of a node in state 1 to interact with a node in state 0 or to interact
with another node in state 1. Knowing this, we concentrate on applying the state
transition probabilities that we calculated in Section 3.4. The aim of this section
is to provide an example of how the transition probabilities can be used to model
the properties of dissemination for a specific topology.
5.5.1 Replication
One node introduces a new item d into the network at time t = 0, by placing
d into its cache. From that moment on, d is replicated as a consequence of
gossiping among nodes.
Let x(t) represent the percentage of nodes in the network that have d in their
cache at time t, where each gossip round takes one time unit. The variation in
x per time unit dxdt can be derived based on the probability that an item d will
replicate or disappear after an exchange between two nodes, where at least one
of the nodes has d in its cache:
dx
= [P (11|10) + P (11|01)] · (1 − x) · x − [P (10|11) + P (01|11)] · x · x (5.3)
dt
The first term represents duplication of d when a node that has d in its cache
initiates the shuffle and contacts a node that does not have the item. The second
term represents the opposite situation, when a node that does not have the item
d initiates a shuffle with a node that has d. The third and fourth term in the
equation represent the cases where both gossiping nodes have d in their cache,
and after the exchange only one copy of d remains. Substituting P (11|10) =
c−s
P (11|01) = sc n−s and P (10|11) = P (01|11) = sc n−s
n−c c−s
c , we obtain
dx s c−s n
=2· · · x · (1 − · x)
dt c n−s c
1
The solution of this equation, taking into account that x(0) = N because only
one of the N nodes in the network holds d initially, is
eαt
x(t) = n (5.4)
(N − + nc eαt
c)
c−s
where α denotes 2 sc n−s .
By imposing stationarity, i.e. dx c
dt = 0, we find the stationary solution n .
Hence, this calculation confirms the observation in Section 2.2.3 that the net-
work converges to a situation in which replication of d is nc .
500 items 1000 items

0.3 0.3
simulation (average) simulation (average)
model model
0.25 0.25
fraction of nodes
0.2 0.2
0.15 0.15
0.1 0.1
0.05 0.05
0 0
0 200 400 600 800 1000 0 200 400 600 800 1000
rounds rounds
2000 items
0.3
model
0.25
fraction of nodes
0.2
0.15
0.1
0.05
0
0 200 400 600 800 1000
rounds
Figure 5.5: Percentage of nodes in the network with a replica of item d in their
cache, for N = 2500, c = 100, s = 50, and n = 500, n = 1000 or n = 2000.
We evaluate the accuracy of x(t) as a representation of the fraction of nodes

carrying a replica of d, by running a series of experiments where N = 2500 nodes
execute the shuffle protocol, and their caches are monitored for the presence of
d. Unlike the experiments in Section 3.7, we assume full connectivity; that is,
for each node, all other nodes are within reach. After 1000 rounds, where items
are disseminated and replicated, a new item d is inserted at a random node, at
time t = 0. We track the number of replicas of d for the next 1000 rounds. The
experiment is repeated 100 times and the results are averaged. These simulation
results (average and standard deviation) for the protocol, together with x(t), are
presented in Figure 5.5. This figure shows that both curves have the same initial
increase in replicas after d has been inserted, and in all cases the steady state
reaches precisely the expected value nc predicted from the stationary solution.
Using the expression for replication, (5.4), we calculate the optimal size of
the exchange buffer. That is, we repeat the calculation from Section 3.6, but now
against x(t), to determine which size of the exchange buffer yields the fastest
convergence to the steady-state for both replication and coverage. That is, we
search for the s that maximizes the value of x(t). We first compute the derivative
of x(t) with respect to s (z(t, s)), and then derive the value of s that maximizes
x(t), by taking z(·, m) = ∂x
∂s |m = 0:
∂x 2ekt (cN − n)(cn + s(−2n + s))t

z(t, s) = = 2 ,
∂s (cN + (−1 + ekt ) n) (n − s)2
c−s
where k = 2 sc n−s . Let z(t, s) = 0. For t > 0, c · n = s(2n − s). Solving this
p
equation we get s = p n ± n(n − c). Taking into the account that s ≤ n, the only
solution is s = n − n(n − c). This also coincides with the optimal exchange
buffer size found in Section 3.6.
5.5.2 Coverage
We use the term coverage to denote the percentage of nodes in the network
that have seen item d from the moment it was introduced into the network. Let
y(t) represent the coverage of d at time t. The variation in coverage per time
unit, dy
dt , is determined by the fraction of nodes that have not seen d, 1 − y, and
that interact with nodes that have d in their cache, x. Let ∗ ∈ {0, 1}, then:
dy 1
= · (P (1∗|01) · P (∗1|∗1) · (1 − y) · x (5.5)
dt 2
+ P (∗1|10) · P (1∗|1∗) · x · (1 − y))
1
X 1
X
where P (1∗|01) = P (1i|01) and P (∗1|10) = P (i1|10)
i=0 i=0
The first term of the equation represents increased coverage due to nodes dis-
covering d after interacting with nodes that have d in their cache. This can occur
when a node initiates the exchange (P (1 ∗ |01)), or when the node is contacted
(P (∗1|10)). Each of these scenarios has a 50% chance of occurring. The second
part of these terms represents the case when a node discovers and does not give
away its copy of d to another node within the same round. This is because cov-
erage is measured only at the end of a gossip round, meaning that a node that
sees item d for the first time, and drops it in the same round, is considered not
to have seen item d yet.c Since nodes shuffle, on average, twice per round (once
c The reason for this is that in PeerSim a node in the shuffle protocol has an opportunity to read
from the lower-level cache only once every round.

when they initiate the shuffle and again if they are contacted by a neighbor), this
could occur under two scenarios:
i) the node acquired d by initiating an exchange with a node that had d (with
probability P (1∗|01)) and next lost its copy of d when shuffling with a node
that contacted it, or
ii) the node was first contacted by a node that sent a copy of d (P (∗1|10)) and
next initiated a shuffle and gave away its copy of d.
The value of the probability P (∗1| ∗ 1) can be calculated from the following
cases:
a) the gossip partner of the node does not have d, and:
i) the state of two nodes does not change after the gossip (with probability
P (01|01) · (1 − x)), and
ii) the gossip partner obtains a copy of d after the gossip (P (11|01) · (1 − x));
or
b) the gossip partner of the node has d, and:
i) two nodes have the same state after the exchange (P (11|11) · x), and
ii) the gossip partner loses its copy of d after the gossip (P (01|11) · x).
Hence,
P (∗1|∗1) = (P (01|01) + P (11|01))·(1 − x) + (P (11|11) + P (01|11))·x

= P¬select + Pselect · P¬drop ·(1 − x) + Pselect ·x + P¬drop ·P¬select
Due to the symmetry of both gossiping nodes, P (∗1|∗1) = P (1∗|1∗).

Substituting these probabilities into (5.5), we obtain

dy s s s c−s s s c−s
= · 1− + · 2− + − · x · (1 − y) · x
dt c c c n−s c c n−s
1
The solution of this equation, taking into account that y(0) = N, is
n n αt −β −λ
y(t) = 1 − (N − 1) · N β−1 N− + ·e ·e
c c
·( s − c−s )
·κ+ sc ·( sc − n−s
s
1 eαt
n c−s
)
where λ denotes c cα· nn−s N − n n αt
(N − c )+ c e , and β denotes c α·( n 2
) ,
c
c
wherein κ is sc · 1 − sc + sc · n−sc−s
2 − sc .
By imposing stationarity dydt = 0, we find the stationary solution 1, meaning
that eventually all nodes will see d. In order to evaluate how closely y(t) models
coverage, we use the traces from the simulations executed for Section 5.5.1. At
every round, the nodes that carry a replica of d are identified, and a record of
the nodes that have seen d since it was published is kept. Figure 5.6 presents the
coverage measured for three sets of experiments, each set with a different value
for n. As n increases, a newly inserted item requires more time to cover the
whole network. This is due to having more competition from other items to cre-
ate replicas in the limited space available, as was previously shown in Figure 5.5.

1 1
0.8 0.8
fraction of nodes
0.6 0.6
0.4 0.4
0.2 0.2
model model
0 0
0 200 400 600 800 1000 0 200 400 600 800 1000
rounds rounds
2000 items
1
0.8
fraction of nodes
0.6
0.4
0.2
model
0
0 200 400 600 800 1000
rounds
Figure 5.6: Percentage of nodes in the network that have already seen a replica
of item d, for N = 2500, c = 100, s = 50, and n = 500, n = 1000 or n = 2000.
However, as predicted by the stationary solution, in all cases the coverage even-
tually reaches 1. As shown in Figure 5.6, the solution y(t) models the behavior
observed in simulations in case of n = 2000 and n = 1000, falling nicely within
the standard deviation of the simulation results. However, for smaller n (i.e.
n = 500), the solution y(t) becomes less accurate, converging slower than the
simulation results. Next, we describe a more accurate, but also more compli-
cated version of the coverage model for arbitrary number of contacts between
nodes.
5.5.3 Coverage Model Revisited

The coverage model in Section 5.5.2 has an implicit assumption that a node
shuffles two times per round (once when it initiates the gossip and once when it
is contacted by another node). Although this is true on average, what actually
happens is that a node initiates a shuffle once per round (with some random
neighbor), but can be contacted an arbitrary number of times (≤ N − 1). In
Figure 5.7, it is depicted for k ≥ 1 which percentage of nodes, on average, shuffle
k times in a given round. The number of times a node shuffles in a round has
an impact on coverage. In this subsection, we revisit our coverage model to take
into account that there is a probability distribution for the number of times a
node is contacted.
0.4
percentage of nodes
0.35
0.3
fraction of nodes
0.25
0.2
0.15
0.1
0.05
0
0 2 4 6 8 10
number of shuffles
Figure 5.7: Distribution of nodes according to the number of shuffles they exe-
cute per round, for a fully connected network of N = 2500 nodes.
We compute the probability that a node is contacted i times by other nodes

in the network:
i (N −1)−i
N −1 1 N −2
C(i) = , i≤N −1
i N −1 N −1

Namely, there are N −1i ways to choose i nodes in a network of N − 1 nodes
that contact the given node. Those i nodes contact the given node with proba-
bility N1−1 each, and the remaining (N − 1) − i nodes do not contact the given
−2
node with probability N N −1 .
A node may discover d during any of its shuffles. However, coverage is only
measured at the end of a gossip round, meaning that a node that sees item d for
the first time and drops it in the same round is considered not to have seen item
d at all.d Consequently, coverage increases only under the following conditions:
i) a node that has not seen item d obtains a copy of d during one of the contacts,
and
ii) by the end of the round (after i exchanges), it still holds on to a copy of d.
d Again, the reason for this is that in PeerSim a node in the shuffle protocol has an opportunity to
read from the lower-level cache only once every round.

In order to model coverage, we need to express the probabilities Pget that a node
that does not have a copy of d gets this copy in a shuffle, and Plose that a node
that has a copy of d loses this copy in a shuffle.
Pget = P (1∗|0∗) = x·P (1∗|01) = x·(P (10|01) + P (11|01))
P¬lose = P (1∗|1∗) = x·P (1∗|11) + (1 − x)·P (1∗|10) (5.6)
= x·(P (10|11) + P (11|11)) + (1 − x)·(P (10|10) + P (11|10))
and
P¬get = 1 − Pget and Plose = 1 − P¬lose
The increase in coverage from one round to the next can be modelled by iden-
tifying all the possible cases where a node that has previously not seen item d
discovers it by the end of the round. Let Φi express the probability that a node
that does not hold d, does hold d after performing i shuffles. We have
i−1
X
Φi = (1 − Φm ) · Pget · (P¬lose )(i−m)−1 , i≥0 (5.7)
m=0
where Φ0 = 0. Namely, the expression

(1 − Φm ) · Pget · (P¬lose )(i−m)−1
captures the case where a node does not have the item at the end of m < i
shuffles (with probability 1 − Φm ), then discovers it in the mth shuffle (with
probability Pget ), and does not lose it in the remaining shuffles (with probability
(P¬lose )(i−m)−1 ).
In a given round, only the fraction 1 − y of nodes in the network that did not
yet see item d can contribute to an increase in coverage. Such a node is contacted
i ≥ 0 times in the round with probability C(i), and then performs i + 1 shuffles
in total. With probability Φi+1 the node will hold item d at the end of the round.
Thus, coverage can be modelled by the equation
Xk
dy
= (1 − y) · C(i) · Φi+1 (5.8)
dt i=0
where k is the maximum number of times that a node is contacted in a round,

i.e., k = N − 1. As before, y(0) = N1 , as initially only one of the N nodes holds d.
For a network of 2500 nodes with full connectivity, the probability of a node
being contacted more than four times in one round is negligible (less than 1%).
We therefore use the aforementioned coverage model with a limit of k = 4 to
estimate the coverage and compare with our simulation traces. The results can
be seen in Figure 5.8.
Unlike the results from Figure 5.6, the current model not only falls into the
standard deviation of the shuffle simulation results, but also closely reproduces
the curve of average values in all three cases (n = 500, n = 1000 and n = 2000).
5.6. Difference Equation Model 77

1 1
0.8 0.8
fraction of nodes
0.6 0.6
0.4 0.4
0.2 0.2
model model
0 0
0 200 400 600 800 1000 0 200 400 600 800 1000
rounds rounds
2000 items
1
0.8
fraction of nodes
0.6
0.4
0.2
model
0
0 200 400 600 800 1000
rounds
Figure 5.8: Percentage of nodes in the network that have already seen a replica
of item d, for N = 2500, c = 100, s = 50, and n = 500, n = 1000 or n = 2000.
5.6 Difference Equation Model

We compared the model-checking and simulation results with the results ob-
tained in Section 5.5. Modelling coverage as differential equation assumes that
the fraction of nodes that held the item d in the past is continually changing,
while it maintains at the same time the discrete notion of a round in equation
(5.8) by including the number of contacts for each round (see (5.7)). Since the
right-hand side of equation (5.8) is actually round-based, where a round takes
one unit of time, we expressed coverage alternatively as a difference equation
that includes this assumption explicitly. Differential equations (5.8) and (5.3)
hence become
k
!
X
y(t+1) = y(t) + (1 − y(t)) · C(i) · Φ(i + 1)
i=0
x(t+1) = x(t) + (P (11|10)+P (11|01)) (1 − x(t)) x(t)
− (P (10|11)+P (01|11)) x2 (t)
A problem with the difference equation is that it uses for all contacts the proba-
bilities at time t, while the probabilities do change as nodes initiate contacts, as
illustrated in Figure 5.1. Since these probabilities increase over time, we should
expect that this equational model underestimates the coverage.
0.8
0.6
Probability
0.4
0.2 MDP max

DTMC
MDP min
Differential Equation
Difference Equation
0
0 5 10 15 20 25 30
Rounds
Figure 5.9: Comparison of results for PRISM models (gray) and with the results
for the equational models.
Example 5.3. We compared the model-checking results with the results obtained
in [28]. We implemented both equational models in MATLAB. The solution of the
differential equation was obtained by the numerical integration using the explicit
Runge-Kutta (4, 5) formula (the Dormand-Prince pair) [88]. For this example, we
considered a network with ten nodes, cache size c = 100, number of different data
items n = 500, and the size of the message s = 50. As shown in Figure 5.9, the
5.7. Fractions and Probabilities 79
coverage for the difference and differential equations roughly coincide. They are
lower than the result for the fair DTMC PRISM model, and they are mostly within
the MDP bounds. One potential cause for this observation was mentioned in the
previous paragraph, namely that the difference equation does not account for the
fact that the probabilities change during a round. In addition both equations treat to
some extent fractions as probabilities and probabilities as fractions. In the following
subsection we investigate which effects this might have.
5.7 Fractions and Probabilities

In equational models, the initial condition is expressed as a fraction of net-
work that holds the observed item. Interpreting this fraction as a probability and
vice versa is only applicable for models with a sufficiently large number of nodes.
But even for large models these notions are fundamentally different. To illustrate
this, it is best to consider the initial state of the equational model of the shuffle
protocol. All models assume that initially exactly one node holds item d. This
means that the fraction of nodes that hold the item initially is N1 . Having one or
more nodes with d in their cache initially guarantees that the item will spread
through the network and that the coverage will eventually converge to 1. If in
contrast each node would hold d with probability 1/N , it would mean that with
probability (1 − 1/N )N no node would hold the data item initially. In that case,
the coverage would remain 0 forever. Even if the network size increases toward
infinity, we have that with probability
N
1 1
lim 1− =
N →∞ N e
the coverage will remain constant 0, where e is the Euler constant. Thus, the
overall coverage will converge to 1 − 1e .
Regarding the initial state, all previous models assume consistently that a
fraction of N1 holds the item. However, in later rounds both equational models
treat fraction as probabilities and vice versa, which should result in a similar,
maybe less pronounced, effect.
To investigate this influence we conducted another set of experiments. We
considered networks with 5, 10 and 15 nodes, where initially 1/5th of all nodes
hold item d. Since the differential equation model has the most fuzzy notion
of round, we compared it with the probabilistic PRISM model without rounds
presented in Section 5.3.5.
Figure 5.10 depicts the results of the comparison. For the differential equation
model we see that the network size has almost no impact. Only equations (5.6)
and (5.7) incorporate the network size N . It seems that the increasing number of
potential contacts in a round does not significantly increase the probability that a
node acquires or loses an item. For the PRISM model we observe that, while the
parameters s, n and c are the same for all models, and while all models assume
0.9
0.8
0.7
0.6
Probability
0.5
0.4
0.465
0.3
0.455 DTMC, N=5
0.2 DTMC, N=10
DTMC, N=15
0.445 ODE, N=5
0.1 4 ODE, N=10
ODE, N=15
0
0 5 10 15 20 25 30
Rounds
Figure 5.10: Comparison of the PRISM results with equational models, using the
same initial fraction.
that initially 1/5 of all nodes hold the data item d, it matters whether the network
has 5, 10, or 15 nodes. The network with 5 nodes realizes the highest coverage,
then the network with 10, and finally 15 nodes. As the number of nodes in
the PRISM model increases, and as probabilities behave more like fractions, the
coverage converges towards the results for the differential equation. This result
agrees with the theoretical result on the limiting behavior of the probabilistic
systems, such as epidemics (cf. [70, 136]).
5.8 Related Work

Several works have focused on model checking of gossip protocols, and stud-
ied scheduling in the the context of gossip protocols.
Fehnker and Gao [84] investigated the impact of modelling choices on the
analysis of message dissemination in flooding and probabilistic broadcast proto-
cols. In particular, the authors explored the effect of message collisions, lossy
channels, and random delay in the gossip frequency of nodes in addition to a
nondeterministic execution order. For a comparison, the protocols were mod-
elled in PRISM and implemented in MATLAB for Monte Carlo simulations.
Kwiatkowska et al. [138] presented the case study of a gossip peer sampling
5.9. Conclusion 81
service [123] with PRISM. The authors considered two properties of the protocol,
the longest path between nodes and the time for the network graph to become
connected. To clarify the discrepancy between values of the best- and worst-case
behaviors, the authors studied the problem of scheduling. Due to the state space
explosion problem, their analysis is restricted to networks with three and four
nodes.
In [46], a gossip protocol has been modelled as a DTMC, and implemented
in PeerSim. The author analyzed the evolution of the indegree distribution of
nodes under two different schedulers. The first one is a random scheduler, in
which each node has the same probability to perform a shuffle step. In the second
scheduler, one round ends when all nodes have initiated exactly one interaction.
This study has shown how schedulers may affect the complexity of the model
and produce different results.
Differential equations have been previously used, for example in [99, 133,
147], to model probabilistic protocols for large-scale distributed systems. More-
over, they have been widely used in modelling epidemics (see Section 1.4.1).
5.9 Conclusion
This chapter considered a distributed algorithm for sharing data in a dis-
tributed network to explore the different choices for PRISM models. It then
compared the PRISM results with the result of alternative approaches; two equa-
tional models and a network simulator. While all models are based on a rigorous
formalization of the algorithm – or, in the case of the simulator, on an actual
implementation – the results did not match, except the models from Section 5.5
showing large-scale behavior. A careful analysis helped to find limitations of
PRISM, and uncovered hidden assumptions in the alternative modelling frame-
works.
The PRISM models were designed with the state explosion problem in mind.
We abstracted away the part of the algorithm that deals with pairwise commu-
nication, and modelled it as an atomic probabilistic transition. This abstraction
was made possible by the analysis in Chapter 3 of transition probabilities of the
shuffle protocol. The PRISM model covered only which nodes initiate in what or-
der contact with other nodes to exchange information. This abstraction allowed
us to model networks with up to 15 nodes, and instances of model checking took
up to 10 minutes on an Intel Core Duo CPU, 1.2 GHz, with 1 GB of RAM.
Under the assumption of full connectivity, we modelled the properties of the
dissemination of a generic item with transition probabilities. Each property is ex-
pressed as a formula which is shown to display the same behavior as the average
behavior of the protocol.
This work uses PRISM to study the different scheduling policies, and to de-
termine best- and worst-case bounds. However, the specification logic PCTL did
not let us define the exact equivalent of coverage. PCTL provides the capability
to study state probabilities, while coverage is best defined as an expected value

over CTL state formulae. Fortunately, this limitation was only relevant for MDP
models; since all nodes symmetric and due to the assumption that the network
is completely connected, in the DTMC PRISM models PCTL alternative turns into
the exact equivalent of the coverage property.
Another limitation of PRISM and PCTL is that it does not allow to query state
probabilities after k iterations. These state probabilities could have been used
to compare also other measures, such as replication. PRISM provides the transi-
tion matrices to compute replication in, for example, MATLAB, but unfortunately,
these sparse matrices were too large to manipulate. While this demonstrates the
efficiency of the MTBDD encoding used in PRISM [112], it would be an even
more useful tool if it could give state probabilities after k steps, if only for DTMCs.
A contributing factor to the difference between the PRISM and equational
models is that the latter treat fractions as probabilities and vice versa. We were
able to illustrate that this kind of models are more accurate the larger the number
of nodes is. PRISM distinguishes between probabilities and fractions inherently,
since it captures fractions explicitly in the state of the model.
While we were able to specify networks with up to 15 nodes in PRISM, it
should be noted that the alternative models can be used for a few thousand
nodes in equational models. The PRISM analysis of a relatively small network
proved useful nevertheless, since it helped to uncover simplifying assumptions
that allow these models to deal with much larger networks, and thus with the
interpretation of the obtained results.
It would be interesting to extend presented results with equational models,
assuming other network topologies than a fully-connected network. It is more
difficult to represent models of a non-uniform network because of limitations
of the PRISM specification language. For some networks, like a static torus, a
self-implemented script (e.g., in Python) can assist one in generation of a PRISM
specification.
5.10. Appendix: Nondeterministic PRISM Model 83
5.10 Appendix: Nondeterministic PRISM Model

nondeterministic
const int c = 100;

const int s = 50;
const int n = 500;
const int N = 3;
global n1: bool init true;

global n2, n3: bool init false;
formula p0000 = 1;
formula p0101 = (c-s)/c;
formula p1001 = (s/c)*(n-c)/(n-s);
formula p1101 = (s/c)*(c-s)/(n-s);
formula p0110 = (s/c)*(n-c)/(n-s);
formula p1010 = (c-s)/c;
formula p1110 = (s/c)*(c-s)/(n-s);
formula p0111 = (s/c)*((c-s)/c)*((n-c)/(n-s));
formula p1011 = (s/c)*((c-s)/c)*((n-c)/(n-s));
formula p1111 = 1-2*(s/c)*((c-s)/c)*((n-c)/(n-s));
module node1
sent1: bool init false;
[] !sent1 & !n1 -> 1/(2)*(!n2?p0000:0): (n1’=false) & (n2’=false) & (sent1’=true)
+ 1/(2)*( n2?p0101:0): (n1’=false) & (n2’=true) & (sent1’=true)
+ 1/(2)*( n2?p1001:0): (n1’=true) & (n2’=false) & (sent1’=true)
+ 1/(2)*( n2?p1101:0): (n1’=true) & (n2’=true) & (sent1’=true)
+ 1/(2)*(!n3?p0000:0): (n1’=false) & (n3’=false) & (sent1’=true)
+ 1/(2)*( n3?p1101:0): (n1’=true) & (n3’=true) & (sent1’=true);
[] !sent1 & n1 -> 1/(2)*(!n2?p0110:0): (n1’=false) & (n2’=true) & (sent1’=true)
+ 1/(2)*(!n2?p1010:0): (n1’=true) & (n2’=false) & (sent1’=true)
+ 1/(2)*(!n2?p1110:0): (n1’=true) & (n2’=true) & (sent1’=true)
+ 1/(2)*( n2?p1111:0): (n1’=true) & (n2’=true) & (sent1’=true)
+ 1/(2)*(!n3?p0110:0): (n1’=false) & (n3’=true) & (sent1’=true)
+ 1/(2)*(!n3?p1010:0): (n1’=true) & (n3’=false) & (sent1’=true)
+ 1/(2)*(!n3?p1110:0): (n1’=true) & (n3’=true) & (sent1’=true)
+ 1/(2)*( n3?p1111:0): (n1’=true) & (n3’=true) & (sent1’=true);
[done] sent1 -> sent1’=false;
endmodule
module node2 = node1 [sent1=sent2,sent2=sent1,sent3=sent3,n1=n2,n2=n1,n3=n3] endmodule

module node3 = node1 [sent1=sent3,sent2=sent1,sent3=sent2,n1=n3,n2=n1,n3=n2] endmodule
Figure 5.11: Nondeterministic PRISM model of the shuffle algorithm for 3 nodes.
Mean-field Framework
6
We employ mean-field approximation for an analytical evaluation of gossip
protocols. Nodes in the network are represented by small identical stochastic
processes. Joining all nodes would result in an enormous stochastic process. If
the number of nodes goes to infinity, however, mean-field analysis allows us to
replace this intractably large stochastic process by a small deterministic process.
This process approximates the behavior of very large gossip networks, and can
be evaluated using simple matrix-vector multiplications.
Traditionally, the mean-field method has been applied to the protocols with
unidirectional communication. This chapter introduces a mean-field framework
extended for the evaluation of push-pull protocols. We describe the necessary
mean-field theory, and devise a simple analytical model for gossip-based infor-
mation dissemination as an illustrative example. This example demonstrates the
mean-field theory for unidirectional communication models (called push-based
or pull-based in terms of gossip protocols). We further show how pairwise node
interaction can be modelled in the mean-field framework in the case of the shuf-
fle protocol with the help of the state transition model from Chapter 3. Moreover,
we present an analysis of a basic version of the gossip-based clock synchroniza-
tion protocol GTP.
6.1 Introduction
We consider large-scale gossip networks where a large number of nodes inter-
act. As mentioned before, the study of the emergent behavior of gossip protocols
demands the consideration of large-scale networks [123]. Clearly, the large size
of the network poses a serious challenge on modelling and analysis of a system
of interacting nodes, even if the behavior of a single node is relatively simple.
As we have seen in the previous chapter, the analysis of gossip protocols with
standard verification tools is hard – it is, for example, beyond the capabilities of
current probabilistic model-checking tools, as it is shown in the previous chapter.
Moreover, when a large number of nodes interact in a connected environment,
85
86 Chapter 6 — Mean-field Framework
various phenomena emerge that cannot be explained in terms of the behavior of

a single node. We are therefore interested in going from a detailed local model
at node level to an abstract global model of the system.
In this work, we resort to so-called mean-field analysis. The stochastic process
representing the modelled system converges to a deterministic process if the size
of the network tends to infinity (see, e.g., [48]). This convergence result provides
an approximation for networks consisting of large numbers of interacting nodes
that are symmetrically defined. The system is modelled on an abstract level as
the distribution of nodes in the set of possible states; the mean-field method then
allows us to observe the evolution of this distribution over time.
The notion of “mean-field” is often used in the literature, with different mean-
ings. The mean-field concept was first introduced in statistical mechanics (e.g.,
[36]). It has been used in the context of Markov chain models of systems like
plasma and dense gases where the strength of the interaction between particles
is inversely proportional to the size of the system. A particle is seen as under a
collective force generated by the other particles in a continuous time and space
setting. Subsequently, the mean-field approximation enjoyed the attention of the
neural networks community (e.g., [154]), and was used in the area of chemical
reaction networks (e.g., [135, 146]). Furthermore, a number of deterministic
continuous models including mean-field approximation have been considered in
the theory of epidemics; for more details see, e.g., [6, 7, 148, 166].
In the area of communication networks, mean-field approximation has been
applied in various forms to a variety of case studies, e.g., HTTP flows [11], TCP
connections [9, 10, 12, 171], bandwidth sharing [134], transportation networks
[2], swarm robotic systems [142], reputation determination [48], queueing net-
works [71, 126, 178] and Internet congestion control [124].
We show that the mean-field method is well-suited for a performance eval-
uation of gossip protocols. Applying the method to these protocols is a natural
choice, since gossip protocols are meant to operate in very large networks, con-
sisting of interacting nodes that are symmetrically defined. In this chapter, we
present a modelling framework for gossip protocols with pull, push and push-
pulla communication models.
Related Work
In the context of gossip protocols, mean-field modelling has been considered
for the analysis of the age of gossip [59], epidemic routing [185]. Several pre-
vious papers on gossip protocols have used a notion of mean-value and infinite
limit for the asymptotic analysis of algebraic gossip [72, 169] and gossip-based
membership protocols [46, 47].
Harwood and Ohrimenko [103] have proposed a model to analyse the effect
of churn in a gossip network for the case of probabilistic broadcast. The authors
a that is, both nodes gossiping with each other share their local data
6.2. Gossiping Time Protocol 87
have derived equations of a message throughput for different message buffer

size, message arrival rate, and number of message sources. The presented model
relies on exponential distribution for the rate of nodes arrivals and departures.
Stojanovic et al. [169] analyzed and compared delay performance of network
coding and cooperative diversity in a single-hop wireless network. The authors
performed an asymptotic analysis (for the number of nodes N → ∞) of the
expected delay associated with the broadcasting of a file consisting of a certain
number of packets.
We refer an interested reader to a survey on clock synchronization, for ex-
ample, [156], and recent work on gossip-based clock synchronization protocol
[90]. Our case study, the GTP protocol, assumes the presence of at least one clock
source. In the absence of external clock sources, there is a recent protocol, de-
veloped in [32]. In [108], the combination of the Uppaal model checker and the
proof assistant Isabelle has been used for analysis of a wireless synchronization
protocol, developed by the company Chess.
6.2 Gossiping Time Protocol

Protocols based on epidemic and gossip concepts have found various practical
applications [132], including nontraditional gossip applications [66], such as
gossip-based clock synchronization. The Gossiping Time Protocol (GTP) [118,
119] is a self-managing gossip time synchronization protocol for peer-to-peer
networks.
The protocol operates in a network of nodes, each equipped with a local
clock, and assumes the presence of at least one node with accurate and robust
time in the network. Time is disseminated throughout the network by letting
nodes periodically gossip their clock samples. That is, each node periodically se-
lects (initiates a gossip with) a random peer from the network to exchange time
information with. The initiating period is determined by a value of the gossip
delay parameter, which is the current delay between subsequent gossip interac-
tions. The nodes subsequently exchange their local settings such that afterwards
the node with the worse-quality time has adopted the higher-quality time of the
other node. The protocol assumes a presence of the peer-sampling service [123],
which allows a node to contact a uniformly randomly selected alive node.
In basic GTP, the quality of the time sample at a node is based on the distance
from the time source to the node (hop count metric), that is, the number of
nodes on the synchronization path from the node to the time source. The time
source has hop count equal to 0. Completely unsynchronized nodes have a hop
count ∞. A gossiping node rejects the time sample if the hop count of its gossip
partner is not smaller than its own. Furthermore, a node adopts a time sample if
it has not been synchronized for a long time. That is, if the difference between
the last update and the current time is larger than a timeout period, then a node
accepts a time sample even though it may degrade its time quality (with respect
to the hop count metric). Concisely, if the node decides to accept the sample, it
synchronizes its clock, and updates the values of the local variables. Namely, the
node records the value of current time as the time of the last clock update, and
sets the hop count to the value of the gossip partner hop count incremented by
one. The GTP protocol parameters described above are stored as the following
local variables, according to [118]: a gossip delay as GOSSIPING DELAY, a time
of the last clock update as LAST UPDATE, a hop count as TS DISTANCE, a timeout
as STANDALONE PERIOD .
Alternatively, each node may decide to adapt the rate at which it initiates a
timestamp exchange (gossip frequency) based on its local settings. For instance,
the better synchronized the node is, the lower the gossip frequency it may as-
sume. In doing so, the gossip frequency gradually decreases when the network
is synchronized and stable. Note that dynamic gossip frequency is beyond basic
GTP, and is a simple optimization idea for the gradual version of GTP. The moti-
vation behind it is that even though the timestamp exchange can be implemented
to be inexpensive, it still consumes resources.
We show how a mean-field framework can be applied to a nontraditional
application of gossip protocols, on the example of the basic GTP. That is, nodes
execute basic GTP based on an immediate clock adjustment model, and change
gossip frequencies depending on a gossip delay. For the original protocol and its
design details, we refer to [118, 119].
6.3 Mean-field Modelling and Convergence

This section introduces the theory needed to apply mean-field results to gossip
protocols. We stay close to the presentation in [48], but change notation when
appropriate and simplify matters if possible in the gossip context.
6.3.1 Modelling and State Space

A discrete-time Markov chain (DTMC) is a stochastic process {Y (t) | t ∈ N}
that takes values in a countable state space S. A DTMC obeys the Markov prop-
erty, that is, the next state j ∈ S is independent of the past, given the present
state il ∈ S:
Pr{Y (t + 1) = j | Y (0) = i1 , . . . , Y (t) = it } = Pr{Y (t + 1) = j | Y (t) = it }.
We consider a system of N ∈ N interacting objects that are identically defined.

The object with index n ∈ {1, . . . , N } is represented by the discrete-time stochas-
tic process
{XnN (t) | t ∈ N}
which takes values in the set S = {0, . . . , K − 1} where K = |S| is the number of
different states.
6.3. Mean-field Modelling and Convergence 89
Example 6.1. In a gossip network, a node is represented by an interacting object.

As a running example we consider a simple information dissemination protocol. A
piece of information, e.g., the current time, is forwarded through the net. A node
can be in one of two states: either it already has the information (state 0) or it is
not yet informed (state 1). Hence, the state space for a node is S = {0, 1} with
|S| = K = 2. Let m0 be a fraction of informed nodes, and pN (m0 ) the probability
of moving from state 1 to state 0. Figure 6.1 shows a graphical representation of the
state-transition diagram describing such a node; the possible transitions and their
probabilities will be explained later in this section.
1 1 − pN (m0 )
pN (m0 )
0 1
Figure 6.1: A single node transition
The complete system is composed of the N objects and is, consequently, also
described by a discrete-time stochastic process:

Y N (t) = X1N (t), . . . , XN
N
(t) .
Its state space is S N which has |S|N = K N elements. For the mean-field conver-
gence result we assume that we cannot distinguish objects that are in the same
state. It then suffices to keep track of the fraction of objects in each state. These
fractions are collected in another stochastic process
M N (t) = (M0 (t), . . . , MK−1 (t))
called the occupancy measure. Its elements are defined as
N
1 X
MiN (t) = 1{XnN (t) = i}, i ∈ S,
N n=1
where 1{XnN (t) = i} is equal to 1 if XnN (t) = i, and 0 otherwise. Its state space
N
SM ⊂ RK has

N
S = K + N − 1
M
K −1
elements (the number of ways to distribute N objects over the K states they can
be in). One state from this state space is denoted
N
m = (m0 , m1 , . . . , mK−1 ) ∈ SM ,
where mi is the fraction of nodes in state i.
Example 6.2. For the information dissemination example, the state space of the
occupancy measure is

N k k
SM = ,1 − k ∈ {0, . . . , N } .
N N
Its size is
N 2+N −1
SM = = N + 1.
2−1
Normally, the choice of the occupancy measure depends on the local param-
eters of a node, which, in its turn, depends on the application of gossip.
6.3.2 Local Transition Probabilities

The evolution of the system of interacting objects is described by the local
transition probabilities of each object. The next state of any object not only
depends on the current state of the object but also on the current occupancy
measure m:
N
Pi,j (m) = Pr{XnN (t + 1) = j | XnN (t) = i, M N (t) = m}, i, j ∈ S, m ∈ SM
N
(6.1)
These probabilities are the same for all objects. They are gathered into the tran-
sition probability matrix P N (m). These local transition probabilities determine
the unique transition probability matrix for the global system Y N (t), which is a
DTMC because its next state (= occupancy measure) depends only on the current
state.
In contrast to Example 6.1, the exchange of information between nodes can
be bidirectional as, for instance, in case of push-pull based gossip protocols. In
these gossip protocols, both interacting nodes update their local states depending
on both of their current states.
Formally, two communicating objects (n1 , n2 ) undergo a state transition ac-
cording to
P(iN1 ,j1 ),(i2 ,j2 ) (m) = Pr{XnN1 (t + 1) = j1 , XnN2 (t + 1) = j2 |

XnN1 (t) = i1 , XnN2 (t) = i2 , M N (t) = m} (6.2)
N
where i1 , i2 , j1 , j2 ∈ S, and m ∈ SM . Analogous observations regarding pair-
wise interaction of objects in the mean-field models have been made in [41].
In the pairwise interaction model, each transition probability P(iN1 ,j1 ),(i2 ,j2 ) (m)
takes into account current occupancy measures Mi1 (t) and Mi2 (t) of both states
i1 and i2 . Thus, at every time step t the value of the probability P(iN1 ,j1 ),(i2 ,j2 ) (m)
is recomputed based on the current value of the occupancy measure Mi2 (t).
In the next sections, we model pairwise node interaction and pairwise state
update of the nodes. Using the pairwise interaction model (6.2) in the mean-
field framework, we analyze the push-pull information dissemination protocol in
Section 6.4 and a basic version of GTP in Section 6.5. In this section, however, we
keep to the pull model of a node, introduced in Example 6.1, and the transition
equation (6.1).
Example 6.3. A node can only move from being uninformed (state 1) to being
informed (state 0). Afterwards it stays in state 0 forever, that is, it never forgets.
Suppose that in each time step a node A initiates a gossip interaction with proba-
bility g. It randomly chooses a partner node B among the N − 1 other nodes. If
B is already informed and A is not, A moves to state 0, so that we model a simple
pull protocol. Note that m0 is the fraction of informed nodes in the system and
m1 = 1 − m0 the fraction of uninformed nodes. The total probability for moving
from state 1 to state 0 equals
m0 · N
pN (m0 ) = P1,0
N
((m0 , m1 )) = g · .
N −1
Here, m0 · N is the number of informed nodes and m0 · N/(N − 1) is the probability
that a node chooses an informed node out of the N − 1 possible nodes (it does not
pick itself) as gossip partner. The complete probability matrix is then given by

1 0
P N ((m0 , m1 )) = .
pN (m0 ) 1 − pN (m0 )
For the global system, the probability to move from m0 informed nodes to m′0
informed nodes, for m′0 ≥ m0 , equals

m1 · N (m′ −m0 )N m′ N
′ pN (m0 ) 0 1 − pN (m0 ) 1 ,
(m0 − m0 ) · N
where m1 = 1 − m0 , m′1 = 1 − m′0 . This binomial expression is composed of the

number of possibilities to choose exactly the “missing” (m′0 − m0 ) · N objects out of
the m1 · N uninformed nodes; these then all have to take the transition to state 0,
and all other m′1 · N nodes remain in state 1.
Consider now the occupancy measure M N (t) of the system at a given time
t ∈ N. Recall that M N (t) is a random variable. For a given initial occupancy
measure mN N
0 , there are two ways to determine the distribution of M (t): first,
we can calculate the transient distribution analytically at time t, requiring t
N
vector-matrix multiplications with a vector of size |SM |. Second, we can employ
discrete-event simulation to estimate the distribution. Often only discrete-event
simulation is possible since, for large N , the size of the state space makes the
analytical computation of the transient probabilities practically infeasible. But
even discrete-event simulation of this large DTMC is expensive.
Suppose the number of local states of nodes is s. Then the time complexity
per step in the mean-field method is s2 , i.e. the cost of matrix-vector multi-
plication. For our simple example, the time complexity is only 2 · 2 = 4 per
time step. Already a single Monte Carlo simulation of the system with only 100
nodes is much more expensive. Moreover, for a reasonable confidence interval
one needs to repeat the simulations for at least 100 times.b For more complex
systems, in general, to observe the emergent behavior may require several thou-
sands of nodes and thousands of runs of the simulations, whereas the mean-field
method requires only one run (independent of the number of nodes), providing
the limiting behavior of the system. Roughly speaking, the complexity is s2 for
the mean-field method compared to f (N ) · r of the Monte Carlo simulations,
where f (N ) is a cost for simulating one run, which is at least linear with respect
to the number of nodes N , and r is the number of runs. Note that f (N ) can
be large; for example, the Monte Carlo simulations of the shuffle protocol for a
single pairwise communication costs c · log c (see Section 3.8 for details), where
c is the local storage size of a node.
1
0.8
Pr{M0N (10) ≤ m0 }
0.6
0.4
N=100, DTMC
simulation
N=1000, DTMC and
0.2 simulation
N=10000, DTMC and
simulation
mean eld
0
0 0.01 0.02 0.03 0.04 0.05
m0
Figure 6.2: Distribution of M0N (10) for g = 0.1 and M N (0) = (0.01, 0.99)
Example 6.4. Figure 6.2 shows the analytically computed distribution of the frac-
tion of informed nodes at time t = 10, for gossip probability g = 0.1 and initial
occupancy measure
M N (0) = (0.01, 0.99).
The analytical results in the figure are obtained using mean-field analysis (the ver-
tical line at 0.025), and by using matrix-vector multiplications with a vector size
b In general, it is difficult to estimate how many times Monte Carlo simulations have to be re-
peated to obtain a reasonable confidence interval, especially if there are rare events with impact on
the system behavior.
N
|SM |, as described above. Note that the distribution is “more deterministic” for
larger N .
We also simulated this simple dissemination protocol in a round-based fashion
similar to simulations in PeerSim [122]. In one round, which equals one time
step, each uninformed node gossips with probability g and picks a random peer.
If this peer is already informed, the number of informed nodes for the next round
is increased by one. Using 10000 independent runs for each curve, the resulting
distributions for M0N (10) are also shown in Figure 6.2.
6.3.3 Convergence Result

At this point, the so-called mean-field convergence result applies. It captures
the limiting behavior of the complete system if the number of objects N goes to
infinity and so provides an approximation for the occupancy measure for large
N . The requirement is that for all local states i, j ∈ S, all m ∈ RK and N → ∞:
N
Pi,j (m) converges uniformlyc in m to some Pi,j (m), which is a continuous
function of m.
If this requirement is satisfied, the occupancy measure converges almost surely

to a deterministic limit. This means that in case N → ∞, for each local state i
the fraction MiN (t) of objects in state i at time t is known with probability one.
Theorem 6.1 (cf. [81]). Fix the initial occupancy measure to be identical for all
N ∈ N:
M N (0) = µ(0).
Define the limit of the local probability matrix:
P (m) = lim P N (m), m ∈ RK .

N →∞
Define the deterministic process
µ(t + 1) = µ(t) · P (µ(t)).
Then for any t ∈ N,
lim M N (t) = µ(t), with probability 1,

N →∞
that is, µ(t) is the deterministic limit occupancy measure for N → ∞.

For large N we can now approximate the stochastic process for the occupancy
measure by this deterministic process.
c A sequence f of real valued functions converges uniformly with limit f if for every ε > 0 there
N
exists a natural number n such that for all x and all N ≥ n we have |fN (x) − f (x)| < ε.
Example 6.5. The limit of the probability to move from state 1 to state 0 is
m0 · N
p(m0 ) = lim g · = g · m0 ,
N →∞ N −1
which is continuous in m0 . The requirement for the application of the mean-field
convergence result is thus satisfied. If we set
µ(0) = (0.01, 0.99)
and g = 0.1, the deterministic limit for time t = 10 is
µ(10) = (0.0256, 0.9744),
which is computed by ten matrix-vector multiplications. It is indicated by the vertical
line for m0 in Figure 6.2.
6.3.4 A Methodology for the Mean-field Analysis of Gossip

Protocols
We summarize how mean-field analysis can be used for the performance eval-
uation of gossip protocols. Our methodology consists of the following steps:
Step 1 – System description The specification of a system helps to obtain not
only a better (more modular) description, but also a clear understanding
and an abstract view of the system. In general, it is hard to give a full speci-
fication of a system or protocol under study. Such a study is usually done on
a simplified system model of the actual protocol: one has to decide which
characteristics of the protocol should be studied, and which parameters of
the protocol should be modelled in order to study these characteristics. In
order to simplify the system model, assumptions should be made. These
assumptions should be supported by experimental study.
Step 2 – Identification of local states and transitions This step requires iden-
tification of the set S of local states of a node. The states should reflect
all relevant situations a node can be in. Transitions between local states
usually occur because of gossip interactions.
Step 3 – Transition probabilities The (local) transition probabilities depend on

the global state of the gossip network model. The probabilities have to be
investigated thoroughly. A node might also behave intrinsically in a prob-
abilistic way. At the end of this step stands a directive of how to compute
the transition probability matrix depending on the current global state.
Step 4 – Mean-field convergence requirements Only if the local transition prob-

abilities converge appropriately for N → ∞, can we successfully apply the
mean-field convergence theorem.
6.4. A Mean-field Model for the Shuffle Protocol 95
Step 5 – Mean-field limit Finally, we can compute the mean-field limit for our
model using Theorem 7.1. With the obtained results we can test and com-
pare different designs.
6.4 A Mean-field Model for the Shuffle Protocol

We apply mean-field theory to the shuffle protocol, using the pairwise inter-
action model of Chapter 3. The specification of the shuffle protocol in Chapter 2
and [94] corresponds to Step 1 in our mean-field framework. Steps 2 and 3 are
covered by the following subsections (6.4.1–6.4.3). Finally, Section 6.4.4 covers
Steps 4 and 5.
We demonstrate how the analytical model, presented in Chapter 2 for pair-
wise node interaction, can be used for our mean-field framework.
6.4.1 State Space

We use the abstraction of the shuffle protocol, the pairwise interaction model,
and analyze the spread of a distinguished item in the network. The state of a
node in the shuffle protocol is defined by a pair (g, d). The first component g
denotes the gossip delay. It defines the speed with which nodes communicate
with each other. When the gossip delay reaches zero a node initiates a shuffle.
The second element d is a two-valued integer (bit) indicating whether a node
possesses the item.
The state space of a node is
S = {0, . . . , Gmax } × {0, 1},
and the size of the state space is
|S| = (Gmax + 1) · 2,
where Gmax is the maximal gossip delay. Note that we consider a discrete time
model, that is, the system proceeds in discrete time steps t ∈ N.

The behavior of a node is based on its state and the current occupancy mea-
sure m. The expression (g, d | g > 0) denotes any state where g > 0 and d is
chosen arbitrarily from the set {0, 1}. The expression m(g,d|g≤g′ ) is an example
for the abbreviation of a sum of occupancy fractions, calculated as:
′ ′
g
X g
X
m(g,d|g≤g′ ) = m(g,0) + m(g,1) .
g=0 g=0
The transition probability

N
P(g,0|g>0),(g−1,0) (m)
denotes the transition probability from the state (g, 0) to state (g − 1, 0) for any
g > 0.
Figure 6.3 depicts the two-dimensional diagram for a single node transition.
According to our model in Section 6.4.1, each possible state of the node is defined
by its gossip delay and by the presence of the item at the node. The x-axis
in Figure 6.3 corresponds to the value of the gossip delay g, and the y-axis to
the value of d. For all transitions represented by the arrow, we calculate their
respective state transition probabilities.
Active Nodes
Periodically, a node A initiates the interaction with a random peer B. The
period is governed by the gossip delay g, and the length is equal to Gmax . Thus,
whenever g of a node A reaches 0, the node becomes active and initiates the
interaction with a peer B, selected at random among N − 1 nodes (excluding A
itself).
According to the shuffle protocol, two gossiping nodes cannot become in-
volved in another interaction until the current contact is finished. We address
this assumption by introducing the notion of a collision. Namely, once active
node A randomly selected B as its gossip partner, the probability that no colli-
sion occurs, nocN (m), ensures that other active nodes select neither A nor B.
1 ···
...
...
0 ···
g
0 1 Gmax
Figure 6.3: The state transitions of a node

Thus, the probability that no collision occurs is equal to:

m(0,d) ·N −1
N −3
nocN (m) = (6.3)
N −1
That is, all active nodes m(0,d) ·N excluding A choose one of the remaining N − 3
nodes as their gossip partner, excluding A, B and the node itself.
We calculate the transition probabilities
N
P(0,d),(G ′ (m)
max ,d )
for all combinations of d, d′ ∈ {0, 1}. An active node is in one of two possible
states:
(a) the node holds the item (i.e. state (0, 0)), or
(b) the node does not have the item (i.e. state (0, 1)).
In either case, the node will either not have the item after the shuffle or obtain
the item during the exchange. That is, the new state after the interaction is either
(Gmax , 0) or (Gmax , 1).
If an active node does not hold the item before the contact, the value of d
remains 0 in the following three cases:
(1) collision occurred while talking to a passive node,
(2) the chosen peer is active itself, or
(3) independent of the peer state, a node did not obtain the item after the con-
tact (by the probabilities P (00|00) and P (01|01)).
1
X
N m(g′ ,i|g′ >0) · N
P(0,0),(G max ,0)
(m) = · (1 − nocN (m))
i=0
N −1
X1
m(0,i) · N − 1 + i
+ (6.4)
i=0
N −1
1
X m(g′ ,i|g′ >0) · N
+ · P (0i|0i) · nocN (m)
i=0
N −1
First, an active node selects a passive node with probability
m(g′ ,i|g′ >0) · N/(N − 1),
but the interaction fails due to a collision that occurs with probability 1−nocN (m).
Second, the active node chooses another active node with probability
(m(0,0|g′ >0) · (N − 1) + m(0,1|g′ >0) · N )/(N − 1),

and thus, the contact fails. Third, the active node chooses a passive node with
probability
m(g′ ,i|g′ >0) · N/(N − 1)
and successfully communicates with probability nocN (m). The node does not
get the item with probability P (00|00), if the peer did not have the item as well,
and with P (01|01), if the peer had the item.
The item is obtained after the interaction, if a passive peer holds the item
before the interaction, according to the probabilities P (11|01) and P (10|01):
1
N m(g′ ,1|g′ >0) · N X
P(0,0),(G max ,1)
(m) = · P (1j|01) · nocN (m) (6.5)
N −1 j=0
That is, an active node chooses a passive peer that had the item with probability
m(g′ ,1|g′ >0) · N/(N − 1),
and after the communication without collision nocN (m), the active node obtains
the item with probability P (11|01) + P (10|01).
If the active node holds the item before interaction, the value of d stays as 1,
because
(1) a collision occurred while talking to passive nodes,
(2) the chosen peer is an active node, or
(3) the item stays in the local cache, independent of whether the peer possesses
the item before or after the shuffle, determined by the probabilities P (10|10),
P (11|10), P (10|11) and P (11|11).
Thus, the probability of this transition is
1
X
N m(g′ ,i|g>0) · N
P(0,1),(G max ,1)
(m) = · (1 − nocN (m))
i=0
N −1
X1
m(0,i) · N
+ (6.6)
i=0
N −1
X1 1
m(g′ ,i|g′ >0) · N − i X
+ · P (1j|1i) · nocN (m)
i=0
N −1 j=0
However, the item is given away by an active node after the exchange with prob-
ability
1
X
N m(g′ ,i|g′ >0) · N
P(0,1),(G max ,0)
(m) = · P (01|1i) · nocN (m) (6.7)
i=0
N −1
since, either
(1) the active node chooses a passive peer that did not have the item with the
probability m(g′ ,0|g′ >0) · N/(N − 1), successfully communicated with proba-
bility nocN (m), and passed the item to the peer with probability P (01|10);
or,
(2) the active node chooses a passive peer that had the item with probability
m(g′ ,1|g′ >0) ·N/(N −1), successfully communicated with probability nocN (m),
and dropped its copy of the item with probability P (01|11).
Passive Nodes
A passive node, that is, a node with g > 0, can be contacted by an active
peer, resulting in an update of its state. In all cases, the gossip delay is decreased
by one, counting down to the next gossip initiation. Similar to active nodes, we
distinguish four main transitions, depending on whether the item is at the node
before or after gossiping.
The probability that a passive node does not obtain the item after the gossip
is equal to the following sum:
1
X
N m(0,i) · N
P(g,0|g>0),(g−1,0) (m) = · (1 − nocN (m))
i=0
N −1
1
!
X m(0,i) · N
+ 1− (6.8)
i=0
N −1
1
X m(0,i) · N
+ · P (i0|i0) · nocN (m)
i=0
N −1
The sum consists of the three components:

(1) the probability m(0,i) · N/(N − 1) that the node is chosen by an active node,
but a collision occurred with probability 1 − nocN (m);
(2) the probability that a node is not chosen as a peer by any active node;
(3) the probability m(0,i) · N/(N − 1) to be chosen by an active node, and af-
ter successful communication (with probability nocN (m)), not to obtain the
item with probability P (00|00), if the active node did not have the item, and
with P (10|10), otherwise.
A passive node obtains the item with probability
1
N m(0,1) · N X
P(g,0|g>0),(g−1,1) (m) = · P (j1|10) · nocN (m) (6.9)
N −1 j=0
that is, the probability that the node is chosen by an active peer that holds the
item m(0,1) · N/(N − 1), successfully communicates according to the probability
nocN (m), and the node obtains the item with probability P (01|10) + P (11|10).
The remaining two probabilities cover the case when a passive node has the
item before the interaction. It can either keep the item or give it away as the
result of an interaction with an active peer with the following probabilities.
The probability that a passive node keeps the item after the interaction is
computed as the sum
1
X
N m(0,i) · N
P(g,1|g>0),(g−1,1) (m) = · (1 − nocN (m))
i=0
N −1
1
!
X m(0,i) · N
+ 1− (6.10)
i=0
N −1
X1 1
m(0,i) · N X
+ · P (j1|i1) · nocN (m)
i=0
N − 1 j=0
First, the node is chosen by an active node with probability m(0,i) · N/(N − 1),
but a collision occurred during the interaction with probability 1 − nocN (m). The
second summand expresses the probability that the passive node is not selected
by any active node as a peer. Third, after the successful communication with an
active peer with probability
m(0,i) · N/(N − 1) · nocN (m),
the passive node keeps the item with probability P (01|i1) + P (11|i1), where i
indicates whether the active peer has the item or not.
The probability that a node gives away its copy of the item is calculated as:
1
X
N m(0,i) · N
P(g,1|g>0),(g−1,0) (m) = · P (10|i1) · nocN (m) (6.11)
i=0
N −1
Namely, after the successful communication with an active peer with probability
m(0,i) · N/(N − 1) · nocN (m),
the passive node drops its copy of the item with probability P (10|01), if the peer
did not have the item, and P (10|11), otherwise.
6.4.3 Coverage Property of the Protocol

The model, described in Section 6.4.1-6.4.2, is sufficient to perform the anal-
ysis of the shuffle protocol based on the replication property. Recall that replica-
tion expresses the fraction of nodes that possess a copy of the data item at a given
time. Other properties of the shuffle protocol include coverage, i.e., the fraction
of nodes that have seen the item within the given period.
In order to analyze the performance of the protocol according to coverage,
we need to extend the state space with an additional parameter, o ∈ {0, 1}. The
parameter o = 0, if a node has not held the item previously, and o = 1, otherwise.
Once the item is acquired in one of the exchanges, o stays equal to 1. Thus, the
state of a node is a triple (g, d, o). The state space of the extended model is
S = {0, . . . , Gmax } × {0, 1} × {0, 1}
with the size |S| = (Gmax + 1) · 4.

The transition probabilities for the replication model have to be adapted ac-
cording to the new state space. Clearly, if a node holds the item (d = 1), the node
becomes informed about the item (o = 1). Moreover, once a node has obtained
the item, it cannot ‘forget’ even after losing the copy during one of the exchanges:
N N
P(g,d,1|g>0),(g−1,d ′ ,0) (m) = P(0,d,1),(G ′
max ,d ,0)
(m) = 0
Thus, several transition probabilities are computed in a similar way. The transi-
tion probabilities for active nodes from (6.4), (6.7), and (6.6) remain:
N N
P(0,0,o),(G max ,0,o)
(m) = P(0,0),(G max ,0)
(m), o ∈ {0, 1}
N N
P(0,1,1),(G max ,0,1)
(m) = P(0,1),(G max ,0)
(m)
N N
P(0,1,1),(G max ,1,1)
(m) = P(0,1),(G max ,1)
(m)
All four probabilities cover the transitions, where a value of the parameter o
does not change. Likewise, the corresponding transition probabilities for passive
nodes from (6.8), (6.11), and (6.10) remain:
N N
P(g,0,o|g>0),(g−1,0,o) (m) = P(g,0|g>0),(g−1,0) (m), o ∈ {0, 1}
N N
P(g,1,1|g>0),(g−1,1,1) (m) = P(g,1|g>0),(g−1,1) (m)
N N
P(g,1,1|g>0),(g−1,0,1) (m) = P(g,1|g>0),(g−1,0) (m)
We distinguish the transitions in which a node acquires the item for the first time,
and in which a node that held the item previously obtains it again. Although we
differentiate these cases, the transition probabilities are equal to the probability
that a node obtains a copy of the item of (6.9) and (6.5):
N N N
P(g,0,0|g>0),(g−1,1,1) (m) = P(g,0,1|g>0),(g−1,1,1) (m) = P(g,0|g>0),(g−1,1) (m)
N N N
P(0,0,0),(G max ,1,1)
(m) = P(0,0,1),(G max ,1,1)
(m) = P(0,0),(G max ,1)
(m)
Note that in all probabilities the occupancy measure m(g,0) for any g is now
treated as the sum m(g,0,0) + m(g,0,1) , and m(g,1) for any g simply becomes
m(g,1,1) . The same applies to the probability that no collision occurs:
(m(0,0,0) +m(0,0,1) +m(0,1,1) )·N −1
N −3
nocN (m) = .
N −1
That is, given two nodes interacting with each other, all other active nodes choose
peers different from the gossiping pair.
6.4.4 Mean-field Limits

In the limit, the value of the probability nocN (m) for the replication model is
m(0,d) ·N −1
N −3
N
noc(m) = lim noc (m) = lim = e−2·m(0,d) ,
N →∞ N →∞ N −1
where m(0,d) denotes the sum m(0,0) + m(0,1) , and for the coverage model is:
m(0,d,o) ·N −1
N −3
N
noc(m) = lim noc (m) = lim = e−2·m(0,d,o) ,
N →∞ N →∞ N −1
where
m(0,d,o) = m(0,0,0) + m(0,0,1) + m(0,1,1) .
We note that in the transition probabilities for active and passive nodes calculated
above, factors like N/(N − 1) converge to 1 if N → ∞.
6.4.5 Comparison with Previous Results

We compare the results of our mean-field model with the results of replication
and coverage observed while running the actual protocol in a large-scale deploy-
ment. In case of the shuffle protocol, we simulate the network of 2500 nodes on
a single workstation, using a Java-based implementation. Each node has local
storage size c = 100, and exchange message size s = 50. All nodes are randomly
divided into 10 different groups with the different values of gossip delay. The
period between two consecutive contact initiations is fixed and set to 10. The
nodes from the group with g = 0 are active nodes that initiate simultaneously an
exchange with the peers chosen uniformly at random. If a node contacts two gos-
siping nodes, the interaction between all three nodes fails. Initially a new item
is introduced at one node in a network, in which n = 500 items are uniformly
distributed over the local storage of all nodes. After each step, we measure the
total number of copies of the distinguished item in the network, and the number
of nodes that have seen this item. Each simulation curve in Figure 6.4 represents
the average calculated over 500 runs.
Relating our mean-field model to these settings, we set the fraction of nodes
that possess the item initially to 1/2500. The maximum gossip delay Gmax = 9.
All nodes have a gossip delay uniformly distributed between 0 and Gmax . Each
discrete time step, we record the values of the occupancy measure m.
Figure 6.4(a) shows the evolution of the number of copies of the distinguished
item in the network over time. For the mean-field model we have multiplied the
fraction of nodes with d = 1 by 2500 to compare with the simulation curve. Both
curves of simulation and mean-field model are quite close, with a small difference
between steps 650 and 1300. Nevertheless, the mean-field curve falls nicely
within the standard deviation of the simulation results. Both curves approach
the same stable state nc = 500 items after 1300 steps.
Figure 6.4(b) shows the evolution of the total number of nodes that have held
the distinguished item at some moment in time. In the mean-field model, we sum
the fraction of nodes that are in state (g, d, 1) for all g and d. Like replication,
in the beginning both curves grow at the same rate for around 650 steps, after
500 1
400 0.8
number of replicas
fraction of nodes
300 0.6
200 0.4
100 0.2
mean-field mean-field
simulation simulation
0 0
0 500 1000 1500 2000 2500 3000 0 500 1000 1500 2000 2500 3000
time time
(a) Replication (b) Coverage
Figure 6.4: Comparison with simulation results for a network of 2500 nodes.
which there is a slight difference that accumulates over time. Still, the mean-
field curve falls within the standard deviation of the simulation results, and both
curves reach the stable state 1 by 1500 steps.
The difference between the simulation and mean-field curves in the network
of 2500 nodes arises from the fact that the distribution of the items slightly fluc-
tuates around the average 500 (average deviation is 20). As a consequence the
uniform distribution assumed in our model is not perfect in practice. However,
when considering larger networks (i.e. 25000 nodes) the simulations come very
close to the behavior predicted by the mean-field analysis; see Figure 6.5.
In Chapter 5, the pairwise interaction model is used for the differential equa-
tions for replication and coverage, presented in Section 5.5. Thus, we compare
our mean-field model with the differential equations for both replication (see
500 1
400 0.8
number of replicas
fraction of nodes
300 0.6
200 0.4
100 0.2
mean-field mean-field
simulation simulation
0 0
0 500 1000 1500 2000 2500 3000 0 500 1000 1500 2000 2500 3000
time time
Figure 6.5: Comparison with simulation results for a network of 25000 nodes.
Figure 6.6(a)) and coverage (see Figure 6.6(b)). Due to the deterministic nature
of both models, we are interested to see how different the results are. We im-
plemented the differential equations in MATLAB; the solutions of the differential
equations are obtained by numerical integration. The number of possible con-
tacts by a node per round is limited to 4. To take into account the fact that the
equational models disregard the possibility of collisions, we increase the maxi-
mal gossip delay for the mean-field analysis to Gmax = 200. Intuitively, there
are N/Gmax nodes that have the same gossip delay g, and nodes with g = 0 talk
simultaneously. By increasing Gmax and initially distributing all nodes uniformly
at random into states (g, d, o) with different g, fewer nodes become active at the
same time, which leads to more sequential initiation of a gossip by each node. For
Gmax = 200 and N = 2500, 2500/200 = 12.5 nodes have the same g compared to
2500/10 = 250 nodes, thus having a smaller chance that an active node contacts
another active node or a communicating pair than in the latter case. Formally,
since the equational models do not include the notion of collisions, we have to
maximize the probability of no collisions noc(m) in the mean-field model. Maxi-
mizing the value of the probability of no collision e−2·m(0,d,o) implies minimizing
m(0,d,o) since e > 1, which can be achieved by increasing Gmax . The maximal
gossip delay Gmax = 200 keeps the probability of no collisions in the mean-field
model fairly small, i.e. 1/e25 , while preserving the relatively small size of the
state space, i.e. 201 × 2 × 2 = 804.
We assume that one round of the differential equation corresponds to 200
steps in the mean-field model, because in case of the equational models, every
node has to initiate a gossip once per round.
Figure 6.6(a) shows the comparison of both models for replication, in terms of
the evolution of the fraction of nodes that have seen the distinguished item over
time. The differential equation curve overlaps nicely with the mean-field curve.
Figure 6.6(b) presents the comparison of two models for the coverage property.
0.2 1
0.18
0.16 0.8
fraction of nodes
fraction of nodes
0.14
0.12 0.6
0.1
0.08 0.4
0.06
0.04 0.2
0.02 mean-field ODE
ODE mean-field
0 0
0 50 100 150 200 250 300 0 50 100 150 200 250 300
rounds rounds
Figure 6.6: Comparison of the deterministic models.

6.5. A Mean-field Model for Basic GTP 105
Both curves are very close to each other, with a tiny difference starting after
around 60 rounds; both curves settle to the same stable state after 80 rounds.
The difference is explained by the assumed value for the number of maximum
contacts by a node per round, and a small collision probability 1−1/e25 compared
to 0 of the differential equation.
6.5 A Mean-field Model for Basic GTP

A detailed description of basic GTP can be found in Section 6.2 and in [118,
119]. This corresponds to Step 1 in our methodology. Steps 2 and 3 are ac-
complished in the following three subsections (6.5.1–6.5.2). Section 6.5.3 cor-
responds to Steps 4 and 5. We compare the obtained mean-field model against
the emulation results from [118] in Section 6.5.4. We perform an analysis of
properties of basic GTP in Section 6.5.5, based on the mean-field model.
6.5.1 State Space

The state of a node in a basic GTP network is given by a triple (g, l, h). The first
component g denotes the gossip delay, which defines the frequency of initiating
a contact. When the value of g becomes equal to zero, a node initiates a gossip
interaction. The second component l represents a counter for the last update. In
GTP, the time of the last clock update is stored and, if necessary, compared to
the current time. If the difference exceeds the standalone period, an update is
enforced. We replace it with a counter which is set to the length of the standalone
period at every update. Reaching zero, a clock update is enforced at the next
interaction. Finally, h is the number of hops the timing information has travelled
from the time source.
Let Gmax be the maximal gossip delay, and let L be the standalone period.
We introduce H to be the maximal hop count. A node in a state with h = H has
a hop count of at least H. A node with h = ∞ is said to be unsynchronized. The
state space of a single node then is
S = {0, . . . , Gmax } × {0, . . . , L} × {0, . . . , H, ∞},
which is of size |S| = (Gmax + 1)(L + 1)(H + 2).
Gossip Delay
Though basic GTP has a fixed gossip delay, we design the model in such a
way that it allows for the gossip delay to vary, depending on the hop count of a
node. We assume that there is a function
G : {0, . . . , H, ∞} 7→ {0, . . . , Gmax }
that gives the gossip delay G(h) for any hop count h. As described in Section 6.2,
the varying gossip delay is an optimization feature of gradual GTP.

The behavior of a single node is determined by its state and the current oc-
cupancy measure m. In the sequel, we again use a kind of pattern matching
notation: for example, (g, l, h | g > 0) denotes any state where g > 0 while
l and h are chosen arbitrarily from their respective value sets. The expression
m(g,l,h|h<H) is an example for the abbreviation of a sum of occupancy fractions,
defined by XX X
m(g,l,h|h<H) = m(g,l,h) .
g l h<H
Time Sources
We begin with the description of the behavior of a time source, that is, a node
in a state with h = 0. Time sources never update their clock, hence, component l
has no meaning and we always set it to L. If the gossip delay is larger than zero
(g > 0), we just decrement it by one. If it is equal to zero, the gossip delay is
reset to G(0).
N
P(g,l,0|g>0),(g−1,L,0) (m) = 1
N
P(0,l,0),(G(0),L,0) (m) = 1.
As one can see, time sources act independently of their environment (as depicted
in Figure 6.7).
Active Nodes
If the gossip delay g of a node A is equal to zero, it becomes active and
initiates a gossip interaction with a peer B randomly chosen from the remaining
N − 1 nodes. In this interaction, the clock of A might get updated. In GTP, an
interaction is discarded if during its course there has been another interaction
leading to an update of the clock. In the model we require that for each node
only one interaction can be active, otherwise we say that there is a collision.
An update can take place only if the interaction prevails, that is, no collision
occurs. After A has chosen a suitable peer B, the probability nocN (m) that
there is no collision is given by the probability that all other active nodes select
peers different from A and B. The probability that a node chooses neither A
nor B (given that it does not try to interact with itself) is (N − 3)/(N − 1). We
1 1 1
(0, L, 0) (1, L, 0) ··· (G(0), L, 0)
1
Figure 6.7: Transitions of time sources.

consequently have m(0,l,h) ·N −1

N N −3
noc (m) = .
N −1
We further have to distinguish between nodes with an enforced update (l = 0)

and those without (l 6= 0). If an update is enforced, the clock will be updated
as long as the peer is synchronized, having a hop count h′ different from ∞. If
the update is optional, the clock value is only changed if this does not increase
the hop count, that is, if h′ < h. In either case, the new state after a successful
update is (G(h′ + 1), L, h′ + 1). The probability to select a passive peer with hop
count h′ is
m(g′ ,l′ ,h′ |g>0) · N/(N − 1)
and so the probability of a successful update is

m(g′ ,l′ ,h′ |g′ >0) · N
N
P(0,0,h|h>0),(G(h ′ ),L,h′ +1) (m) = · nocN (m), ∀h′ < ∞,
N −1
m(g′ ,l′ ,h′ |g′ >0) · N
N
P(0,l,h|l>0,h>0),(G(h ′ ),L,h′ +1) (m) = · nocN (m), ∀h′ < h.
N −1
Clearly, the successful update takes place if no collision occurred during the in-
teraction (with the probability nocN (m)). If h = ∞ and the active node chooses
a passive peer with hop count H −1 or H, the probability of the successful update
is:
X1
N m(g′ ,l′ ,H−i |g′ >0) · N
P(0,l,∞),(G(H),L,H) (m) = · nocN (m),
i=0
N − 1
since the interaction occurred with a synchronized peer.

The case remains where the interaction does not lead to an update of the
clock. This can happen if
(1) a collision occurs when gossiping with a passive node,
(2) an active node is selected as peer, also leading to a collision, or
(3) the interaction has prevailed but the peer cannot provide a suitable hop
count; that is, the peer is an unsynchronized node if l = 0, and h′ ≥ h if
l > 0.
We again distinguish enforced and optional updates and get the probabilities:
N m(g′ ,l′ ,h′ |g′ >0) · N

P(0,0,h|h>0),(G(h),0,h) (m) = · (1 − nocN (m))
N −1
m(0,l′ ,h′ ) · N m(g′ ,l′ ,∞|g′ >0) · N
+ + · nocN (m),
N −1 N −1
N m(g′ ,l′ ,h′ |g′ >0) · N
P(0,l,h|l>0,h>0),(G(h),l−1,h) (m) = · (1 − nocN (m))
N −1
m(0,l′ ,h′ ) · N m(g′ ,l′ ,h′ |g′ >0,h′ ≥h) · N
+ + · nocN (m).
N −1 N −1
We treat the case where an active node with hop count H contacts another
node with hop count H as an unsuccessful update, since in our model H sub-
sumes all hop counts h ≥ H of the synchronized nodes of the actual protocol.
The calculation of the probability of the successful update in this case requires
knowledge of the actual distribution of nodes with these hop counts, which we
do not know a priori.
Passive Nodes
A passive node with g > 0 has to be contacted by an active peer with hop
count h′ to be able to update its hop count to h′ + 1. This happens with prob-
ability m(0,l,h′ ) · N/(N − 1). The gossip delay is decreased by one in all cases,
shortening the time until the next gossip initiation. Following the same line of
argumentation as for active nodes, the probabilities for successful interactions
are
m(0,l′ ,h′ ) · N
N
P(g,0,h|g>0,h>0),(G(h ′ +1),L,h′ +1) (m) = · nocN (m), ∀h′ < ∞,
N −1
m(0,l′ ,h′ ) · N
N
P(g,l,h|g>0,l>0,h>0),(G(h ′ +1),L,h′ +1) (m) = · nocN (m), ∀h′ < h.
N −1
That is, the probability of being selected by an active peer with hop count h′ is
m(0,l′ ,h′ ) · N/(N − 1), and the probability that no collision occurred during the
interaction is nocN (m).
If h = ∞ and the passive node is chosen by an active peer with hop count
H − 1 or H, the probability of the successful update is:
1
X
N m(0,l′ ,H−i) · N
P(g,l,∞|g>0),(G(H),L,H) (m) = · nocN (m),
i=0
N −1
because the passive node is contacted by a synchronized peer.

The probability of not updating the clock is again composed of three terms:
(1) the probability of having a collision during the interaction with an active
node,
(2) the probability of not being chosen as a peer by any active node, and
(3) the probability of having an interaction with an active peer that does not
have a suitable hop count: either an unsychnronised node if l = 0, or with
hop count h′ ≥ h if l > 0.
Hence,
N m(0,l′ ,h′ ) · N
P(g,0,h|g>0,h>0),(g−1,0,h) (m) = · (1 − nocN (m))
N −1

m(0,l′ ,h′ ) · N m(0,l′ ,∞) · N
+ 1− + · nocN (m),
N −1 N −1
N m(0,l′ ,h′ ) · N
P(g,l,h|g>0,l>0,h>0),(g−1,l−1,h) (m) = · (1 − nocN (m))
N −1

m(0,l′ ,h′ ) · N m(0,l′ ,h′ |h′ ≥h) · N
+ 1− + · nocN (m).
N −1 N −1
Again, we treat the case where a passive node with hop count H is chosen by an
active peer with hop count H as an unsuccessful update, since in our model H
subsumes all hop counts h ≥ H of the synchronized nodes of the actual protocol.
Like in the case of active nodes, the calculation of the probability of the successful
update in this special case requires knowledge of the actual distribution of nodes
with these hop counts, which again we do not know a priori.
6.5.3 Mean-field Limits

The probability nocN (m) of having no collision converges for N → ∞:
m(0,l,h) ·N −1
N −3
N
noc(m) = lim noc (m) = lim = e−2·m(0,l,h) .
N →∞ N →∞ N −1
For all the local transition probabilities the number of nodes N only appears in
the factor N/(N − 1), which has limit 1 for N → ∞. The limit probabilities are
thus easily obtained by removing the factor N/(N − 1) from the expressions and
by replacing nocN (m) by the above limit noc(m).
6.5.4 Comparison with Emulation Results

In [118], emulation is used to explore how GTP behaves in practice. For basic
GTP a network of 1500 nodes was emulated on a single workstation, using local
object passing implementation for communication. One node is a time source,
having hop count zero, all other nodes are not synchronized. The gossip delay is
fixed and independent of the state of a node and set to 25 seconds. The maximum
standalone period is also set to 25 seconds.
Fitting our model to this scenario, we set the fraction of nodes being a time
source to 1/1500. We assume that one step in the model corresponds to one
second in the emulation. This slightly overestimates the duration of a gossip in-
teraction, which is reported to be in the sub-second range. The maximum gossip
delay is Gmax = 25 seconds, and since it is fixed we have G(h) = Gmax for any
hop count h. The maximum delay between two updates is L = 25 seconds. The
maximum hop count is chosen to be H = 15. A single node thus can assume
26 · 26 · 17 = 11492 states. The time source fraction starts off with g = 12, that
is, it initiates a gossip interaction for the first time after 12 seconds. The unsyn-
chronized nodes have remaining gossip delays uniformly distributed between 0
and Gmax .
Figure 6.8(a) shows the evolution of the number of nodes that are aware of
the time source over time. A node becomes aware of the time source existence
when its hop count changes to a finite value. For the mean-field model we have
multiplied the fraction of nodes with a hop count smaller than ∞ by 1500 to
12
1400
10
1200
number of nodes
1000 8
average hop count

800
6
600
4
400
2
200
mean-field
emulation
0 0
0 200 400 600 800 1000 1200 0 800 1600 2400 3200 4000 4800
t t
(a) Discovering time source existence (b) Avg. hop count for different number of time sources
Figure 6.8: Comparison with emulation results (data taken from Figs. 6.1(a),
6.5(b) in [118])
obtain the depicted curve. The curves of emulation and analytical model proceed
close to each other, both approaching 1500 after about 200 seconds, that is, after
about 8 gossip cycles.
In Figure 6.8(b) we compare the evolution of the average hop count when the
number of time sources is multiplied by 10 and 100, respectively. For the mean-
field model, the average hop count is computed for synchronized nodes only, thus
being a kind of underestimation as long as not all nodes are aware of the time
source existence. At the beginning, the average hop count increases faster in the
mean-field, however, mean-field model and emulation settle to similar values.
The change in the average hop count depends logarithmically on the number of
time sources. Moreover, having the higher number of time sources result in the
nonzero elements in the occupancy measure m, have values closer to each other.
Figure 6.9 finally shows the histogram of hop counts after the protocol stabi-
lizes. For the mean-field curve we have taken the distribution at time t = 600,
350
mean field
emulation
300
number of nodes
250
200
150
100
50
0
0 5 10 15 20
hop count
Figure 6.9: Distribution of hop count (emulation results from Figure 6.6 in
[118])
neglecting the fact that there are minor oscillations because of time source gossip.
Taking this into account, and the fact that we are considering a fully connected
network in the mean-field model in contrast with a special peer sampling ser-
vice in the emulations, there is a close match between the emulation and the
mean-field result.
Figures 6.8 and 6.9 document that the presented mean-field model captures
the main features of basic GTP. The evolution of the hop counts in the considered
scenario is quite precisely represented. In the following we concentrate on the
mean-field model. We show further measures that were not considered in the
emulation experiments in [118] and also evaluate the influence of varying the
gossip delay.
6.5.5 More Properties of Basic GTP

We stick to the scenario of the previous section. Following Figure 6.9 we
depict in Figure 6.10(a) the evolution of the hop count distribution over the first
10 minutes. Each curve corresponds to the fraction of nodes that have a hop
count of at most a given value. The distance between two curves corresponds to
the fraction of nodes that have exactly a given hop count, as shown for a hop
count of seven.
At the beginning, almost all nodes are unsynchronized, which results in the
area to the left of the graph. Gradually, the nodes acquire hop counts smaller
than ∞. Since this change originates from the time source, hop counts different
from ∞ are relatively small at first. Over time, the hop count distribution settles
to a quasi stable state, with – on average – higher hop counts than at the moment
the network got fully synchronized.
Figure 6.10(a) suggests that the hop count distribution reaches a stable state.
This is not completely true, since there is a periodic disturbance whenever the
1 0.004
hop count = 0
0.9 hop count = 1
0.0035
0.8 hop count = 2
0.003
0.7
fraction of nodes
fraction of nodes
0.0025
0.6
0.5 0.002
0.4 hop count = 7

0.0015
0.3
0.001
0.2
0.0005
0.1
0 0
0 100 200 300 400 500 600 300 350 400 450 500
t t
(a) Distribution of hop count (b) Periodicity of hop count probabilities
Figure 6.10: Hop counts for constant gossip delay (25 seconds)
time source fraction gossips every 25 seconds. Figure 6.10(b) shows the fraction
of nodes having hop count zero, one, or two for the time interval from 300 to
500 seconds. The fraction of time sources (hop count 0) is constant since no
node with a hop count larger than zero can ever become a time source. The
fraction of nodes having hop count one oscillates: with each time source gossip,
the fraction increases abruptly, decreasing gradually afterwards. Also for hop
count two, there is still a visible periodicity. The change is already very small, for
higher hop counts the periodicity effect wears off completely.
0.18
interactions
collisions
0.16
changes
upgrades
0.14
0.12
per node
0.1
0.08
0.06
0.04
0.02
0
0 100 200 300 400 500 600
t
Figure 6.11: Interaction activity with gossip delay Gmin = Gmax = 25
Figure 6.11 depicts the interaction activity per node over the first 10 minutes.
The nature of the mean-field model makes it necessary to state interactions per
node. Mapping back to the original scenario, multiplying the indicated values
with 1500 results in the total number per second. Interactions are gossip attempts
by nodes which have reached hop count zero. The number stays constant due to
the fact that we have a constant gossip delay of 25 seconds. Only when the time
source fraction gossips, there is a small spike in the curve. A collision occurs if
there is more than one attempt of a gossip interaction with a node. The number
of collisions is also constant, being a function of the number of interactions. A
change means that a node adjusts its clock and hop count to a different value.
The hop count might be larger than before if the update has been been forced by
the last-update flag. At the beginning, changes are rare, since most interactions
take place between unsynchronized nodes, not leading to any updates. In the
synchronization phase, the number of changes increases and finally settles down
to a stable level. Because of enforced updates there are always changes to be
expected. With upgrades we denote changes that actually lead to a better hop
count. Their number is of course smaller than the number of changes, settling
down to a positive number as well: as long as there are “downgrading” changes
because of enforced updates, there will also be upgrading interactions in the
sequel.
Later in the chapter, we compare this graph to the mean-field results with the
1 1
0.9
0.8 0.8
fraction aware of timesource

0.7
fraction of nodes
0.6 0.6
3 seconds 6 seconds
0.5 2 seconds
0.4 0.4 5 seconds
0.3 4 seconds
0.2 0.2 1 second
0.1
0 0
600 700 800 900 1000 1100 1200 10 20 30 40 50 60 70
t t
Figure 6.12: Distribution of hop count Figure 6.13: Synchronization speed for
after time source fails for constant gos- different static gossip delays
sip delay (25 seconds).
other values of the gossip delays Gmin and Gmax in order to show the impact
of different values on the protocol performance. Thus, the scale of the y-axis is
chosen to be the same for the convenient comparison of these results.
Finally we show the behavior of the network when the time source fails. We
do this by starting with a single time source, and removing it after 10 minutes
(time t = 600). That is, by redistributing the fraction of nodes being a time source
(i.e. with hop count 0) to the other states with hop counts > 0. Figure 6.12 shows
what happens to the hop count distribution in the following 10 minutes. In this
figure, like in Figure 6.8(a), each curve represents the fraction of nodes that
have at most a certain hop count. The highest curve on the graph corresponds
to the fraction of nodes with hop count at most H = 15, i.e., the fraction of
synchronized nodes. As could be expected, nodes with a low hop count die out
over time, leaving all nodes at the chosen maximum hop count of H = 15.
6.5.6 Different Static Gossip Delays

With the next graphs we want to clarify the influence of the gossip delay on
the performance of the complete network. In general, one expects that a higher
gossip delay leads to a lower synchronization speed. On the other hand, it also
implies less communication. The question we asked ourselves was: can a short
gossip delay lead to a slower synchronization than a longer gossip delay because
of too many interactions and, subsequently, collisions?
Figure 6.13 shows the speed of synchronization for gossip delays between
one and six seconds. In general, synchronization slows down with increasing
gossip delay. But if nodes gossip every other second, that is, if the gossip delay
is one, synchronization proceeds slower than for a delay of two, three, four or
five seconds. In this case, collisions impede a fast dissemination of the timing
information through the network.
1 6
5
0.1
average hop count

0.01
per node
0.001
2
0.0001
1
1 second 1 second
2 seconds 2 seconds
6 seconds 6 seconds
1e-05 0
0 50 100 150 200 0 50 100 150 200
t t
(a) Upgrading interactions (b) Average hop count
Figure 6.14: Varying the static gossip delay
Figures 6.14(a) and 6.14(b) further substantiate this insight. In 6.14(a), the
upper set of curves depicts the total number of initiated interactions, and the
lower set shows the number of interactions really leading to an upgrade. Even
though with a gossip delay of one second the total number of interactions is
highest, the number of upgrades is lower than for a gossip delay of two seconds.
Figure 6.14(b) documents that also the average hop count for a gossip delay of
one is higher than for a gossip delay of two.
6.5.7 Dynamic Adaptation of Gossip Delay

Our model allows us to dynamically adapt the gossip delay to the state of the
system, that is, to its current hop count, via the function G(h). Gradual GTP also
offers this possibility, thereby taking more parameters (not only the hop count)
into account. In line with the description in [118] we want a node to gossip
more often if its time has “bad quality”. For our model this translates to a high
hop count.
For this purpose we introduce a minimal gossip delay Gmin . The gossip delay
of a node is then set to

Gmin , h = ∞,
G(h) = h
Gmax − ⌊ H+1 ⌋ · (Gmax − Gmin ), 0 ≤ h ≤ H.
Unsynchronized nodes initiate a gossip as often as possible while a time source

does so as seldom as possible. For all other hop counts, the gossip delay spreads
linearly between Gmin and Gmax . Note that numerically a hop count of ∞ is
treated like H + 1.
We compare three cases: the scenario considered so far with Gmin = Gmax =
25 seconds (see Figure 6.11), a small range of gossip delays with Gmin = 15 sec-
onds and Gmax = 35 seconds, and a large range with Gmin = 5 seconds and
Gmax = 45 seconds.
1 10
0.9 9
fraction not aware of timesource

0.8 8
0.7 7
average hopcount
0.6 6
0.5 5
0.4 4
0.3 3
0.2 Gmin = Gmax = 25 2

Gmin = 15, Gmax = 35
0.1 1
Gmin = 5, Gmax = 45
0 0
0 100 200 300 400 500 600
t
Figure 6.15: Synchronization speed and average hop count.
Figure 6.15 depicts both the synchronization speed (left set of curves) and the
average hop count (right set of curves) in the first 10 minutes. Since nodes with
a high hop count have more chances to upgrade if the range of the gossip delay is
enlarged, the synchronization speed is increased as opposed to the static gossip
version. But this higher gossiping frequency of nodes with a high hop count also
leads to a slightly increased average hop count.
Figures 6.16(a) and 6.16(b) compare the interaction activity of two scenarios.
While for a static gossip delay the number of interactions is independent of the
state of the system (as shown in Figure 6.11), it highly depends on the state
if the gossip delay is computed dynamically. In the beginning, when most of
the nodes are unsynchronized, there are many interactions, leading to faster
synchronization. In the long run, the activity pattern settles to similar values
for all three scenarios, with a slightly increasing number of interactions with
0.18 0.18
interactions interactions
collisions collisions
0.16 0.16
changes changes
upgrades upgrades
0.14 0.14
0.12 0.12
per node
per node
0.1 0.1
0.08 0.08
0.06 0.06
0.04 0.04
0.02 0.02
0 0
0 100 200 300 400 500 600 0 100 200 300 400 500 600
t t
(a) Gmin = 15, Gmax = 35 (b) Gmin = 5, Gmax = 45
Figure 6.16: Interaction activity.

increasing range of gossip delay.
6.6 Conclusion
We have demonstrated that mean-field analysis is suitable for gossip proto-
cols. The following premises enable mean-field analysis:
• there is a very large number of identically behaving nodes (symmetry prop-
erty, Section 1.1);
• there are no central servers or global resources;
• the behavior of a single node can be described in a local way;
• the number of states of a single node is small in comparison to the number
of nodes;
• transient measures (“at time t”) are of interest.
Extensions of the theory presented here would also allow for the incorporation of
a global memory, the failure or entering/leaving of nodes [48], the employment
of continuous-time models, and steady-state measures [45]. However, the mean-
field approach does not allow for the evaluation of a centrally managed network
or the separate modelling of one single node.
In this chapter we considered two applications of gossip protocols: along with
the presentation of the necessary theory, we developed a simple information dis-
semination model. The suitability of the mean-field approximation method was
shown by comparing the results obtained by analytically solving the resulting
DTMC, by computing the mean-field limit, and by simulating the system. The
mean-field method is applied to the bidirectional (push-pull) communication
model for the case of an application of gossip-based information dissemination,
the shuffle protocol.
As a larger case study for a gossip protocol we derived a mean-field model
for basic GTP. It includes the hop count metric and the constant gossip delay, and
also takes into account enforced updates due to the expiration of the standalone
period. We validated the fit of the mean-field model, matching it to emulation
results taken from [118]. Then we used the mean-field model to derive a variety
of interesting measures, also considering dynamically adjusted gossip delays.
With the conducted experiments we have shown how to successfully derive a
large variety of useful measures for basic GTP using a mean-field model. While
emulation like in [118] requires the availability of suitable hardware and runs
in real-time (20 minutes for most shown measures), the evaluation of the mean-
field model for a given parameter setting is done in a couple of minutes.
Automating the Mean-Field
for Dynamic Networks 7
In the previous chapter, we have investigated an abstraction method, called
mean-field method, for the performance evaluation of gossip protocols. Although
the protocols fit well the analytical framework we developed, the calculation of
the transition matrix is a cumbersome and error-prone procedure. The natural
solution is to automate the process. Furthermore, while the mean-field analysis
is well-established in epidemics and for chemical reaction systems, it is rarely
used for communication networks because a mean-field model tends to abstract
away the underlying topology.
To represent topological information, however, we extend the mean-field
analysis with concepts of classes of states and multiplicities in the state tran-
sitions. At the abstraction level of classes we define the network topology by
means of connectivity between nodes. This enables us to encode physical node
positions and model dynamic networks by allowing nodes to change their class
membership whenever they make a local state transition. Based on these exten-
sions, we derive and implement algorithms for automating a mean-field based
performance evaluation.
7.1 Introduction
We consider dynamic networks with pairwise communication between nodes.
The mean-field method allows us to evaluate systems with very large numbers of
nodes, that is, systems of a size where traditional performance evaluation meth-
ods fall short. However, as we have already seen, the calculation of a transition
probability matrix is often a cumbersome and error-prone procedure. In addi-
tion, the mean-field framework does not offer the possibility to model dynamic
networks or topological information.
Thus, we now focus on the automation of mean-field analysis, and demon-
strate how a network with pairwise communication between nodes can be mod-
117
118 Chapter 7 — Automating the Mean-Field for Dynamic Networks
elled and analyzed. For this we explore the mean-field method in two directions.
(1) We introduce a notion of classes (of states) of nodes that can represent
group membership or topological information. By changing their states, we
allow nodes to change their class membership. This enables us to model
mobile networks, where nodes are moving around.
(2) We introduce multiplicities in the state transitions such that nodes can be
created and removed. This extension is important for modelling of dynamic
networks with node departures and arrivals.
To this end, we identify a class of stochastic systems of nodes for which the con-
struction of the mean-field model can be automated. Using these results, we
have automated the mean-field method, which allows us to model large dynamic
networks. Given as user input the probabilities of communication between dif-
ferent classes of nodes, and an initial distribution of nodes, a new tool calculates
the evolution of the system in discrete time. Apart from that, the tool allows the
mean-field method to be used by a broad community, ranging from the nonexpert
outside of academia to the expert in mean-field theory. The tool is suitable for the
nonexpert since it releases the user from the burden of the details of mean-field
theory. For the expert, the tool is of interest since carrying out mean-field analysis
requires the cumbersome, error-prone computation of probabilities that describe
the complete behavior of the nodes. Moreover, the tool can easily be extended to
include other aspects of mean-field analysis (some of which we mention in the
last section of the chapter). However, it is important to stress is that this work
is not about building the tool, but aims at developing the mean-field framework
for dynamic gossip networks.
7.2 Related Work

The idea of different classes of nodes for mean-field approximation emerged
from the mean-field model of GTP, presented in Chapter 6. The model distin-
guishes classes of nodes with respect to communication behavior. Moreover,
nodes adapt their communication behavior depending on their state, and by
changing the state, nodes can change their class membership.
The multiple classes approach has also been used by [4, 33, 59, 185]. In
these works, the authors give a mean-field approximation in the form of differ-
ential equations, whereas we observe the evolution of the system in discrete time
based on matrix-vector multiplication. We focus on the automation of the gener-
alized mean-field method for dynamic gossip networks. The main advantage of
this automated framework over the equation-based solutions is that the frame-
work allows to change a format of states of nodes (a tuple) without the need to
manually derive a corresponding set of equations.
A mean-field related technique has been developed in [62, 107, 110], sup-
ported by a tool in [168]: an ODE-approximation for a specific class of continuous-
time process algebra models (PEPA). In contrast to their approach, we employ a
discrete-time model. Whether to choose continuous or discrete time of course
7.3. Modelling Framework 119
depends on the system to be modelled. Moreover, the mean-field modelling of

this chapter allows for time-dependent events and discrete time changes of node
states and network topology, cf. Section 7.5.2. These are crucial ingredients of
communication networks, but fall outside of the capabilities of an ODE-based
analysis.
The tool automates mean-field based performance evaluation for dynamic
gossip networks, an indispensable step to facilitate the mean-field analysis in the
area of communication systems.
7.3 Modelling Framework

As setup for the subsequent sections, we describe the class of models of com-
munication networks for which we automate the mean-field method. We con-
sider large networks of nodes interacting in a peer-to-peer or gossip fashion. The
nodes are characterized by discrete-time stochastic processes
{On (τ ) | τ ∈ N, n ∈ N },
with N = {1, . . . , N }, where N is the network size, n is the node identity, and τ
is the (global) discrete time. In other words, the nodes interact in a round-based
fashion. The values of On (τ ) are taken from a finite sample set Ω, called (local)
state space.
internal connectivity 0.2
connectivity 0.01 connectivity 0.1
A C
connectivity 0.05
Figure 7.1: An example of a clustered network.
We classify local states s ∈ Ω of nodes into one or more different groups,

called ‘classes’. A class C is a collection of states of nodes, C ⊆ Ω. The classes of
nodes can represent group membership, topological information, a combination
of several aspects, etc. We impose no restriction on the formation of these classes,
e.g., states are permitted to be in multiple classes. We have chosen for ‘classes
of local states’ over ‘classes of nodes’ since it enables the modelling of mobile
networks with a changing topology; nodes changing their local states may change
their class memberships. An example topology based on three classes is shown
in Figure 7.1.
Example 7.1. Figure 7.1 depicts a network consisting of three clusters (classes)
A, B and C. Each cluster in the network has a uniform link probability between
any member of the cluster, and there is a specified probability of a communication
between the clusters A, B and C. Namely, nodes of cluster A interact with nodes of
cluster B with probability 0.01; however, nodes of cluster A interact with each other
with probability 0.9, and nodes of cluster B with probability 0.2.
user user
movement
communication
base station
user user
base station
Figure 7.2: An example of a small mobile network.
C5
C1 C2 C3 communication
s1 s2 s3
C4
s4
Figure 7.3: Modelling mobile networks using classes of states.
Example 7.2. Figure 7.2 illustrates an example of a small mobile network. The
network consists of three classes of nodes, two users and one base station. One of
the users is mobile, and moves around. The base station can communicate with both
users at all times, but the communication between the users depends on the distance
between them; in the lower half of Figure 7.2 interaction between the users directly
is not possible.
7.3. Modelling Framework 121
This scenario is encoded in the framework by means of states and classes, as

shown in Figure 7.3. For the mobile user, we distinguish two states s2 and s3 ,
representing its position. Two other states, s1 and s4 represent the non-mobile user
and the base station, respectively. All states are divided into different classes, as
shown in Figure 7.3. The station in class C4 communicates with both users C5 =
{s1 , s2 , s3 }, irrespective of the position of the mobile user. The mobile user can
undergo the state transitions s2 → s3 and s3 → s2 , that express the user movement.
By moving from s2 to s3 , the mobile user changes its class from C2 to C3 , and vice
versa. The stationary user in state s1 ∈ C1 cannot communicate with the mobile
user, when the latter is in the state s3 ∈ C3 . However, this communication is possible
if the mobile user is in state s2 ∈ C2 .
We introduce the following terminology. A node is called active if it initiates a
contact at the current time step, and passive, otherwise. Moreover, a node is idle
if it is passive, and is not currently contacted by any node. On the contrary, a
node is nonidle if it is active, or a node attempts to contact it.
Definition 7.1. Let S be a set. We use Distr (S) to denote Xthe distributions over
S, that is, the set of functions µ : S 7→ [0, 1] such that µ(s) = 1. Moreover,
s∈S
by Distr ≤1 (S) we denoteXthe subdistributions over S, that is, the set of functions
µ : S 7→ [0, 1] such that µ(s) ≤ 1.
s∈S
The behavior of the network is defined by the following four functions (spec-
ified by the user):
• contacts: the network topology,
• κ: state transition for successful communication,
• φ: state transition for unsuccessful communication, and
• ǫ: state transition for idle nodes.
The function contacts defines the topology of the network. The nodes periodi-
cally interact with each other; at each time step every node picks a communica-
tion partner. The choice of the partner is governed by a probability distribution:
contacts : Ω → Distr ≤1 (P6=∅ (Ω)) ,
which determines the probability that a certain node contacts different classes of
nodes. More precisely, for any state s ∈ Ω, contacts(s) is a subdistribution
µ : P6=∅ (Ω) 7→ [0, 1]
meaning that a node in state s contacts with probability µ(C) a node from class
C 6= ∅. Then the partner is chosen uniformly at random from all nodes that are
in a state in this class C.
The probability of a node in state s ∈ Ω being active is:

X
Pact (s) = contacts(s)(C)
C⊆Ω, C6=∅
Then 1 − Pact (s) is the probability of a node in state s to be passive.

We explain this distribution in the following example.
Example 7.3. Assume that Ω = {s1 , s2 , s3 }, and consider a network with 10 nodes
in state s1 , 10 in state s2 , and 20 in state s3 . Let the contact distribution for s1 be
defined as:
contacts(s1 ) = {{s2 , s3 } 7→ 0.3, {s1 } 7→ 0.5}
and we tacitly assume C 7→ 0 for all other classes C ⊆ Ω.

This means that a node in state s1 contacts with probability 0.3 a node in state s2
or s3 (uniformly at random), with 0.5 another node in state s1 , and is idle (not
initiating communication) with the remaining probability 0.2. Hence,
Pact (s1 ) = 0.8.
Assume that a node in state s1 decides to contact the class {s2 , s3 }. The choice of
the communication partner inside this class is uniformly at random, that is, every
node among the 30 nodes in state s2 or s3 has an equal probability of being picked.
Since there are 10 nodes in state s2 and 20 nodes in state s3 , with probability 13 the
choice will fall on a node from s2 , and with 32 on s3 .
Remark. For simplicity, we will refer to “a node in state s” as “a node in s”.
When a node in s (attempts to) contact a node in t, then depending on the
success of the contact and the state t of the peer, both nodes stochastically change
their states in a manner that depends on s and t. By moving from one state to
another, the nodes may change their class membership, which can be used to
encode dynamic networks.
Since nodes pick their communication partner randomly, it can happen that
multiple nodes choose the same partner simultaneously. Moreover, the chosen
target can itself be active. In this case, the result of the interactions depends on
their sequential order. In this work, we handle the cases when a node is involved
in multiple interaction attempts as collisions. That is, a node has a collision either
if (a) it is contacted at least twice, or (b) it is active and contacted at least once.
A communication fails whenever there is a collision either in the source or in
the target. In other words, considering the pairwise communication between
the nodes as a directed graph, then every isolated edge, i.e., strongly connected
component with one edge, is a communication success, and edges that are not
isolated amount to communication failures. Such collision models are used, for
example, in the setting of wireless networks [117].
7.4. Mean-field Modelling 123
In the framework, we distinguish between the state transition of idle nodes

and nodes with a collision. The function κ defines the state transition of nodes
for the case of successful communication,
κ : Ω×Ω → Distr (Ω × Ω).
If a node in s (successfully) interacts with a node in t, then κ(s, t) defines the

distribution of the next states of both nodes, that is, κ(s, t)(s′ , t′ ) is the probability
of s′ and t′ being the successor states of s and t, respectively.
The function ǫ describes the state transition for idle nodes,
ǫ : Ω → Distr(Ω).
ǫ(s)(t) is the probability of an idle node in s to progress to the next state t.

The function φ describes the state transition for nodes whose communication
fails due to a collision,
φ : Ω → Distr(Ω),
That is, if a node in s is either the source or the target of a failed communication,
then φ(s)(s′ ) is the probability that the node moves to the state s′ .
7.4 Mean-field Modelling

In this section, we describe the theory behind the tool, and illustrate the
presented framework for two case studies.
7.4.1 Introduction and Notations

The format of the states of Ω, be it an integer, a tuple or another data struc-
ture, is irrelevant to the computations presented in this section. Therefore, we
simply identify states by their indices (with respect to a fixed total order), i.e.,
{0, . . . , l − 1}, where l is the number of different states.
Normally, the complete system is composed of the N nodes as a discrete-time
stochastic process Z N (τ ) = (O1 (τ ), . . . , ON (τ )). The size of this state space is
|Ω|N = lN . However, in case of the mean-field approximation, nodes that are in
the same state are indistinguishable. Thus, we can describe the overall state at
time τ with a vector
∆(τ ) = (∆0 (τ ), . . . , ∆l−1 (τ )),
where an entry ∆i (τ ) is the fraction of nodes in state i. This discrete-time vec-

tor is called the occupancy measure,
and the size of the state space ΩN ∆ of this
l+N −1
“occupancy process” is l−1 . That is, there are

l+N −1 l+N −1
=
N l−1
ways to combine N (not necessarily distinct) nodes in l states. The evolution of

the system is then described by the local transition probabilities of each node:
N
M∆ (t, s) = Pr{OnN (τ + 1) = t | OnN (τ ) = s, ∆(τ ) = ∆}
In other words, the next state t ∈ Ω of a node depends on its current state
s ∈ Ω and on the current occupancy measure ∆. These transition probabilities
N
form the transition probability matrix M∆(τ ).
7.4.2 Transition Probabilities and Matrix Calculation

N
We now describe how the state transition probability matrix M∆(τ ) can be
computed based on user-input probabilities, presented above. In order to com-
pute the matrix, we assume ∆(τ ) to be the current state distribution, and simply
write ∆.
We first compute the probability that a given node in s contacts any node in
t. Recall that a node in s ∈ Ω picks a communication partner from class C ⊆ Ω
with probability contacts(s)(C). It does so uniformly from the nodes that are
currently in one of the states of C. For every pair of states s, t ∈ Ω and current
distribution ∆, the probability that a node in s contacts a node in t is:
X ∆t
contact (s, t) = contacts(s)(C) ·
∆C
C⊆Ω, t∈C
P
where ∆C = u ∈ C ∆u for any set of states C ⊆ Ω. Given the current distribu-
tion ∆, the expected fraction of nodes that contact the state s ∈ Ω:
X
ξ(s) = ∆t · contact (t, s)
t∈Ω
Example 7.4. Consider the occupancy measure ∆ = (∆s , ∆t ) = (0.4, 0.6). Let
contact (t, s) = 0.5 and contact (s, s) = 1. Then the number of nodes to contact s is
ξ(s) = 0.6 · 0.5 + 0.4 · 1 = 0.7,
that is 70% of all nodes in the network will contact nodes in s.
For a node in s ∈ Ω, the probability of not being contacted equals
ξ(s)·N
1
Pc0 (s) = 1 − , (7.1)
N · ∆s
if ∆s 6= 0, and Pc0 (s) = 0, otherwise.a Here, ξ(s) · N is the expected number of
1
nodes that contact s, N · ∆s is the total amount of nodes in s, and 1 − is
N · ∆s
the probability that nodes that contact s do not choose a given node in s.
a This guarantees that the probability P
colInc (s), defined later, is equal to 1 for ∆s = 0, that is,
contact attempts amount to collisions.
Similarly, the probability of a node in s to be contacted once equals

ξ(s)·N −1
ξ(s) · N 1 1
Pc1 (s) = · · 1−
1 N · ∆s N · ∆s
ξ(s)·N −1
ξ(s) 1
= · 1− , (7.2)
∆s N · ∆s
1
if ∆s 6= 0, and Pc1 (s) = 0, otherwise.b Here, is the probability that a
N · ∆s
given node in s is contacted by nodes that contact s. Thus, Pc1 (s) expresses that
only one of ξ(s)·N nodes contacts a given node in s and the remaining ξ(s)·N −1
nodes that contacted the class s did not choose the node.
The probability that a node in s ∈ Ω is idle, i.e., neither initiating contact nor
being contacted, is:
Pidle (s) = (1 − Pact (s)) · Pc0 (s)
There are two probabilities of communication failure. Firstly, we have to take

into account the active nodes that are contacted themselves and the non-active
nodes that are contacted more than once. The sum of both renormalized to the
nonidle nodes yields the probability that a nonidle node in s has a collision:
PcolNonIdle (s) = Pact (s) · (1 − Pc0 (s))

+ (1 − Pact (s)) · (1 − Pc0 (s) − Pc1 (s)) /(1 − Pidle (s)),
if Pidle (s) 6= 1, and PcolNonIdle (s) = 0, otherwise.

Secondly, the probability that a communication with a target in s ∈ Ω failed
due to a collision at the target is:
Pc1 (s) · (1 − Pact (s)) · ∆s · N ∆s

PcolInc (s) = 1 − = 1 − Pc1 (s) · (1 − Pact (s)) · ,
ξ(s) · N ξ(s)
if ξ(s) 6= 0, and PcolInc (s) = 1, otherwise. We briefly explain the derivation of

this formula. The probability of a node in s to be contacted without collision in s
is Pc1 (s) · (1 − Pact (s)), that is, the probability of being contacted once multiplied
with the probability of being passive. Then Pc1 (s) · (1 − Pact (s)) · ∆s · N is the
expected number of nodes in s that are contacted without collision in s. As the
communication is pairwise, this number coincides with the number of nodes that
contact a node in s without collision in the target. Dividing this number by the
expected number of nodes contacting s, ξ(s) · N , we obtain the probability that
a contact with target s does not have a collision in the target.
b Again, this guarantees that the probability P
colInc (s), defined later, is equal to 1 for ∆s = 0,
that is, contact attempts amount to collisions.
s′ s
node has a collision
peer has a collision
c
b a
Figure 7.4: The three causes of communication failure.
Using these probabilities of communication failure, we can derive the proba-

bilities of successful communication. The probability of a node in s ∈ Ω to talk
to a node in t ∈ Ω, without a collision at t, is
PtalkOkTgt (s, t) = contact (s, t) · (1 − PcolInc (t))
Then, the probability of a successful contact, i.e., there is no collision at the target
nor at the source, equals
PtalkOk (s, t) = PtalkOkTgt (s, t) · Pc0 (s).
The three causes for communication failures are displayed in Figure 7.4:
(a) a nonidle node in s has a collision,
(b) a node in s′ is contacted and the source of the contact has a collision, and
(c) the target in s′ of an active node has a collision.
N
Finally, we describe the calculation of the transition probability matrix M∆ .
N
Let M∆ (t, s) be the entry in the matrix at the crossing of the row t and the
column s. For all nodes in states s, t ∈ Ω,
N
M∆ (t, s) = Pidle (s) · ǫ(s)(t)
X
+ PtalkOk (s, s′ ) · κ(s, s′ )(t, t′ )
s′ ∈ Ω, t′ ∈ Ω
X ∆s′
+ PtalkOk (s′ , s) · κ(s′ , s)(t′ , t) · (7.3)
∆s
s′ ∈ Ω, t′ ∈ Ω
+ (1 − Pidle (s)) · PcolNonIdle (s) · φ(s)(t)

X ∆s′
+ (PtalkOkTgt (s′ , s) − PtalkOk (s′ , s)) · φ(s)(t) ·
∆s
s′ ∈ Ω
X
+ (contact (s, s′ ) − PtalkOkTgt (s, s′ )) · Pc0 (s) · φ(s)(t)
s′ ∈ Ω
Note that we take ∆s′ /∆s = 0, if ∆s = 0. The matrix consists of the six sum-
mands that express the corresponding probabilities of:
(i) the state transition from s to t, if a node in s is idle;

(ii)–(iii) the state transitions after the successful contact;
(iv)–(vi) cover the cases (a)–(c) of interaction failure depicted in Figure 7.4,
namely,
(iv) the state transition from s to t, if node in s is nonidle and a
collision occurs;
(v) the state transition of nodes in s contacted by nodes in s′ , if a
collision occurs at the source nodes in s′ ; and
(vi) the state transition of active nodes in s whenever a collision
occurs at the target nodes in s′ .
7.4.3 Mean-field Convergence

The mean-field approximation captures the asymptotic behavior of the com-
plete system if the network size N tends to infinity. Namely, the occupancy mea-
sure ∆ for large networks converges to a deterministic limit, if the following
is satisfied: for all local states s, t ∈ Ω, all ∆ ∈ ΩN∆ and assuming the initial
distribution ∆(0),
N
if N → ∞, M∆ (t, s) converges uniformly in ∆ to M∆ (t, s), which is a
continuous function of ∆.
If N tends to infinity, for each local state i the fraction ∆i (τ ) of nodes in state i
at time τ converges to a deterministic limit. This requirement is clearly satisfied
N
for the matrix M∆ , since the dependence on the network size N vanishes in the
limit for all transition probabilities. Notably, when N → ∞, (7.1) and (7.2) take
their limit in forms
ξ(s) −ξ(s)/∆s
Pc0 (s) → e−ξ(s)/∆s , Pc1 (s) → ·e
∆s
if ∆s 6= 0, and 0 otherwise.
Theorem 7.1 (cf. [81]). Fix the initial occupancy measure: ∆(0) = δ(0); define
the limit of the local probability matrix:
N
M∆ = lim M∆ , ∆ ∈ ΩN
∆.
N →∞
Define the deterministic discrete time process

δ(τ + 1) = Mδ(τ ) · δ(τ ).
Then for ∀τ ∈ N,
lim ∆(τ ) = δ(τ ), with probability 1.
N →∞
In other words, δ(τ ) is the deterministic limit occupancy measure at time τ for
N → ∞.
In Section 7.6, we show that the convergence theorem holds even for the more
general setting of dynamic networks.
7.5 Cases
This section presents two case studies, an analysis of a clock synchronization
protocol GTP and of a viral spread in a mobile network. Revisiting the clock
synchronization study serves two purposes; first, the comparison highlights the
advantages of an automated approach, and second, it verifies the results of au-
tomation against the previously obtained results of Chapter 6 and [118].
7.5.1 Case Study on Clock Synchronization

The first case study illustrates that the tool can cope with large dense ma-
trices. Moreover, it shows a nontrivial encoding of the states of nodes for the
mean-field analysis. Using the model of the clock synchronization protocol GTP
from Section 6.5, we compare the results obtained by the tool with the results of
the protocol emulation from [118].
According to the protocol, the nodes are equipped with local clocks. At least
one node (time source) has the accurate time. Nodes periodically interact with
randomly chosen peers, gossiping their clock samples, and update the clocks
depending on the quality of the sample, measured according to the distance
to the time source on a synchronization path. In Section 6.5, the state of a
node is defined as a triple (g, l, h) of local parameters. The delay g defines a
period of interaction of the node. The counter l defines a period of the enforced
synchronization. The hop count h shows the distance to the time source on the
synchronization path, and is a metric of the sample quality. A node with h = ∞
is unsynchronized. The state space of a node is
Ω = {0, . . . , Gmax } × {0, . . . , L} × {0, . . . , H, ∞}
of the size |Ω| = (Gmax + 1)(L + 1)(H + 2). The occupancy measure is the fraction
of nodes in state (g, l, h).
For the mean-field analysis, we consider the following values of Gmax = 25,
1
L = 25, H = 15. The initial distribution of the time sources ∆(12,L,0) is 1500 in
1 10 100
Figure 7.5 and 1500 , 1500 , 1500 in Figure 7.6. All other nodes are unsynchronized
and have remaining gossip delays uniformly distributed between 0 and Gmax .
We compare the mean-field results with the experimental results from [118].
The latter results were obtained by emulating a network of 1500 nodes that
execute GTP, on a single workstation. We assume one step in the GTP model to
be one second in the emulations.
Figures 7.5 and 7.6 show the results of two experiments: the evolution of
the hop count distribution over the first 600 seconds, and the evolution of the
average hop count for the different number of time sources, respectively.
7.5. Cases 129
1.0
0.9
0.8
hop ≤ 7
fraction of nodes
0.7
0.6
0.5 hop ≤ 6
0.4
0.3 hop ≤ 5
0.2 mean-field
0.1 emulations
0.0
0 200 400 600
time
Figure 7.5: Distribution of the hop count (automated mean-field results) and
fraction of the synchronized nodes for the network size 1500 (emulation results)
in GTP
In Figure 7.5, each mean-field curve shows the fraction of nodes that have at
most a certain hop count. All but one node are initially unsynchronized. By com-
municating to the time source, first some of the nodes obtain small hop counts.
The hop count distribution then gradually increases, and finally converges to-
wards the stable state. The local maxima in the hop count distribution before the
final stable state is due to the forced update, governed by the parameter l. Nodes
that have not synchronized for 25 seconds are forced to update their hop count
if they interact with a synchronized node. Thus, as more nodes become synchro-
nized, the forced updates become more frequent. For the small hop counts such
as 1, this update leads to the increase of the hop count. For more details on the
effect of the forced updates, we refer to Sections 6.5.5.
The highest curve corresponds to the fraction of nodes with the hop count
at most H = 15, that is, the fraction of synchronized nodes. The curve of the
emulation results shows the fraction of nodes that are synchronized, and thus,
corresponds to the mean-field curve of at most hop count 15.
Figure 7.6 shows the average hop count for 1, 10 and 100 time sources. The
respective curves for the mean-field (gray) and emulation (black) results settle to
similar values. As one can observe, the average hop count decreases with larger
number of time sources.
7.5.2 Case Study on Mobile Networks

The purpose of this case study is to show how the framework can be used to
encode mobility of nodes. We consider the spread of a (fictive) virus between
three villages A, B and C.
12
11
10
hop count (avg) 9
8
7
6
5
4
3
2
1
0
0 600 1200 1800 2400 3000 3600 4200 4800
time
Figure 7.6: Average hop count for different number of time sources in GTP (pro-
duced by automated mean-field framework)
Every person (node) can be either healthy or infected. Additionally, every

person has a parameter resistance with a value in {0, . . . , 20} (initially, 0). For
the infected population, the resistance to the virus is increased by 1 at every
step. When it reaches the maximum value, infected people recover and remain
healthy afterwards. Note that immunity is a requirement of mean-field conver-
gence result for epidemics (cf. [81]); however, this condition is not necessary for
the framework. Whenever an infected person successfully communicates with
a healthy person with resistance lower than 20, the healthy person becomes in-
fected. The connectivity among the villages is defined as follows:

contacts(A) = {{A} 7→ 0.1, {B} 7→ 0.005}

contacts(B) = {{B} 7→ 0.1, {A, C} 7→ 0.005}


contacts(C) = {{C} 7→ 0.1, {B} 7→ 0.005}
The mobility is introduced by a “ship” S that commutes in-between the vil-
lages A and C. The ship “moves” according to the parameter position, with a
value in {S0 , . . . , S5 } (see Figure 7.7). If the ship is in position S0 or S3 , it indi-
cates that the ship is docking at village A or C, respectively. The ship positions
S1 and S5 represent the same location, but indicate the movement in opposite
directions; likewise for positions S2 and S4 .
The connectivity of the ship thus depends on its position. Let Sp denote a ship
in position p, and define:

{{A} 7→ 1}, if p = 0,

contacts(Sp ) = {{C} 7→ 1}, if p = 3,


{ }, otherwise.
7.5. Cases 131
Figure 7.7: Commuting ship
People that get on and off the ship are represented as follows. If a person
from the ship interacts with a person from a village, then they exchange their
health/infection and resistance properties. Thus we simulate that one person
leaves the ship, while the communication peer enters the ship.
0.3
infected nodes
village A
fraction of
0.2 village B
village C
ship S
0.1
0.0
0 10 20 30 40 50 60 70 80 90 100
time
Figure 7.8: Infection spread, mean-field results.
Figure 7.8 shows the evolution of the infected people over time at the differ-
ent classes of people. All villages have an equal number of inhabitants, and the
ship can carry up to half of the size of any village. Initially, 50% of the popula-
tion of village C is infected; the remaining population is healthy. For village C,
the corresponding curve starts with the highest value. Over time, the infection
spreads to the next village A, since it is connected to C via the ship S. The infec-
tion spreads fast on the ship, due to its small capacity, increasing the probability
of infecting people in the village A. At the same time, occasional travellers be-
tween the villages make it possible for the virus to spread onto villages B and
C. At some point, infected people with a value of resistance 20 recover from the
virus. Hence, the fraction of infected people decreases and approaches zero after
90 steps.
Comparison with Monte Carlo Simulations

We compare the mean-field results with Monte Carlo simulations; Figure 7.9
show simulation results of the infection spread with 100 people travelling by the
ship, and 200 people in each of the three villages. Every line in the figure is the
average of 1000 simulation runs (each 100 time steps). Note the accuracy (i.e.,
resemblance between Figure 7.8 and Figure 7.9) of the mean-field approximation
(Figure 7.8) even for the system of moderate size.
0.3
infected nodes
village A
fraction of
0.2 village B
village C
ship S
0.1
0.0
0 10 20 30 40 50 60 70 80 90 100
time
Figure 7.9: Monte Carlo simulations with 100 people on the ship and 200 people
per village
7.6 Framework for Dynamic Networks

So far, the number of nodes in the network has been invariant over time. This
section extends this to the dynamic creation and removal of nodes.
7.6.1 Introduction and Notations

We generalize the semantics of the occupancy measure ∆. Up to now, ∆s has
been the fraction of nodes in state s, and consequently
X
∆s = 1.
s∈Ω
We relax this requirement, and henceforth interpret ∆s as the number of nodes

in state s relative to the initial distribution ∆(0).
We use NS to denote the multisets over S, that is, the set functions
S 7→ N.
Accordingly, we generalize the functions κ, ǫ and φ to include, along with proba-

bilities, “multiplicity”:
κ : Ω × Ω → Distr (NΩ ), (7.4)
7.6. Framework for Dynamic Networks 133
such that for all states s, t ∈ Ω, the domain of κ(s, t) is finite. For every pair of
communicating nodes there is only a finite number of possible outcomes. Here,
the finiteness ensures the Lipschitz condition in Lemma 7.1 (below).
If two nodes in state s and state t communicate successfully, then κ(s, t)(µ)
is the probability that the communicating nodes in s and t will transform into
a multiset of nodes in the states defined by µ, that is, for every state u ∈ Ω we
obtain µ(u) nodes in u.
Example 7.5. κ(s, t)({s 7→ 3, u 7→ 1}) = 0.5 means that with 50% probability a
successful communication of nodes in s and t will result in 3 nodes in s, and 1 in u.
Note that, for the multisets, we suppress entries with multiplicity 0, e.g., t 7→ 0.
Similar to κ, we also generalize the following distributions:
ǫ : Ω → Distr (NΩ ) (7.5)

Ω
φ : Ω → Distr (N ) (7.6)
Again, we require that for every s ∈ Ω, the distributions ǫ(s) and φ(s) have a
finite domain.
Example 7.6. ǫ(s)(s 7→ 1, t 7→ 2) = 0.3 means that with 30% probability an idle
node in s will transform into one node in s, and two nodes in t.
Since the evolution of a very large system can be expressed by the equation of
δ(τ ) (as described in Section 7.4.3), the expected number of states after a state
transition can be obtained using the corresponding multiplicity.
N
We generalize the calculation of the transition matrix M∆ . The computa-
tion of contact , Pidle , PtalkOk , PcolNonIdle and PtalkOkTgt remain as described in
N
Section 7.4. For all nodes in states s, t ∈ Ω we compute M∆ (t, s) as shown in
Figure 7.10. We assume ∆s′ /∆s = 0, if ∆s = 0.
The crucial difference with the matrix (7.3) from Section 7.4 is that along
with the probabilities, we take into account the corresponding multiplicities to
calculate the expected number of nodes in state t after the state transition. For
example,
X
ǫ(s)(µ) · µ(t)
µ ∈ NΩ
expresses the expected number of nodes in t after an idle transition of a node in

s.
The second difference with the matrix (7.3) in Section 7.4 is the following.
For successful communication of two nodes, the function κ, defined in (7.4), ex-
presses the resulting distribution without distinguishing which of the two nodes
progresses to which state. This simplifies the matrix in so far that, in comparison
with (7.3), the third summand is dropped. In Figure 7.10, the second summand
‘takes care’ of the whole state transition.
X
N
M∆ (t, s) = Pidle (s) · ǫ(s)(µ) · µ(t)
µ ∈ NΩ
X X
+ PtalkOk (s, s′ ) · κ(s, s′ )(µ) · µ(t)
s′ ∈ Ω µ ∈ NΩ
X
+ (1 − Pidle (s)) · PcolNonIdle (s) · φ(s)(µ) · µ(t)
µ ∈ NΩ
 
X X ∆s′
+ (P talkOkTgt (s′ , s) − PtalkOk (s′ , s)) ·  φ(s)(µ) · µ(t) ·
∆s
s′ ∈ Ω µ ∈ NΩ
X X
+ (contact (s, s′ ) − PtalkOkTgt (s, s′ )) · Pc0 (s) · φ(s)(µ) · µ(t)
s′ ∈ Ω µ ∈ NΩ
Figure 7.10: Matrix computation for dynamic networks
7.6.2 Mean-field Convergence Revisited

To ensure that Theorem 7.1 holds for the generalized setting we show that all
required conditions are valid.
N
Lemma 7.1. The following conditions hold for M∆ :
N
C1. For all time steps τ , M∆(τ ) is independent of the network size N in the limit.
C2. The number of possible state transitions per time step is bounded.
N
C3. For all time steps τ , M∆(τ ) satisfies the Lipschitz condition.
N
Proof. Condition C1 is satisfied for M∆ , since all local state transition functions
are independent of the network size N , and for the contact probabilities Pc0 (s)
and Pc1 (s) the dependence vanishes in the limit, as shown in Section 7.4.3.
Conditions C2 and C3 hold, since there are only finitely many pairs of states,
each of which has finitely many possible transitions per time step; see (7.4),
(7.5), (7.6) where the domain of the distribution functions is required to be
finite. For condition C3 note that the maximum multiplicity is bounded for all
local transitions and this yields a global bound since there are only finitely many
possible transitions.
Hence, Theorem 7.1 holds for the generalized setting. That is, for the initial
distribution ∆(0) = δ(0) and the limit
N
M∆ = lim M∆ and δ(τ + 1) = Mδ(τ ) · δ(τ ),
N →∞
the occupancy measure converges towards the mean-field limit:

lim ∆(τ ) = δ(τ ), with probability 1.
N →∞
7.6. Framework for Dynamic Networks 135
7.6.3 Case Study on Dynamic Networks

The current example illustrates the dynamic creation and removal of nodes.
We consider the scenario inspired by the well-known Lotka-Volterra equations
[150] which model a biological system. Two species, predator A and prey B,
inhabit environment E. They periodically interact with each other, which results
in the following population dynamics.
Prey feeds from the environment, that is, contacts(B) = {{E} 7→ β}, and their
population grows whenever they eat:
κ(B, E) = {(α, (2, B), (1, E)), (1 − α, (1, B), (1, E))}.
Predators hunt prey: contacts(A) = {{B, E} 7→ 1}; they may find prey B or
nothing E. In case of a successful hunt, the population of the predators grows
while the prey is eaten:
κ(A, B) = {(λ, (2, A), (0, B), (1 − λ, (1, A), (0, B)}.
For a nonsuccessful hunt
κ(A, E) = {(1 − ν, (1, A), (1, E)), (ν, (0, A), (1, E))},
that is, the predators die naturally. Likewise, idle predators may starve to death,
or a predator dies as the result of a fight (collision) between the two:
ǫ(A)(A) = φ(A)(A) = (1 − ν, 1).
The environment stays invariant ǫ(E)(E) = φ(E)(E) = (1, 1). Likewise we define
for prey: ǫ(B)(B) = φ(B)(B) = (1, 1). All other transition or contact probabilities
are 0.
0.5
fraction of nodes
0.4 predators
prey
0.3
0.2
0.1
0.0
0 200 400 600 800 1000 time
Figure 7.11: Predator and Prey; mean-field results
Figure 7.11 shows the results of mean-field analysis using the following pa-
rameters: α = 0.04, β = 0.5, λ = 0.5, and ν = 0.05, and initial distribution
∆E = 0.8, ∆B = 0.14, and ∆A = 0.06. We can clearly see that the predator-
prey system undergoes a simple harmonic motion, with the population sizes of
predator and prey fluctuating.
Comparison with Monte Carlo simulations

We compare the mean-field results with Monte Carlo simulations. Figures 7.12
and 7.13 show simulation results of the predator-prey system with 100 predators
and 300 prey, and 500 predators and 1500 prey, respectively. Although the mean-
field approximation is precise only for large populations, we have a surprisingly
close match already for the simulation with only 100 predators. For the system
with 500 predators, the match is nearly perfect and the standard deviation stays
small over the whole period of 1000 steps.
0.5
fraction of nodes
0.4 predators
prey
0.3
0.2
0.1
0.0
0 200 400 600 800 1000 time
Figure 7.12: Monte Carlo simulations with 100 predators and 300 prey
0.5
fraction of nodes
0.4 predators
prey
0.3
0.2
0.1
0.0
0 200 400 600 800 1000 time
Figure 7.13: Monte Carlo simulations with 500 predators and 1500 prey
When dealing with simulations, there is of course always a small probability

that one of the species will become extinct. The graph in Figure 7.14 depicts the
probability that one of the species will become extinct within a period of 1000
steps in dependence of the number of predators (the number of prey is 3-times
this number). To give an impression: for 100 predators the probability of extinc-
tion is 0.12, for 200, 300, and 400 predators it decreases to 0.006, 0.00028, and
0.00004, respectively. For 500 predators we did not experience a single extinction
in over 25000 runs (that is, 25 · 106 time steps in total). Consequently, for 100
predators or more, the probability of extinction is fairly small. We further discuss
the issue of extinction in the mean-field modelling at the end of the chapter.
The graphs of both Figures 7.12 and 7.13 are the average of 1000 simulation
runs (each 1000 time steps). For the graph with 100 predators we have filtered
out the runs where one of the species became extinct (approximately 12% of the
7.7. Tool Support 137
1.0
0.8 probability of extinction
probability
0.6
0.4
0.2
0.0
0 100 200 300 400 500 predators
Figure 7.14: Probability of extinction within a period of 1000 time steps
runs, see Figure 7.14). For the graph with 500 predators no filtering was needed
(we did not encounter any extinctions).
7.7 Tool Support

We implemented a tool allowing the user to specify a system specification with
a network topology, defined by the distribution contacts and the initial distribu-
tion ∆(0), and communication behavior in the form of the transition functions
κ, ǫ, and φ. The tool then performs an automatic mean-field analysis using the
algorithm, which proceeds in two steps:
N
(i) calculation of matrix M∆ , according to (7.3) or Figure 7.10, and finding
the limit Mδ ;
(ii) matrix-vector multiplications Mδ · δ, according to Theorem 7.1.
In order to avoid inhibitions towards a new tool-dependent specification lan-
guage we decided to use existing programming languages for the system specifi-
cation. Users can either use Scala [159] to specify the system in a functional pro-
gramming style, or, alternatively, Java. Beside the fact that users might already
be familiar with these languages, there are some important advantages over tool-
dependent languages. First, we have the full power of the platform-independent
Scala/Java library at our disposal for writing the specification. Second, the writ-
ing process can be immensely accelerated by specialized development environ-
ments (such as Eclipse, IntelliJ).
Figure 7.15 shows a complete specification of a simple gossip protocol. The
class Clock defines a node type that is parameterized by a hop count ranging
from 0 to 5. The hop count is updated when a node talks to another one with
smaller hop count, as defined in talk. It corresponds to the transition function κ
of the theoretical framework.
In the specification, the hop count remains unchanged for idle nodes and in
case of collisions. That is, if a node in state s is idle or has a collision, then a
node will stay in s, ǫ(s)(s) = 1 and φ(s)(s) = 1.
Using contacts that corresponds to the distribution contacts in the mod-
elling, we define that a node contacts a random partner from all with probability
object SimpleNetDef extends NetDef {

val MAX HOP = 5
case class Clock(hop: Int) extends Node {
talk = { case b: Clock => {
Array( 1 -> Clock(hop min (b.hop+1)),
1 -> b ) } }
idle = () => Array(1 -> Clock(hop))
collision = idle
contacts = () => Array( 0.1/MAX_HOP*hop -> all )
}
val all = ContactClass( {
for (h <- (0 to MAX HOP)) yield Clock(h)}: _*)
initial ( 0.1 -> Clock(0), 0.9 -> Clock(MAX_HOP) )
}
Figure 7.15: A specification of a simple gossip protocol
0.1/5 · hop. The initial distribution ∆(0) of the occupancy measure is expressed
by the statement initial of the specification.
To handle specifications with a very large number of node states within a
reasonable time period, we implemented different performance optimizations.
For the mean-field analysis, we need a fast mapping from nodes to indices in
the vector, which represents the occupancy measure. Thus, the tool determines
a local state space of nodes in a first phase. From the local state space the tool
extracts the range for each node parameter (e.g., hop), and derives a function,
mapping states to unique IDs from the set N. For a better performance, the tool
extends the original specification with this function, and recompiles the source
code. As an additional optimization, by replacing object creation with look up
tables, the tool instantiates each state only once to reduce memory consumption.
Mean-field analysis is perfect for a parallel computation. Its central ingredi-
ent, matrix multiplication, can be easily distributed over multiple CPUs. The tool
supports computation on multi-core computers. We leave, however, distributed
execution on a cluster of computers as future work.
For the GTP case study (see Section 7.5.1), the matrix is dense (contains no
zeros) and has a size of 11492 × 11492. For this example, the tool needs 4.3
seconds per discrete time step (on a Core 2 Duo with 2.66GHz PC). Surprisingly,
the throughput of the memory (RAM) channel turns out to be the bottleneck for
parallel execution on more than two cores. However, on an 8 core machine we
reached 200% speedup of the performance of 3 cores, while more cores did not
yield further acceleration.
7.8. Conclusions 139
In this chapter so far, we presented three case studies, all supported by the
toolc .
7.8 Conclusions
We presented an automated mean-field method for large-scale systems with
pairwise communication between nodes. To model dynamic systems, we intro-
duced the notion of classes of states in our framework, and allow nodes to move
between the classes by changing their states. We reported the results of three case
studies, performed by the tool; each of the case studies illustrates one aspect of
our framework.
We have demonstrated that it is possible to model mobility in our mean-field
framework. In general, for the mean-field analysis the state space has to be
known in advance. To this end, the mean-field tool first computes the closure of
reachable states. Moreover, our framework allows for creation and removal of
nodes, and we have illustrated this aspect by modelling a dynamic system in the
style of Lotka-Volterra equations. However, for modelling biological systems, the
mean-field analysis as well the Lotka-Volterra equations assume that the number
of individuals tends to infinity, and hence, cannot capture the phenomena that a
certain population becomes instinct. In other words, in the long term run (time
τ → ∞), when extinction becomes probable, the mean-field approximation does
not give realistic results.
Regarding the future of the tool, we will experiment with different represen-
tations of the matrix, to distribute the computations over a cluster of multiple
PCs, and to extend the user-interface for more convenient input of distributions
and probabilities.
The mean-field framework presented in this chapter can be extended as fol-
lows. First, the communication between the classes can be governed by a certain
distribution, e.g., the Poisson distribution (instead of discrete uniform distribu-
tion). In the current framework, arbitrary distributions can be approximated by
assigning nodes to different classes. Second, we plan to extend the method to
allow multiple communications per node. Since the result depends on the order
of communications, we need to introduce a notion of schedulers or oracles.
Finally, the mean-field method for discrete-time models without a notion of
classes in [41, 48] allows for the incorporation of a global memory (history of the
occupancy measure). The tool currently only supports models without memory,
and the extension could be integrated in the future.
c Source code of the tool and all experimental data are available at [22].
7.9 Appendix: Specification of Basic GTP Protocol

object ClockNetDef extends NetDef {
val MAX_HOP = 16; val MAX_DELAY = 25; val MAX_TIMEOUT = 25
val allContacts = Array(
1.0 -> ContactClass({ for (g <- (0 to MAX_DELAY);
t <- (0 to MAX_TIMEOUT);
h <- (0 to MAX_HOP))
yield Clock(g, t, h) }: _*)
)
case class Clock(delay: Int, timeout: Int, hop: Int) extends Node {
talk = { case b: Clock => Array(
1 -> update(this,b),
1 -> update(b,this)) }
idle = () => Array(
1 -> Clock(ndelay(delay), 0 max (timeout - 1), hop))
collision = idle
contacts = () => if (delay == 0) allContacts else Array()
}
def update(a: Clock, b: Clock): Clock = {
if(a.hop == 0) return Clock(ndelay(a.delay), a.timeout, a.hop);
if(a.timeout == 0) {
if (b.hop == MAX_HOP)
return Clock(ndelay(a.delay), a.timeout, a.hop);
return Clock(ndelay(a.delay), MAX_TIMEOUT, b.hop + 1);
} else {
var timeout = if (b.hop + 1 < a.hop) MAX_TIMEOUT
else (a.timeout - 1)
return Clock(ndelay(a.delay), timeout, a.hop min (b.hop + 1));
}
}
def ndelay(delay: Int): Int = if (delay == 0) MAX_DELAY
else delay - 1
val SOURCE = 1.0 / 1500.0;
val NOSOURCE = (1.0 - SOURCE) / MAX_DELAY
initial (
( List(SOURCE -> Clock(12, MAX_TIMEOUT, 0)) :::
((0 to 11) ++ (13 to MAX_DELAY)).map{
delay => (NOSOURCE -> Clock(delay, MAX_TIMEOUT, MAX_HOP))
}.toList ) : _*)
}
steps (600)
7.10. Appendix: Specification for Mobile Case 141
evaluationDiagram (
new Diagram.Config(7.5, 5, new Diagram.Range(0, 1), 200, 0.1),
(0 to MAX_HOP).map{
filterHop => nodeFilter{ case node: Clock => node.hop < filterHop }
}.toArray
)
distributionDiagram (
new Diagram.Config(7.5, 5,
new Diagram.Range(0, 350d/1500), 5, 350d/1500/7),
(0 to MAX_HOP).map{
filterHop => nodeFilter{ case node: Clock => node.hop == filterHop }
}.toArray
)
7.10 Appendix: Specification for Mobile Case

object VirusNetDef extends NetDef {
def getContacts (villages: Int*) = ContactClass(
for (village <- villages; ill <- List(false, true); h <- (0 to 20))
yield Village(village, ill, h) )
trait Crowd { def vilNr: Int; def ill: Boolean; def heal: Int }
case class Village (vilNr: Int, ill: Boolean,
heal: Int) extends Node with Crowd {
contacts = () => {
val neighbours =
((vilNr-1 max 0) to (vilNr+1 min 2)).filter(_ != vilNr)
Array(0.005 -> getContacts(neighbours: _*),
0.1 -> getContacts(vilNr))
}
talk = { case vil: Crowd => Array(
1 -> Village(vilNr, (ill || vil.ill) && heal < 20, heal
1 -> Village(vil.vilNr, (ill || vil.ill) && vil.heal < 20, vil.heal)
)}
def nHealing = Math.min(heal + (if (ill) 1 else 0), 20)
idle = () => Array( 1 -> Village(vilNr, ill && heal < 20, nHealing) )
collision = idle
}
case class Ship (pos: Int, ill: Boolean,
heal: Int) extends Node with Crowd {
def vilNr = 0
contacts = () => pos match {
case 0 => Array(1 -> getContacts(0))
case 3 => Array(1 -> getContacts(2))

case _ => Array()
}
talk = { case vil: Crowd => Array(
1 -> Ship(nPos(pos), vil.ill, vil.heal),
1 -> Village(vil.vilNr, ill, heal)
)}
def nPos(pos: Int) = if (pos == 5) 0 else pos + 1
idle = () => Array(1 -> Ship(nPos(pos), ill, heal))
collision = idle
}
var ship: Double = 1.0 / (2 * 2 + 3)
var vd: Double = (1.0 - ship) / (2 + 1)
initial( (
(0 to 2).toList.map(vilNr => vd -> new Village(vilNr, false, 0)) ++
List(0.5 * vd -> Village(2, false, 0)) ++
List(0.5 * vd -> Village(2, true, 0)) ++
List(ship -> Ship(0, false, 0))
): _*)
}
steps(100)
evaluationDiagram(
new Diagram.Config(10, 5, new Diagram.Range(0, 1), 10, 0.1),
Array(nodeFilter("red",
{ case node: Ship => node.infected;
case _ => false }),
nodeFilter("" ,
{ case node: Village => node.villageNr == 0
&& node.infected;
case _ => false }),
nodeFilter("dotted",
{ case node: Village => node.villageNr == 1
&& node.infected;
case _ => false }),
nodeFilter("dashed",
{ case node: Village => node.villageNr == MAX_VILLAGE
&& node.infected;
case _ => false })
))
7.11. Appendix: Specification for Dynamic Case 143
7.11 Appendix: Specification for Dynamic Case

object PredatorNetDef extends NetDef {
case class Water() extends Node { }
case class Prey() extends Node {
talk = { case b: Water => Array(1.04 -> Prey(), 1 -> Water())
case _ => Array() }
contacts = () => Array(0.5 -> ContactClass(Water()))
}
case class Predator() extends Node {
talk = { case b: Prey => Array(1.5 -> Predator(), 0 -> Prey())
case b: Water => Array(0.95 -> Predator(), 1 -> Water())
case _ => Array() }
idle = () => Array(0.95 -> Predator())
collision = idle
contacts = () => Array(1 -> ContactClass(Prey(), Water()))
}
initial ( 0.8 -> Water(), 0.14 -> Prey(), 0.06 -> Predator() )
}
steps (1000)
evaluationDiagram (
new Diagram.Config(7.5, 5, new Diagram.Range(0, 1), 200, 0.1),
Array( nodeFilter{ case node : Water => true
case _ => false },
nodeFilter{ case node : Prey => true
case _ => false },
nodeFilter{ case node : Predator => true
case _ => false } )
)
Concluding Remarks
8
Large-scale applications running on thousands of nodes are popular nowa-
days. These systems are often diagnosed by performing simulations to discover
insight in the relation between design parameters and observed behavior. Such
experimental results provide essential data on system behavior, and can aid in
understanding the emergent behavior of the system. However, the infinite space
of the parameter settings for probabilistic systems is often too large to be ex-
plored experimentally. Moreover, it is very difficult to predict what the effects of
certain design decisions are.
This thesis focused on gossip protocols, and explored both traditional and
non-traditional ways for analyzing them. This chapter discusses the observations
made during my work on the analytical frameworks, presented in the thesis.
8.1 Analytical Models

Traditional analysis methods for computer systems such as model checking
with its exhaustive state space search, fail to cope with large gossip networks. Al-
though quite useful for studying certain behavior of small-sized networks, these
methods do not scale well for gossip protocols. On the other hand, standard
methods for modelling of epidemics scale well, but abstract away the details of a
protocol, showing only simple emergent behavior of the system.
A challenge is to develop analytical models that capture (part of) the behavior
of a system, and then subsequently optimize design parameters, at the right level
of abstraction.
8.1.1 Pairwise Exchange Model

Gossip networks consist of thousand nodes interacting with each other. I be-
lieve that modelling a gossip protocol at the level of local interactions, as demon-
strated in Chapter 3 and developed further, is a good approach to analyze such
protocols. On the one hand, this analytical model includes the mechanics of
145
146 Chapter 8 — Concluding Remarks
communication between nodes on the level of the protocol details. On the other
hand, such a model allows for studying the impact of system parameters on the
performance of the protocol, and can be used to optimally design and fine-tune
it.
Moreover, the pairwise exchange model can be integrated with traditional
analysis techniques, for instance, model checking and differential equations, as
it is shown in Chapter 5. In case of model checking, the size of the network
analyzed cannot be large, but it is a significant improvement over straightforward
models. Additionally, such a combined model captures intricacy of a protocol.
The pairwise exchange model combined with differential equations results in a
Markov Chain-style model, which enjoys scalability, at the same time bringing
the mechanics of a protocol into the model.
Figure 8.1: State transition diagram
As mentioned in Chapter 3, the pairwise exchange model is at a high abstrac-

tion level, but it can be extended to model the spread of spam (for example,
introduced for the shuffle protocol in [93]) in gossip networks. To show ‘the fla-
vor’ of the extended model, one can simply add to Figure 3.2 one more state 2,
indicating that an observed item is corrupted (see Figure 8.1). This ‘naı̈ve’ model
has nine states, and all state transitions and their corresponding probabilities
can be calculated according to the description in [93]. It could be interesting to
automate this framework in the future.
The pairwise exchange model in Chapter 3 assumes a simplified Medium Ac-
8.1. Analytical Models 147
cess Control (MAC) layer with no message losses or collisions. In Chapter 4, the
model is revised under the assumption of lossy communication channels. The
pairwise exchange model can be further combined with a model of MAC layer
embedding access mechanism in, for example, IEEE 802.11e ad hoc networks
[43, 80].
The well-known Bianchi model [43] assumes that nodes always have a mes-
sage in the queue ready for transmission, referred to as saturated or saturation
condition. That model is an analytical evaluation of the saturation throughput.
Since in gossip protocols nodes periodically gossip with each other, the Bianchi
model is not suitable for these protocols. Its extension, the Engelstad et al. model
[80], covers also a nonsaturated condition, computing the throughput for nodes
from different access categories. Both models are Markov Chains.
Integrating the pairwise exchange model with one of them can be in principle
done. To that end, however, one needs to consider, for example, one of the
following configurations:
• densely populated networks (resulting in communication interference be-
tween nodes);
• mobile networks, where the travelling time is larger than the transmission
range;
• the gossip protocol is not the only application running at the nodes;
• a probabilistic broadcast protocol with a reply mechanism (bidirectional
interaction);
• a network where nodes continually synchronize with each other.
If nodes communicate as in the shuffle protocol, the probability that a significant
number of nodes gossip simultaneously is very small (see equation (6.3)). This
leads to a low communication load, and message collisions can be neglected.
Thus, in the current settings the integrated model is simply an overkill.
The pairwise exchange model abstracts away from the underlying network
topology. As a first step, it has been used to model the properties for a fully-
connected network in Chapter 5 and 6. It would be interesting to investigate
how the pairwise exchange model can be applied to analyze properties of the
shuffle protocol for other network topologies, and to apply this framework to
another gossip protocol.
8.1.2 Statistical Sampling Method

The interplay between traditional analysis techniques and abstraction is of-
ten suitable for gossip protocols. However, despite the natural simplicity of the
protocols, they exhibit a complex probabilistic behavior which is different for dif-
ferent configurations (that is, network topology and the probability of message
loss). This thesis introduced a hybrid method to compute a part of an analytical
expression with statistical data sampling from large-scale experiments.
The calculation of the correction factor in Section 3.4.4 can be seen as a
traditional curve fitting method, and a somewhat similar approach is taken in
148 Chapter 8 — Concluding Remarks
Chapter 4. In this chapter, the calculation of the probability Pdrop combines both
rigorous modelling of the message exchange in the shuffle protocol and statisti-
cal data sampling from large-scale Monte Carlo simulations for different network
topologies. This approach allows for building a framework that captures the be-
havior of the gossip protocol without incorporation of scenario-specific elements
into the analytical model. It is a challenge, however, to decompose the model
into its basic components in order to separate configuration-dependent ones and
compute them separately.
On the contrary, traditional analytical methods would allow only for mod-
elling each specific scenario configuration, reducing the applicability of the frame-
work to those very specific configurations. Thus, I see the statistical sampling
method as a golden mean between high-level models such as for epidemics show-
ing only the emergent behavior, and the low-level models of the protocol that
depend on particular implementation settings.
8.1.3 Mean-field Framework

The use of deterministic models like differential equations has been extremely
popular for modelling of epidemics since 1920s. However, one of them, mean-
field analysis, originating from statistical mechanics, has only been applied re-
cently for the effective analysis of very large communication systems, including
gossip protocols as demonstrated in Chapters 6 and 7.
With the mean-field framework developed in this thesis, one can choose spe-
cific local parameters for the performance analysis of a gossip protocol. Chap-
ter 6 provided step-by-step guidelines for a manual analysis. The framework
requires the calculation of a transition probability matrix, which is often a te-
dious and error-prone procedure. Therefore, the mean-field framework has been
automated and extended for dynamic networks in Chapter 7.
Mean-field models assume that the network size tends to infinity, and pro-
vides an asymptotic analysis of the system. It also predicts the behavior of the
system before the steady state with a reasonable accuracy (Sections 6.4.5 and
6.5.4). I believe that mean-field method and gossip protocols constitute a perfect
symbiosis.
However, some questions with respect to these models remain open. The
most important issue is related to the fact that the state space of a node has to be
finite, which is quite tricky (if possible at all) to achieve for real numbers that are
used in gossip-based aggregation protocols. For certain protocols this problem
can be dealt with an abstraction.
Furthermore, it would be interesting to investigate mean-field analysis for al-
ternative stochastic models for the nodes, for example, by moving to the continuous-
time context [45] or by introducing nondeterminism using Markov decision pro-
cesses [160]. The automated mean-field framework can be extended to allow for
calculation of the accuracy of the mean-field results using the second and higher
order moments [31, 57].
8.2. Industrial Importance 149
8.2 Industrial Importance

I think that the future of gossip protocols is bright. Along with the advances in
computer science and robotics, autonomous robots become more and more com-
pact, and at the same time more ‘intelligent’. Already today autonomous robots
show impressive skills in playing football, and swarms of autonomous robots
help cleaning the recent oil spill in the Mexican Gulf. The spectrum of applica-
tion of autonomous robots extends rapidly. A long-term vision in bioinformatics
is to develop nanorobots than can be injected in the human bloodstream to re-
pair damaged organs and cells from inside our bodies. Although this vision will
take decades to materialize, the general trend is obvious: large swarms of small,
autonomous robots will cooperate to reach a certain goal. In such a setting,
communication is lossy and the single units are prone to failure. Here, gossip
protocols are a natural design choice.
Besides robotics, technologies for renewable energy can also use gossip pro-
tocols. For example,
• sensors embedded in wind turbines, gossiping the wind direction and ag-
gregating this information to decide on the optimal angle and direction;
• sensors embedded in solar panels, gossiping and making mutual decisions
on the optimal angle of the mirrors.
Gossip protocols with their built-in redundancy offer the perfect solution to
deal with changing topologies, node failures and message corruption. Therefore,
we need to put more work in the development of analytical frameworks for gossip
protocols.
Summary of the Thesis
9
9.1 Samenvatting
Met het toenemend aantal roddelprotocollen is er steeds meer behoefte aan
een inzichtelijke en systematische analyse hiervan. Echter, de formele analyse
van dergelijke protocollen is nog steeds een tamelijk onontgonnen onderzoeks-
gebied, met vele uitdagingen en open problemen. Dit proefschrift richt zich op
de analyse van roddelprotocollen in peer-to-peer en ad-hoc netwerken, en on-
twikkelt verscheidene modelleer-raamwerken die schaalbaar zijn ten opzichte
van de grootte van het netwerk.
Dit proefschrift bestaat uit drie delen. Het eerste deel evalueert een roddel-
gebaseerd disseminatieprotocol, op basis van een systeemmodel met perfecte
zowel als imperfecte communicatiekanalen. Het analytische model verheldert de
relatie tussen de parameters van een roddelprotocol, en maakt het kiezen van
optimale waardes voor deze parameters mogelijk.
Het model wordt in het tweede deel van het proefschrift gebruikt om te
bepalen hoe snel data zich voortplant in een netwerk. Disseminatie wordt gemod-
elleerd met behulp van de probablistische model-checker PRISM, als ook door
middel van differentiaalvergelijking. Dit deel onderzoekt verder de impact van
verschillende schedulers op de prestaties van een protocol.
In het laatste deel wordt een zogenaamde mean-field methode uit de statis-
tische mechanica aangepast voor en toegepast op roddelprotocollen. Een klasse
van stochastische systemen wordt geı̈dentificeerd waarvoor de constructie van
een mean-field model kan worden geautomatiseerd, en een tool wordt gepre-
senteerd tezamen met drie verschillende toepassingen (op uniforme, mobiele en
dynamische netwerken).
9.2 Summary
With the growing number of gossip protocols, there is an increasing demand
for their analysis in an insightful and systematic way. However, the formal anal-
151
152 Chapter 9 — Summary of the Thesis
ysis of these protocols is still a rather unexplored research field, with many chal-
lenges and open problems. This thesis focuses on the analysis of gossip protocols
in peer-to-peer and ad hoc networks, and develops various modeling frameworks
that are scalable with respect to the network size.
The thesis consists of three main parts. The first part evaluates a gossip-
based dissemination protocol, based on a system model with perfect and lossy
communication channels. The analytical model clarifies the relationship between
the protocol parameters and allows for their optimization.
The model is used in the second part of the thesis to show how fast a data item
is disseminated through a network. To this end, dissemination is modeled using
the probabilistic model checker PRISM, and by means of differential equations.
This part also explores the impact of different scheduling policies on the protocol
performance.
In the last part, a mean-field framework originating from statistical mechanics
is adapted and applied to gossip protocols. A class of stochastic systems is iden-
tified for which the construction of the mean-field model can be automated, and
a tool is presented which is applied to three different case studies (on uniform,
mobile and dynamic networks).
Bibliography
[1] M. Abramowitz and I. A. Stegun. Handbook of Mathematical Functions

with Formulas, Graphs, and Mathematical Tables. Dover, New York, 9th
edition, 1972.
[2] L. Afanassieva, S. Popov, and G. Fayolle. Models for transporation net-
works. J. of Math. Sciences, 84(3):1092–1103, 1997.
[3] A. Allavena, A. Demers, and J. E. Hopcroft. Correctness of a gossip based
membership protocol. In Proc. Symp. on Principles of Distributed Comput-
ing (PODC), pages 292–301. ACM Press, 2005.
[4] E. Altman, P. Nain, and J.-C. Bermond. Distributed storage management
of evolving files in delay tolerant ad hoc networks. In Proc. Conf. of Com-
puter and Communications Societies (INFOCOM), pages 1431–1439. IEEE,
2009.
[5] T. Andel and A. Yasinsac. On the credibility of MANET simulations. Com-
puter, 37(7):48–54, 2006.
[6] H. Andersson and T. Britton. Stochastic epidemic models and their statistical
analysis, volume 151 of Lecture Notes in Statistics. Springer-Verlag, 2000.
[7] J. P. Aparicio, M. A. Natiello, and H. G. Solari. The quasi-deterministic
limit of population dynamics. Presented at Int. ISAAC Congress, July 25-
30, 2005. Preprint, 2007.
[8] S. Assaf, P. Diaconis, and K. Soundararajan. A rule of thumb for riffle
shuffling. Technical report, Stanford University, 2008.
[9] F. Baccelli, A. Chaintreau, D. De Vleeschauwer, and D. McDonald. A mean-
field analysis of short lived interacting TCP flows. ACM SIGMETRICS Per-
formance Evaluation Review, 32(1):343–354, 2004.
153
154 BIBLIOGRAPHY
[10] F. Baccelli, A. Chaintreau, D. De Vleeschauwer, and D. McDonald. HTTP

turbulence. AMS Networks and Heterogeneous Media, 1:1–40, 2006.
[11] F. Baccelli, D. McDonald, and M. Lelarge. Metastable regimes for multi-
plexed TCP flows. In Allerton Conf. on Communication, Control, and Com-
puting, volume 42, pages 1005–1011. Curran Associates, Inc., 2004.
[12] F. Baccelli, D. McDonald, and J. Reynier. A mean-field model for mul-
tiple TCP connections through a buffer implementing RED. Performance
Evaluation, 49(1-4):77–97, 2002.
[13] N. Bahi-Jaber and D. Pontier. Modeling transmission of directly transmit-
ted infectious diseases using colored stochastic Petri nets. Mathematical
Biosciences, 185(1):1–13, 2003.
[14] C. Baier, L. Cloth, B. R. Haverkort, M. Kuntz, and M. Siegle. Model check-
ing markov chains with actions and state labels. IEEE Transactions Soft-
ware Eng., 33(4):209–224, 2007.
[15] C. Baier, B. R. Haverkort, H. Hermanns, and J.-P. Katoen. Model checking
continuous-time Markov chains by transient analysis. In Proc. Conf. on
Computer Aided Verification (CAV), volume 1855 of LNCS, pages 358–372.
Springer, 2000.
[16] C. Baier, B. R. Haverkort, H. Hermanns, and J.-P. Katoen. Model-checking
algorithms for continuous-time Markov chains. IEEE Transactions Software
Eng., 29(6):524–541, 2003.
[17] N. T. Bailey. Mathematical Theory of Infectious Diseases and Its Applications.
Griffin, London, 2nd edition, 1975.
[18] R. Bakhshi, F. Bonnet, W. Fokkink, and B. Haverkort. Formal analysis tech-
niques for gossiping protocols. ACM SIGOPS Oper. Syst. Rev., 41(5):28–36,
2007.
[19] R. Bakhshi, L. Cloth, W. Fokkink, and B. Haverkort. Mean-field analysis
for the evaluation of gossip protocols. In Proc. Conf. on Quantitative Eval-
uation of SysTems (QEST), pages 247–256. IEEE Computer Society, 2009.
[20] R. Bakhshi, L. Cloth, W. Fokkink, and B. R. Haverkort. Mean-field analy-
sis for the evaluation of gossip protocols. ACM SIGMETRICS Performance
Evaluation Review, 36(3):31–39, 2008.
[21] R. Bakhshi, L. Cloth, W. Fokkink, and B. R. Haverkort. Mean-field frame-
work for performance evaluation of push-pull gossip protocols. Perfor-
mance Evaluation, 2010. In Press.
BIBLIOGRAPHY 155
[22] R. Bakhshi, J. Endrullis, and S. Endrullis. Tool for automated mean-field

framework. http://www.few.vu.nl/~rbakhshi/alg/mean.tar.gz.
[23] R. Bakhshi, J. Endrullis, S. Endrullis, W. Fokkink, and B. Haverkort. Au-
tomating the mean-field method for large dynamic gossip networks. In
Proc. Conf. on Quantitative Evaluation of SysTems (QEST), pages 241–350.
IEEE Computer Society, 2010.
[24] R. Bakhshi, J. Endrullis, W. Fokkink, and J. Pang. Brief announcement:
asynchronous bounded expected delay networks. In Proc. Symp. on Prin-
ciples of Distributed Computing (PODC), pages 392–393. ACM, 2010.
[25] R. Bakhshi and A. Fehnker. On the impact of modelling choices for dis-
tributed information spread. A comparative study. In Proc. Conf. on Quan-
titative Evaluation of SysTems (QEST), pages 41–50. IEEE Computer Soci-
ety, 2009.
[26] R. Bakhshi and W. Fokkink. How probable is it to discard an ace of hearts?
In Liber Amicorum for Roel de Vrijer. Letters and essays dedicated to Roel de
Vrijer on the occasion of his 60th birthday, pages 1–9. Lulu, 2009.
[27] R. Bakhshi, W. Fokkink, J. Pang, and J. van de Pol. Leader Election in
Anonymous Rings: Franklin Goes Probabilistic. In Proc. Conf. on Theoret-
ical Computer Science (TCS), volume 273 of IFIP, pages 57–72. Springer,
2008.
[28] R. Bakhshi, D. Gavidia, W. Fokkink, and M. van Steen. An analytical model
of information dissemination for a gossip-based protocol. Computer Net-
works, 53(13):2288–2303, 2009.
[29] R. Bakhshi, D. Gavidia, W. Fokkink, and M. van Steen. An analytical model
of information dissemination for a gossip-based protocol. In Proc. Conf. on
Distributed Computing and Networking (ICDCN), volume 5408 of LNCS,
pages 230–242. Springer, 2009.
[30] R. Bakhshi and D. Gurov. Verification of Peer-to-Peer Algorithms: a Case
Study. In Proc. Workshop on Methods and Tools for Coordinating Concur-
rent, Distributed and Mobile Systems (MTCoord), volume 181 of ENTCS,
pages 35–47. Elsevier, June 2007.
[31] K. P. Balanda and H. L. MacGillivray. Kurtosis: A critical review. The
American Statistician, 42(2):111–119, 1988.
[32] R. Baldoni, A. Corsaro, L. Querzoni, S. Scipioni, and S. T. Piergiovanni.
Coupling-based internal clock synchronization for large-scale dynamic dis-
tributed systems. IEEE Transactions Parallel Distrib. Syst., 21(5):607–619,
2010.
156 BIBLIOGRAPHY
[33] N. Banerjee, M. D. Corner, D. Towsley, and B. N. Levine. Relays, base

stations, and meshes: enhancing mobile networks with infrastructure. In
Proc. Conf. on Mobile computing and networking (MobiCom), pages 81–91,
2008.
[34] Z. Bar-Yossef, R. Friedman, and G. Kliot. RaWMS: random walk based
lightweight membership service for wireless ad hoc network. In Proc.
Symp. on Mobile Ad Hoc Networking and Computing (MobiHoc), pages
238–249. ACM Press, 2006.
[35] F. Bause and P. Kritzinger. Stochastic Petri Nets: An Introduction to the The-
ory. Advanced Studies in Computer Science. Vieweg Verlagsgesellschaft,
1996.
[36] R. J. Baxter. Exactly Solved Models in Statistical Mechanics. Academic Press,
1982.
[37] D. Bayer and P. Diaconis. Trailing the dovetail shuffle to its lair. Annals of
Applied Probability, 2(2):294–313, 1992.
[38] G. Behrmann, A. David, and K. G. Larsen. A tutorial on Uppaal. In Formal
Methods for the Design of Real-Time Systems: Proc. 4th Int. School on Formal
Methods for the Design of Comput., Commun. and Software Syst. (SFM-RT
2004), number 3185 in LNCS, pages 200–236. Springer, 2004.
[39] A. Bell and B. R. Haverkort. Sequential and distributed model checking

of Petri nets. Journal on Software Tools for Technology Transfer (STTT),
7(1):43–60, 2005.
[40] A. Bell and B. R. Haverkort. Distributed disk-based algorithms for model
checking very large Markov chains. Formal Methods in System Design,
29(2):177–196, 2006.
[41] M. Benaı̈m and J.-Y. Le Boudec. A Class Of Mean Field Interaction Mod-
els for Computer and Communication Systems. Performance Evaluation,
65(11-12):823–838, 2008.
[42] Y. Bertot and P. Castéran. Interactive Theorem Proving and Program De-
velopment Coq’Art: The Calculus of Inductive Constructions, volume XXV of
Texts in Theoretical Computer Science: An EATCS Series. Springer, 2004.
[43] G. Bianchi. Performance analysis of the ieee 802.11 distributed coor-
dination function. IEEE Journal on Selected Areas in Communications,
18(3):535–547, 2000.
[44] K. Birman, M. Hayden, O. Ozkasap, Z. Xiao, M. Budiu, and Y. Minsky. Bi-
modal multicast. ACM Transactions on Comput. Syst., 17(2):41–88, 1999.
BIBLIOGRAPHY 157
[45] A. Bobbio, M. Gribaudo, and M. Telek. Analysis of large scale interacting

systems by mean field method. In Proc. Conf. on Quantitative Evaluation
of SysTems (QEST), pages 215–224. IEEE, 2008.
[46] F. Bonnet. Performance analysis of Cyclon, an inexpensive membership
management for unstructured P2P overlays. Master thesis, ENS Cachan
Bretagne, University of Rennes, IRISA, 2006.
[47] E. Bortnikov, M. Gurevich, I. Keidar, G. Kliot, and A. Shraer. Brahms:
Byzantine resilient random membership sampling. Computer Networks,
53(13):2340–2359, 2009.
[48] J.-Y. L. Boudec, D. McDonald, and J. Mundinger. A generic mean field
convergence result for systems of interacting objects. In Proc. Conf. on
Quantitative Evaluation of SysTems (QEST), pages 3–18. IEEE, 2007.
[49] J. Bowen and M. Hinchey. Ten commandments of formal methods ...ten
years later. Computer, 39(1):40–48, 2006.
[50] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah. Gossip algorithms: design,
analysis and applications. In Proc. Conf. of Computer and Communications
Societies (INFOCOM), volume 3, pages 1653–1664. IEEE Press, 2005.
[51] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah. Mixing times for random
walks on geometric random graphs. In Proc. 7th Workshop on Algorithm
Eng. and Experiments and 2nd Workshop on Analytic Algorithmics and Com-
binatorics (ALENEX/ANALCO 2005), pages 240–249. SIAM, 2005.
[52] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah. Randomized gossip algo-
rithms. IEEE/ACM Transactions on Networking (TON), 14(SI):2508–2530,
2006.
[53] L. Breslau, D. Estrin, K. Fall, S. Floyd, J. Heidemann, A. Helmy, P. Huang,
S. McCanne, K. Varadhan, Y. Xu, and H. Yu. Advances in network simula-
tion. Computer, 33(5):59–67, 2000.
[54] Y. Busnel, R. Beraldi, and R. Baldoni. A formal characterization of uniform
peer sampling based on view shuffling. In Proc. Conf. on Parallel and
Distributed Computing, Applications and Technologies (PDCAT), pages 360–
365, 2009. Extended version to appear in JPDC.
[55] M. Cadilhac, T. Hérault, R. Lassaigne, S. Peyronnet, and S. Tixeuil. Eval-
uating complex MAC protocols for sensor networks with APMC. In Proc.
Workshop on Automated Verification of Critical Syst (AVoCS’06), volume
185 of ENTCS, pages 33–46. Elsevier, 2007.
[56] R. Cardell-Oliver. Why flooding is unreliable in multi-hop, wireless net-
works. Technical Report UWA-CSSE-04-001, University of Western Aus-
tralia, 2004.
158 BIBLIOGRAPHY
[57] G. Casella and R. L. Berger. Statistical Inference. Pacific Grove: Duxbury,

2nd edition, 2002.
[58] D. Cavin, Y. Sasson, and A. Schiper. On the accuracy of MANET simula-
tors. In Proc. 2nd ACM Int. Workshop on Principles of Mobile Computing
(POMC’02), pages 38–43. ACM Press, 2002.
[59] A. Chaintreau, J.-Y. Le Boudec, and N. Ristanovic. The age of gossip:
spatial mean field regime. In Proc. Conf. on Measurement and Modeling of
Computer Systems (SIGMETRICS), pages 109–120. ACM, 2009.
[60] F. Chierichetti, S. Lattanzi, and A. Panconesi. Almost tight bounds for
rumour spreading with conductance. In Proc. ACM Symp. on Theory of
Computing (STOC), pages 399–408. ACM Press, 2010.
[61] F. Chierichetti, S. Lattanzi, and A. Panconesi. Rumour spreading and
graph conductance. In Proc. ACM-SIAM Symp. on Discrete algorithm
(SODA), pages 1657–1663. SIAM, 2010.
[62] F. Ciocchetta, A. Degasperi, J. Hillston, and M. Calder. Some investigations
concerning the CTMC and the ODE model derived from Bio-PEPA. In
Proc. Workshop on From Biology to Concurrency and Back (FBTC), volume
229(1) of ENTCS, pages 145–163. Elsevier, 2009.
[63] A. Clark, S. Gilmore, J. Hillston, and M. Tribastone. Stochastic process
algebra. In Formal Methods for the Performance Evaluation: Proc. 7th
Int. School on Formal Methods for the Design of Comput., Commun. and
Software Syst. (SFM-PE 2007), number 4486 in LNCS, pages 132–179.
Springer, 2007.
[64] E. M. Clarke, O. Grumberg, and D. A. Peled. Model Checking. MIT Press,
2000.
[65] C. Cooper, M. Dyer, and A. Handley. The flip markov chain and a random-
izing P2P protocol. In Proc. Symp. on Principles of Distributed Computing
(PODC), pages 141–150. ACM, 2009.
[66] P. Costa, V. Gramoli, M. Jelasity, G. P. Jesi, E. Le Merrer, A. Montresor, and
L. Querzoni. Exploring the interdisciplinary connections of gossip-based
systems. ACM SIGOPS Oper. Syst. Rev., 41(5):51–60, 2007.
[67] P. Crouzen, J. van de Pol, and A. Rensink. Applying formal methods to gos-
siping networks with mCRL and GROOVE. ACM SIGMETRICS Performance
Evaluation Review, 36(3):7–16, 2008.
[68] P. Dagum, R. Karp, M. Luby, and S. Ross. An optimal algorithm for Monte
Carlo estimation. SIAM J. Comput., 29(5):1484–1496, 2000.
BIBLIOGRAPHY 159
[69] D. J. Daley and J. Gani. Epidemic Modelling: An Introduction. Cambridge

University Press, Cambridge, UK, 1999.
[70] R. Darling and J. Norris. Differential equation approximations for Markov
chains. Probab. Surveys, 5:37–79, 2008.
[71] D. Dawson, J. Tang, and Y. Zhao. Balancing queues by mean field interac-
tion. Queueing Syst. Theory & Appl., 49(3–4):335–361, 2005.
[72] S. Deb, M. Médard, and C. Choute. Algebraic gossip: a network coding
approach to optimal multiple rumor mongering. IEEE/ACM Transactions
on Networking (TON), 14(SI):2486–2507, 2006.
[73] A. Demaille, S. Peyronnet, and T. Hérault. Probabilistic verification of
sensor networks. In Proc. Conf. on Comput. Sci., Research, Innovation and
Vision for the Future (RIVF), pages 45–54. IEEE Computer Society, 2006.
[74] A. Demaille, S. Peyronnet, and B. Sigoure. Modeling of sensor networks
using XRM. In Proc. Symp. on Leveraging Applications of Formal Methods,
Verification and Validation (ISoLA), pages 271–276. IEEE, 2006.
[75] A. Demers, D. Greene, C. Hauser, W. Irish, J. Larson, S. Shenker, H. Stur-
gis, D. Swinehart, and D. Terry. Epidemic algorithms for replicated
database maintenance. In Proc. Symp. on Principles of Distributed Com-
puting (PODC), pages 1–12. ACM Press, 1987.
[76] A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from
incomplete data via the EM algorithm. J. Roy. Statist. Soc. B, 39:1–38,
1977.
[77] A. Dimakis, S. Kar, J. Moura, M. Rabbat, and A. Scaglione.
Gossip algorithms for distributed signal processing. Tech-
nical Report arXiv:1003.5309, CoRR, 2010. Available at
http://arxiv.org/abs/1003.5309.
[78] A. G. Dimakis, A. D. Sarwate, and M. J. Wainwright. Geographic gossip:
efficient aggregation for sensor networks. In Proc. 5th Int. Conf. on Inform.
processing in sensor networks (IPSN’06), pages 69–76. ACM Press, 2006.
[79] N. Drost, E. Ogston, R. van Nieuwpoort, and H. Bal. ARRG: Real-world
gossiping. In Proc. Symp. on High Performance Distributed Computing
(HPDC), pages 147–158. ACM, 2007.
[80] P. E. Engelstad and O. N. Østerbø. Non-saturation and saturation analy-
sis of IEEE 802.11e EDCA with starvation prediction. In Proc. Symp. on
Modeling, analysis and simulation of wireless and mobile systems (MSWiM),
pages 224–233. ACM, 2005.
160 BIBLIOGRAPHY
[81] S. N. Ethier and T. G. Kurtz. Markov Processes: Characterization and Con-

vergence. Wiley, 1986.
[82] P. Eugster, R. Guerraoui, S. Handurukande, A.-M. Kermarrec, and
P. Kouznetsov. Lightweight probabilistic broadcast. ACM Transactions on
Comput. Syst., 21(4):341–374, 2003.
[83] P. T. Eugster, R. Guerraoui, A.-M. Kermarrec, and L. Massoulié. From
epidemics to distributed computing. Computer, 37(5):60–67, 2004.
[84] A. Fehnker and P. Gao. Formal verification and simulation for performance
analysis for probabilistic broadcast protocols. In Proc. 5th Conf. on Ad-Hoc,
Mobile, and Wireless Networks (ADHOC-NOW’06), volume 4104 of LNCS,
[85] U. Feige, D. Peleg, P. Raghavan, and E. Upfal. Randomized broadcast in
networks. In SIGAL Int. Symp. on Algorithms, volume 450 of LNCS, pages
128–137. Springer, 1990.
[86] J. Fernandez, H. Garavel, A. Kerbrat, L. Mounier, R. Mateescu, and
M. Sighireanu. CADP – a protocol validation and verification toolbox.
In Proc. Conf. on Computer Aided Verification (CAV), volume 1102 of LNCS,
[87] G. S. Fishman. Monte Carlo: Concepts, Algorithms, and Applications.
Springer Series in Operations Research. Springer, 1996.
[88] G. Forsythe, M. Malcolm, and C. Moler. Computer Methods for Mathemati-
cal Computations. Prentice-Hall, 1977, New Jersey.
[89] D. Frey, R. Guerraoui, A.-M. Kermarrec, M. Monod, and V. Quéma.
Stretching gossip with live streaming. In Proc. Conf. on Dependable Sys-
tems and Networks (DSN), pages 259–264. IEEE, 2009.
[90] R. Friedman, A.-M. Kermarrec, H. Miranda, and L. Rodrigues. Gossip-
based dissemination. In B. Garbinato, H. Miranda, and L. Rodrigues,
editors, Middleware for Network Eccentric and Mobile Applications, pages
169–190. Springer, 2009.
[91] A. Ganesh, L. Massoulie, and D. Towsley. The effect of network topology
on the spread of epidemics. In Proc. Conf. of Computer and Communica-
tions Societies (INFOCOM), volume 2, pages 1455–1466. IEEE Computer
Society, 2005.
[92] A. J. Ganesh, A.-M. Kermarrec, and L. Massoulie. SCAMP: Peer-to-peer
lightweight membership service for large-scale group communication. In
Proc. Workshop on Networked Group Communication (NGC), volume 2233
of LNCS, pages 44–55. Springer, 2001.
BIBLIOGRAPHY 161
[93] D. Gavidia, G. P. Jesi, C. Gamage, and M. van Steen. Canning spam in gos-
sip wireless networks. In Proc. Symp. on Modeling, analysis and simulation
of wireless and mobile systems (MSWiM), pages 30–37. IEEE, 2007.
[94] D. Gavidia, S. Voulgaris, and M. van Steen. A gossip-based distributed
news service for wireless mesh networks. In Proc. Conf. on Wireless On
demand Network Syst. and Services (WONS), pages 59–67. IEEE Computer
Society, 2006.
[95] M. Gilleland. Working with fractions in Java. Available at
http://www.merriampark.com/fractions.htm, 2002.
[96] R. W. Gosper. Decision procedure for indefinite hypergeometric summa-
tion. Proc. Nat. Acad. Sci. USA, 75(1):40–42, 1978.
[97] R. Grosu and S. A. Smolka. Monte carlo model checking. In Proc. Conf. on
Tools and Algorithms for the Construction and Analysis of Systems (TACAS),
volume 3440 of LNCS, pages 271–286. Springer, 2005.
[98] G. Guirado, T. Hérault, R. Lassaigne, and S. Peyronnet. Distribution, ap-
proximation and probabilistic model checking. ENTCS, 135(2):19–30,
2006.
[99] I. Gupta, M. Nagda, and C. F. Devaraj. The design of novel distributed
protocols from differential equations. Distributed Computing, 20(2):95–
114, 2007.
[100] M. Gurevich and I. Keidar. Correctness of gossip-based membership un-
der message loss. In Proc. Symp. on Principles of Distributed Computing
(PODC), pages 151–160. ACM, 2009.
[101] J. M. Hammersley and K. W. Morton. Poor man’s Monte Carlo. J. Royal
Statistical Soc. Series B (Methodological), 16(1):23–38, 1954.
[102] H. Hansson and B. Jonsson. A logic for reasoning about time and reliabil-
ity. Formal Aspects of Computing, 6(5):512–535, 1994.
[103] A. Harwood and O. Ohrimenko. Mean field models of message through-
put in dynamic peer-to-peer systems. Technical Report arXiv:0705.2065,
CoRR, 2007. Available at http://arxiv.org/abs/0705.2065.
[104] B. R. Haverkort. Are stochastic process algebras good for performance and
dependability evaluation? In Proc. Workshop on Process Algebra and Perfor-
mance Modeling (PAPM), volume 1853 of LNCS, pages 501–510. Springer,
2000.
[105] B. R. Haverkort. Markovian models for performance and dependability
evaluation. In European Educ. Forum: School on Formal Methods and Per-
formance Analysis, volume 2090 of LNCS, pages 38–83. Springer, 2002.
162 BIBLIOGRAPHY
[106] B. R. Haverkort, H. Hermanns, and J.-P. Katoen. On the use of model

checking techniques for dependability evaluation. In Proc. Symp. on Re-
liable Distributed Syst. (SRDS), pages 228–237. IEEE Computer Society,
2000.
[107] R. A. Hayden and J. T. Bradley. A fluid analysis framework for a markovian
process algebra. Theor. Comput. Sci., 411(22-24):2260–2297, 2010.
[108] F. Heidarian, J. Schmaltz, and F. Vaandrager. Analysis of a clock-
synchronization protocol for wireless sensor networks. In Proc. Symp. on
Formal Methods (FM), volume 5850 of LNCS, pages 516–531. Springer,
2009.
[109] T. Hérault, R. Lassaigne, F. Magniette, and S. Peyronnet. Approximate
probabilistic model checking. In Proc. 5th Int. Conf. on Verification, Model
Checking and Abstract Interpretation (VMCAI’04), volume 2937 of LNCS,
[110] J. Hillston. Fluid flow approximation of PEPA models. In Proc. Conf. on
Quantitative Evaluation of SysTems (QEST), pages 33–43. IEEE, 2005.
[111] P. Hilton, D. Holton, and J. Pedersen. Mathematical Reflections. Springer-
Verlag, New York, 1997.
[112] A. Hinton, M. Kwiatkowska, G. Norman, and D. Parker. PRISM: A tool for
automatic verification of probabilistic systems. In Proc. Conf. on Tools and
Algorithms for the Construction and Analysis of Systems (TACAS), volume
3920 of LNCS, pages 441–444. Springer, 2006.
[113] A. Hinton, M. Z. Kwiatkowska, G. Norman, and D. Parker. PRISM: A tool
for automatic verification of probabilistic systems. In Proc. Conf. on Tools
and Algorithms for the Construction and Analysis of Systems (TACAS), vol-
ume 3920 of LNCS, pages 441–444. Springer, 2006.
[114] G. Holzmann. The Spin Model Checker, Primer and Reference Manual.
Addison-Wesley, 2003.
[115] J. Hromkovic, R. Klasing, B. Monien, and R. Peine. Dissemination of in-
formation in interconnection networks (Broadcasting & Gossiping). Com-
binatorial Network Theory, pages 125–212, 1996.
[116] M. Huth and M. Ryan. Logic in Computer Science: Modelling and reasoning
about systems. Cambridge University Press, Cambridge, UK, 2004.
[117] IEEE 802.11 WG. Part 11: Wireless LAN Medium Access Control (MAC) and
Physical Layer (PHY) specification. IEEE, 1999.
BIBLIOGRAPHY 163
[118] K. Iwanicki. Gossip-based dissemination of time. Master’s thesis, Warsaw

University and Vrije Universiteit Amsterdam, 2005.
[119] K. Iwanicki, M. van Steen, and S. Voulgaris. Gossip-based clock syn-
chronization for large decentralized systems. In Proc. Workshop on Self-
Managed Networks, Systems and Services, volume 3996 of LNCS, pages
28–42. Springer, 2006.
[120] M. Jelasity, W. Kowalczyk, and M. van Steen. Newscast computing. Tech-
nical Report IR-CS-006, Vrije Universiteit Amsterdam, 2003.
[121] M. Jelasity, A. Montresor, and O. Babaoglu. Gossip-based aggregation in
large dynamic networks. ACM Transactions on Comput. Syst., 23(3):219–
252, 2005.
[122] M. Jelasity, A. Montresor, G. P. Jesi, and S. Voulgaris. PeerSim: A peer-to-
peer simulator. http://peersim.sourceforge.net/.
[123] M. Jelasity, S. Voulgaris, R. Guerraoui, A.-M. Kermarrec, and M. van Steen.
Gossip-based peer sampling. ACM Transactions on Comput. Syst., 25(3):8,
2007.
[124] W. Kang, F. Kelly, N. Lee, and R. Williams. Fluid and Brownian approxima-
tions for an internet congestion control model. In Proc. Conf. on Decision
and Control, volume 4, pages 3938–3943, 2004.
[125] R. Karp, C. Schindelhauer, S. Shenker, and B. Vocking. Randomized rumor
spreading. In Proc. IEEE Symp. on Foundations of Computer Science (FOCS),
pages 565–574. IEEE Computer Society, 2000.
[126] F. Karpelevich, E. Pechersky, and Y. Suhov. Dobrushin’s approach to queue-
ing network theory. J. of Applied Mathematics and Stochastic Analysis,
9(4):373–397, 1996.
[127] J.-P. Katoen, T. Kemna, I. Zapreev, and D. Jansen. Bisimulation minimi-

sation mostly speeds up probabilistic model checking. In Proc. Conf. on
Tools and Algorithms for the Construction and Analysis of Systems (TACAS),
volume 4424 of LNCS, pages 76–92. Springer, 2007.
[128] J.-P. Katoen, D. Klink, M. Leucker, and V. Wolf. Three-valued abstraction
for continuous-time Markov chains. In Proc. Conf. on Computer Aided Ver-
ification (CAV), volume 4590 of LNCS, pages 311–324. Springer, 2007.
[129] D. Kempe, A. Dobra, and J. Gehrke. Gossip-based computation of aggre-
gate information. In Proc. IEEE Symp. on Foundations of Computer Science
(FOCS), pages 482–491. IEEE Computer Society, 2003.
164 BIBLIOGRAPHY
[130] D. Kempe, J. Kleinberg, and A. Demers. Spatial gossip and resource lo-
cation protocols. In Proc. ACM Symp. on Theory of Computing (STOC),
pages 163–172. ACM Press, 2001.
[131] D. Kempe and J. M. Kleinberg. Protocols and impossibility results for
gossip-based communication mechanisms. In Proc. IEEE Symp. on Founda-
tions of Computer Science (FOCS), pages 471–480. IEEE Computer Society,
2002.
[132] A.-M. Kermarrec and M. van Steen. Gossiping in distributed systems. ACM
SIGOPS Oper. Syst. Rev., 41(5):2–7, 2007.
[133] S. Y. Ko, I. Gupta, and Y. Jo. A new class of nature-inspired algorithms
for self-adaptive peer-to-peer computing. ACM Trans. Auton. Adapt. Syst.,
3(3):1–34, 2008.
[134] S. Kumar and L. Massoulié. Integrating streaming and file-transfer inter-
net traffic: fluid and diffusion approximations. Queueing Syst. Theory &
Appl., 55(4):195–205, 2007.
[135] T. G. Kurtz. The relationship between stochastic and deterministic models
for chemical reactions. J. Chem. Phys., 57:2976–2978, 1972.
[136] T. G. Kurtz. Approximation of population processes, volume 36 of CBMS-
NSF Regional Conference Series in Applied Mathematics. SIAM, 1981.
[137] M. Kwiatkowska, G. Norman, and D. Parker. Game-based abstraction for
markov decision processes. In Proc. Conf. on Quantitative Evaluation of
SysTems (QEST), pages 157–166. IEEE Computer Society, 2006.
[138] M. Kwiatkowska, G. Norman, and D. Parker. Analysis of a gossip protocol
in PRISM. ACM SIGMETRICS Performance Evaluation Review, 36(3):17–22,
2008.
[139] S. Laplante, R. Lassaigne, F. Magniez, S. Peyronnet, and M. de Rouge-

mont. Probabilistic abstraction for model checking: An approach based
on property testing. In Proc. IEEE Symp. on Logic in Comput. Sci. (LICS
2002), pages 30–39. IEEE Computer Society, 2002.
[140] M.-J. Lin and K. Marzullo. Directional gossip: Gossip in a wide area
network. In Proc. European Dependable Computing Conf. (EDCC), volume
1667 of LNCS, pages 364–379. Springer, 1999.
[141] J. Luo, P. T. Eugster, and J.-P. Hubaux. Route driven gossip: Probabilistic
reliable multicast in ad hoc networks. In Proc. Conf. of Computer and
Communications Societies (INFOCOM), volume 3, pages 2229–2239. IEEE
Computer Society, 2003.
BIBLIOGRAPHY 165
[142] A. Martinoli, K. Easton, and W. Agassounon. Modeling Swarm Robotic

Systems: A Case Study in Collaborative Distributed Manipulation. Int.
Journal of Robotics Research, 23(4):415–436, 2004.
[143] S. McCanne and S. Floyd. ns Network Simulator. Available at
http://www.isi.edu/nsnam/ns/.
[144] A. McIver and A. Fehnker. Formal techniques for the analysis of wireless
networks. In Proc. Symp. on Leveraging Applications of Formal Methods,
Verification and Validation (ISoLA), 2006.
[145] G. J. McLachlan and D. Peel. Finite Mixture Models. John Wiley & Sons,
2000.
[146] T. Meyer and C. Tschudin. Chemical networking protocols. In Proc. ACM
Workshop on Hot Topics in Networks (HotNets), 2009.
[147] M. Mitzenmacher. The power of two choices in randomized load balanc-
ing. IEEE Transactions Parallel Distrib. Syst., 12(10):1094–1104, 2001.
[148] D. Mollison, editor. Epidemic Models: Their Structure and Relation to Data.
Cambridge University Press, 1995.
[149] D. Mosk-Aoyama and D. Shah. Computing separable functions via gossip.
In Proc. Symp. on Principles of Distributed Computing (PODC), pages 113–
122. ACM Press, 2006.
[150] J. Murray. Mathematical Biology, volume I: An Introduction. Springer-
Verlag, 2003.
[151] T. Nipkow, L. C. Paulson, and M. Wenzel. Isabelle/HOL: A Proof Assistant
for Higher-Order Logic, volume 2283 of LNCS. Springer, 2002.
[152] G. Norman, D. Parker, M. Z. Kwiatkowska, S. K. Shukla, and R. Gupta. Us-
ing probabilistic model checking for dynamic power management. Formal
Aspects of Computing, 17(2):160–176, 2005.
[153] OPNET Technologies. OPNET modeler. http://www.opnet.com/.
[154] M. Opper and D. Saad, editors. Advanced Mean Field Methods: Theory and
Practice. MIT Press, 2001.
[155] S. Owre, S. Rajan, J. M. Rushby, N. Shankar, and M. K. Srivas. PVS:
Combining specification, proof checking, and model checking. In Proc.
Conf. on Computer Aided Verification (CAV), volume 1102 of LNCS, pages
411–414. Springer, 1996.
166 BIBLIOGRAPHY
[156] O. Ozkasap, M. Caglar, E. Cem, E. Ahi, and E. Iskender. Clock synchroniza-

tion for wireless sensor networks: a survey. Ad Hoc Networks, 3(3):281–
323, 2005.
[157] M. Petkovšek, H. S. Wilf, and D. Zeilberger. A=B. AK Peters, 1996.
[158] J. Pouwelse, P. Garbacki, J. Wang, A. Bakker, J. Yang, A. Iosup, D. Epema,
M.Reinders, M. van Steen, and H. Sips. Tribler: A social-based peer-to-
peer system. Concurrency and Computation: Practice and Experience, 19:1–
11, 2007.
[159] Programming language Scala. http://www.scala-lang.org/.
[160] M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic
Programming. John Wiley & Sons, Inc., New York, NY, USA, 1994.
[161] D. Randall. Rapidly mixing Markov chains with applications in computer
science and physics. Computing in Sci. and Eng., 8(2):30–41, 2006.
[162] A. Remke, B. R. Haverkort, and L. Cloth. Model checking infinite-state
Markov chains. In Proc. Conf. on Tools and Algorithms for the Construction
and Analysis of Systems (TACAS), volume 3440 of LNCS, pages 237–252.
Springer, 2005.
[163] R.Graham, D. Knuth, and O. Potashnik. Concrete Mathematics. Addison-
Wesley, 2nd edition, 1994.
[164] L. Sattenspiel. The geographic spread of infectious diseases. Princeton Se-
ries in Theoretical and Computational Biology. Princeton University Press,
2009.
[165] K. Sen, M. Viswanathan, and G. Agha. Statistical model checking of black-
box probabilistic systems. In Proc. Conf. on Computer Aided Verification
(CAV), volume 3114 of LNCS, pages 202–215. Springer, 2004.
[166] H. G. Solari and M. A. Natiello. Poisson approximation to density depen-
dent stochastic processes: A numerical implementation and test. In Proc.
Workshop on Dynamical Systems from Number Theory to Probability-II, vol-
ume 6 of Math. Modelling, pages 79–94. Växjö Univ. Press, 2003.
[167] A. Stavrou, D. Rubenstein, and S. Sahu. A lightweight, robust P2P system
to handle flash crowds. In Proc. Conf. on Network Protocols (ICNP), pages
226–235. IEEE Computer Society, 2002.
[168] A. Stefanek, R. Hayden, and J. T. Bradley. A new tool for the perfor-
mance analysis of massively parallel computer systems. In Proc. Workshop
on Quantitative Aspects of Programming Languages (QAPL), volume 28 of
EPTCS, pages 159–181. Elsevier, 2010.
BIBLIOGRAPHY 167
[169] I. Stojanovic, M. Sharif, and D. Starobinski. Data dissemination in wireless

broadcast channels: Network coding or cooperation. In Proc. Conf. on
Information Sciences and Systems (CISS), pages 265–270. IEEE, 2007.
[170] A. Tanenbaum and M. van Steen. Distributed Systems: Principles and
Paradigms. Prentice Hall, 2nd edition, 2007.
[171] P. Tinnakornsrisuphap and A. Makowski. Limit behavior of ECN/RED
gateways under a large number of TCP flows. In Proc. Conf. of Com-
puter and Communications Societies (INFOCOM), volume 2, pages 873–
883. IEEE, 2003.
[172] M. Umlauft and P. Reichl. Experiences with the ns-2 network simulator -
explicitly setting seeds considered harmful. In Proc. of Wireless Telecom-
munications Symposium (WTS), pages 1–5, 2007.
[173] R. van Renesse, K. P. Birman, and W. Vogels. Astrolabe: A robust and
scalable technology for distributed system monitoring, management, and
data mining. ACM Transactions on Comput. Syst., 21(2):164–206, 2003.
[174] R. van Renesse, Y. Minsky, and M. Hayden. A gossip-style failure detection
service. In Proc. Conf. on Distributed Syst. Platforms and Open Distributed
Processing (Middleware), pages 55–70. Springer, 1998.
[175] W. Vogels, R. van Renesse, and K. Birman. The power of epidemics: ro-
bust communication for large-scale distributed systems. ACM SIGCOMM
Comput. Commun. Review, 33(1):131–135, 2003.
[176] S. Voulgaris. Epidemic-Based Self-Organization in Peer-to-Peer Systems.
Doctoral thesis, Vrije Universiteit Amsterdam, 2006.
[177] S. Voulgaris, D. Gavidia, and M. van Steen. Cyclon: Inexpensive member-
ship management for unstructured P2P overlays. Journal of Network and
Systems Management, 13(2):197–217, 2005.
[178] N. Vvedenskaya and Y. Suhov. Dobrushin’s mean-field approximation for
a queue with dynamic routing. Markov Proc. Rel. Fields, 3(4):493–526,
1997.
[179] S. Weber, V. Veeraraghavan, A. Kini, and N. Singhal. Analysis of gossip
performance with copulas. In Proc. Conf. on Information Sciences and Sys-
tems (CISS), pages 1212–1217, 2006.
[180] H. Younes, M. Kwiatkowska, G. Norman, and D. Parker. Numerical vs. sta-
tistical probabilistic model checking. Journal on Software Tools for Tech-
nology Transfer (STTT), 8(3):216–228, 2006.
168 BIBLIOGRAPHY
[181] H. L. S. Younes and R. G. Simmons. Probabilistic verification of discrete

event systems using acceptance sampling. In Proc. Conf. on Computer Aided
Verification (CAV), volume 2404 of LNCS, pages 223–235. Springer, 2002.
[182] X. Zeng, R. Bagrodia, and M. Gerla. GloMoSim. Available at
http://pcl.cs.ucla.edu/projects/glomosim/.
[183] X. Zeng, R. Bagrodia, and M. Gerla. GloMoSim: a library for parallel
simulation of large-scale wireless networks. In Proc. Workshop on Parallel
and Distributed Simulation (PADS), pages 154–161. IEEE, 1998.
[184] Q. Zhang and D. Agrawal. Dynamic probabilistic broadcasting in MANETs.
J. of Parallel and Distributed Computing, 65(2):220–233, 2005.
[185] X. Zhang, G. Neglia, J. Kurose, and D. Towsley. Performance modeling of
epidemic routing. Computer Networks, 51(10):2867–2891, 2007.
Titles in the IPA Dissertation Series since 2006
E. Dolstra. The Purely Functional Soft- G. Russello. Separation and Adap-

ware Deployment Model. Faculty of tation of Concerns in a Shared Data
Science, UU. 2006-01 Space. Faculty of Mathematics and
R.J. Corin. Analysis Models for Secu- Computer Science, TU/e. 2006-11
rity Protocols. Faculty of Electrical En- L. Cheung. Reconciling Nondetermin-
gineering, Mathematics & Computer istic and Probabilistic Choices. Faculty
Science, UT. 2006-02 of Science, Mathematics and Com-
P.R.A. Verbaan. The Computational puter Science, RU. 2006-12
Complexity of Evolving Systems. Fac- B. Badban. Verification techniques for
ulty of Science, UU. 2006-03 Extensions of Equality Logic. Faculty
K.L. Man and R.R.H. Schiffelers. of Sciences, Department of Computer
Formal Specification and Analysis of Science, VUA. 2006-13
Hybrid Systems. Faculty of Mathemat- A.J. Mooij. Constructive formal meth-
ics and Computer Science and Fac- ods and protocol standardization. Fac-
ulty of Mechanical Engineering, TU/e. ulty of Mathematics and Computer
2006-04 Science, TU/e. 2006-14
M. Kyas. Verifying OCL Specifications
T. Krilavicius. Hybrid Techniques for
of UML Models: Tool Support and Com-
Hybrid Systems. Faculty of Electri-
positionality. Faculty of Mathematics
cal Engineering, Mathematics & Com-
and Natural Sciences, UL. 2006-05
puter Science, UT. 2006-15
M. Hendriks. Model Checking Timed
M.E. Warnier. Language Based Secu-
Automata - Techniques and Applica-
rity for Java and JML. Faculty of Sci-
tions. Faculty of Science, Mathematics
ence, Mathematics and Computer Sci-
and Computer Science, RU. 2006-06
ence, RU. 2006-16
J. Ketema. Böhm-Like Trees for
Rewriting. Faculty of Sciences, De- V. Sundramoorthy. At Home In Ser-
partment of Computer Science, VUA. vice Discovery. Faculty of Electrical En-
2006-07 gineering, Mathematics & Computer
Science, UT. 2006-17
C.-B. Breunesse. On JML: topics in
tool-assisted verification of JML pro- B. Gebremichael. Expressivity of
grams. Faculty of Science, Mathemat- Timed Automata Models. Faculty of
ics and Computer Science, RU. 2006- Science, Mathematics and Computer
08 Science, RU. 2006-18
B. Markvoort. Towards Hybrid Molec- L.C.M. van Gool. Formalising Inter-
ular Simulations. Faculty of Biomedi- face Specifications. Faculty of Mathe-
cal Engineering, TU/e. 2006-09 matics and Computer Science, TU/e.
2006-19
S.G.R. Nijssen. Mining Structured
Data. Faculty of Mathematics and C.J.F. Cremers. Scyther - Semantics
Natural Sciences, UL. 2006-10 and Verification of Security Protocols.
Faculty of Mathematics and Computer N. Trčka. Silent Steps in Transition
Science, TU/e. 2006-20 Systems and Markov Chains. Faculty of
Mathematics and Computer Science,
J.V. Guillen Scholten. Mobile Chan-
TU/e. 2007-08
nels for Exogenous Coordination of Dis-
tributed Systems: Semantics, Imple- R. Brinkman. Searching in encrypted
mentation and Composition. Faculty data. Faculty of Electrical Engineer-
of Mathematics and Natural Sciences, ing, Mathematics & Computer Sci-
UL. 2006-21 ence, UT. 2007-09
H.A. de Jong. Flexible Heterogeneous A. van Weelden. Putting types to good
Software Systems. Faculty of Natural use. Faculty of Science, Mathematics
Sciences, Mathematics, and Computer and Computer Science, RU. 2007-10
Science, UvA. 2007-01 J.A.R. Noppen. Imperfect Informa-
N.K. Kavaldjiev. A run-time reconfig- tion in Software Development Pro-
urable Network-on-Chip for streaming cesses. Faculty of Electrical Engineer-
DSP applications. Faculty of Electri- ing, Mathematics & Computer Sci-
cal Engineering, Mathematics & Com- ence, UT. 2007-11
puter Science, UT. 2007-02 R. Boumen. Integration and Test
M. van Veelen. Considerations on plans for Complex Manufacturing Sys-
Modeling for Early Detection of Ab- tems. Faculty of Mechanical Engineer-
normalities in Locally Autonomous Dis- ing, TU/e. 2007-12
tributed Systems. Faculty of Mathe- A.J. Wijs. What to do Next?: Analysing
matics and Computing Sciences, RUG. and Optimising System Behaviour in
2007-03 Time. Faculty of Sciences, Department
T.D. Vu. Semantics and Applications of of Computer Science, VUA. 2007-13
Process and Program Algebra. Faculty C.F.J. Lange. Assessing and Improv-
of Natural Sciences, Mathematics, and ing the Quality of Modeling: A Series
Computer Science, UvA. 2007-04 of Empirical Studies about the UML.
L. Brandán Briones. Theories for Faculty of Mathematics and Computer
Model-based Testing: Real-time and Science, TU/e. 2007-14
Coverage. Faculty of Electrical En- T. van der Storm. Component-
gineering, Mathematics & Computer based Configuration, Integration and
Science, UT. 2007-05 Delivery. Faculty of Natural Sci-
I. Loeb. Natural Deduction: Sharing ences, Mathematics, and Computer
by Presentation. Faculty of Science, Science,UvA. 2007-15
Mathematics and Computer Science, B.S. Graaf. Model-Driven Evolution of
RU. 2007-06 Software Architectures. Faculty of Elec-
trical Engineering, Mathematics, and
M.W.A. Streppel. Multifunctional Ge-
Computer Science, TUD. 2007-16
ometric Data Structures. Faculty of
Mathematics and Computer Science, A.H.J. Mathijssen. Logical Calculi for
TU/e. 2007-07 Reasoning with Binding. Faculty of
Mathematics and Computer Science, of Optimistic Fair Exchange Protocols.
TU/e. 2007-17 Faculty of Sciences, Department of
Computer Science, VUA. 2008-07
D. Jarnikov. QoS framework for Video
Streaming in Home Networks. Faculty I.S.M. de Jong. Integration and Test
of Mathematics and Computer Sci- Strategies for Complex Manufacturing
ence, TU/e. 2007-18 Machines. Faculty of Mechanical Engi-
neering, TU/e. 2008-08
M. A. Abam. New Data Structures and
Algorithms for Mobile Data. Faculty of I. Hasuo. Tracing Anonymity with
Mathematics and Computer Science, Coalgebras. Faculty of Science, Math-
TU/e. 2007-19 ematics and Computer Science, RU.
2008-09
W. Pieters. La Volonté Machinale: Un-
derstanding the Electronic Voting Con- L.G.W.A. Cleophas. Tree Algorithms:
troversy. Faculty of Science, Math- Two Taxonomies and a Toolkit. Fac-
ematics and Computer Science, RU. ulty of Mathematics and Computer
2008-01 Science, TU/e. 2008-10
I.S. Zapreev. Model Checking Markov
A.L. de Groot. Practical Automa-
Chains: Techniques and Tools. Faculty
ton Proofs in PVS. Faculty of Science,
of Electrical Engineering, Mathemat-
ics & Computer Science, UT. 2008-11
RU. 2008-02
M. Farshi. A Theoretical and Exper-
M. Bruntink. Renovation of Id-
imental Study of Geometric Networks.
iomatic Crosscutting Concerns in Em-
Faculty of Mathematics and Computer
bedded Systems. Faculty of Electrical
Science, TU/e. 2008-12
Engineering, Mathematics, and Com-
puter Science, TUD. 2008-03 G. Gulesir. Evolvable Behavior Speci-
fications Using Context-Sensitive Wild-
A.M. Marin. An Integrated System cards. Faculty of Electrical Engineer-
to Manage Crosscutting Concerns in ing, Mathematics & Computer Sci-
Source Code. Faculty of Electrical ence, UT. 2008-13
Engineering, Mathematics, and Com-
puter Science, TUD. 2008-04 F.D. Garcia. Formal and Computa-
tional Cryptography: Protocols, Hashes
N.C.W.M. Braspenning. Model-based and Commitments. Faculty of Science,
Integration and Testing of High-tech Mathematics and Computer Science,
Multi-disciplinary Systems. Faculty of RU. 2008-14
Mechanical Engineering, TU/e. 2008-
P. E. A. Dürr. Resource-based Verifi-
05
cation for Robust Composition of As-
M. Bravenboer. Exercises in Free Syn- pects. Faculty of Electrical Engineer-
tax: Syntax Definition, Parsing, and ing, Mathematics & Computer Sci-
Assimilation of Language Conglomer- ence, UT. 2008-15
ates. Faculty of Science, UU. 2008-06
E.M. Bortnik. Formal Methods in Sup-
M. Torabi Dashti. Keeping Fairness port of SMC Design. Faculty of Me-
Alive: Design and Formal Verification chanical Engineering, TU/e. 2008-16
R.H. Mak. Design and Performance J. Markovski. Real and Stochastic
Analysis of Data-Independent Stream Time in Process Algebras for Perfor-
Processing Systems. Faculty of Math- mance Evaluation. Faculty of Mathe-
ematics and Computer Science, TU/e. matics and Computer Science, TU/e.
2008-17 2008-26
M. van der Horst. Scalable Block Pro- H. Kastenberg. Graph-Based Software
cessing Algorithms. Faculty of Mathe- Specification and Verification. Faculty
matics and Computer Science, TU/e. of Electrical Engineering, Mathemat-
2008-18 ics & Computer Science, UT. 2008-27
C.M. Gray. Algorithms for Fat Ob- I.R. Buhan. Cryptographic Keys
jects: Decompositions and Applications. from Noisy Data Theory and Applica-
Faculty of Mathematics and Computer tions. Faculty of Electrical Engineer-
Science, TU/e. 2008-19 ing, Mathematics & Computer Sci-
ence, UT. 2008-28
J.R. Calamé. Testing Reactive Systems
with Data - Enumerative Methods and R.S. Marin-Perianu. Wireless Sensor
Constraint Solving. Faculty of Electri- Networks in Motion: Clustering Algo-
cal Engineering, Mathematics & Com- rithms for Service Discovery and Pro-
puter Science, UT. 2008-20 visioning. Faculty of Electrical En-
gineering, Mathematics & Computer
E. Mumford. Drawing Graphs for Science, UT. 2008-29
Cartographic Applications. Faculty of
M.H.G. Verhoef. Modeling and Vali-
dating Distributed Embedded Real-Time
TU/e. 2008-21
Control Systems. Faculty of Science,
E.H. de Graaf. Mining Semi- Mathematics and Computer Science,
structured Data, Theoretical and Exper- RU. 2009-01
imental Aspects of Pattern Evaluation. M. de Mol. Reasoning about Func-
Faculty of Mathematics and Natural tional Programs: Sparkle, a proof as-
Sciences, UL. 2008-22 sistant for Clean. Faculty of Science,
R. Brijder. Models of Natural Compu- Mathematics and Computer Science,
tation: Gene Assembly and Membrane RU. 2009-02
Systems. Faculty of Mathematics and M. Lormans. Managing Requirements
Natural Sciences, UL. 2008-23 Evolution. Faculty of Electrical Engi-
A. Koprowski. Termination of Rewrit- neering, Mathematics, and Computer
ing and Its Certification. Faculty of Science, TUD. 2009-03
Mathematics and Computer Science, M.P.W.J. van Osch. Automated
TU/e. 2008-24 Model-based Testing of Hybrid Systems.
Faculty of Mathematics and Computer
U. Khadim. Process Algebras for Hy-
brid Systems: Comparison and Devel-
opment. Faculty of Mathematics and H. Sozer. Architecting Fault-Tolerant
Computer Science, TU/e. 2008-25 Software Systems. Faculty of Electri-
cal Engineering, Mathematics & Com- ing, Mathematics & Computer Sci-
puter Science, UT. 2009-05 ence, UT. 2009-14
M.J. van Weerdenburg. Efficient H.L. Jonker. Security Matters: Pri-
Rewriting Techniques. Faculty of Math- vacy in Voting and Fairness in Digital
ematics and Computer Science, TU/e. Exchange. Faculty of Mathematics and
2009-06 Computer Science, TU/e. 2009-15
H.H. Hansen. Coalgebraic Modelling: M.R. Czenko. TuLiP - Reshaping Trust
Applications in Automata Theory and Management. Faculty of Electrical En-
Modal Logic. Faculty of Sciences, De- gineering, Mathematics & Computer
partment of Computer Science, VUA. Science, UT. 2009-16
2009-07
T. Chen. Clocks, Dice and Processes.
A. Mesbah. Analysis and Testing of Faculty of Sciences, Department of
Ajax-based Single-page Web Applica- Computer Science, VUA. 2009-17
tions. Faculty of Electrical Engineer-
C. Kaliszyk. Correctness and Avail-
ing, Mathematics, and Computer Sci-
ability: Building Computer Algebra on
ence, TUD. 2009-08
top of Proof Assistants and making
A.L. Rodriguez Yakushev. Towards Proof Assistants available over the Web.
Getting Generic Programming Ready Faculty of Science, Mathematics and
for Prime Time. Faculty of Science, Computer Science, RU. 2009-18
UU. 2009-9
R.S.S. O’Connor. Incompleteness &
K.R. Olmos Joffré. Strategies for Con- Completeness: Formalizing Logic and
text Sensitive Program Transformation. Analysis in Type Theory. Faculty of Sci-
Faculty of Science, UU. 2009-10 ence, Mathematics and Computer Sci-
ence, RU. 2009-19
J.A.G.M. van den Berg. Reasoning
about Java programs in PVS using JML. B. Ploeger. Improved Verification
Faculty of Science, Mathematics and Methods for Concurrent Systems. Fac-
Computer Science, RU. 2009-11 ulty of Mathematics and Computer
M.G. Khatib. MEMS-Based Stor-
age Devices. Integration in Energy- T. Han. Diagnosis, Synthesis and Anal-
Constrained Mobile Systems. Faculty of ysis of Probabilistic Models. Faculty of
Electrical Engineering, Mathematics & Electrical Engineering, Mathematics &
Computer Science, UT. 2009-12 Computer Science, UT. 2009-21
S.G.M. Cornelissen. Evaluating Dy- R. Li. Mixed-Integer Evolution Strate-
namic Analysis Techniques for Program gies for Parameter Optimization and
Comprehension. Faculty of Electrical Their Applications to Medical Image
Engineering, Mathematics, and Com- Analysis. Faculty of Mathematics and
puter Science, TUD. 2009-13 Natural Sciences, UL. 2009-22
D. Bolzoni. Revisiting Anomaly- J.H.P. Kwisthout. The Computational
based Network Intrusion Detection Sys- Complexity of Probabilistic Networks.
tems. Faculty of Electrical Engineer- Faculty of Science, UU. 2009-23
T.K. Cocx. Algorithmic Tools for Data- Y. Wang. Epistemic Modelling and Pro-
Oriented Law Enforcement. Faculty tocol Dynamics. Faculty of Science,
of Mathematics and Natural Sciences, UvA. 2010-05
UL. 2009-24
J.K. Berendsen. Abstraction, Prices
A.I. Baars. Embedded Compilers. Fac- and Probability in Model Checking
ulty of Science, UU. 2009-25 Timed Automata. Faculty of Science,
M.A.C. Dekker. Flexible Access Con-
RU. 2010-06
trol for Dynamic Collaborative Envi-
ronments. Faculty of Electrical En- A. Nugroho. The Effects of UML Mod-
gineering, Mathematics & Computer eling on the Quality of Software. Fac-
Science, UT. 2009-26 ulty of Mathematics and Natural Sci-
ences, UL. 2010-07
J.F.J. Laros. Metrics and Visualisation
for Crime Analysis and Genomics. Fac- A. Silva. Kleene Coalgebra. Faculty of
ulty of Mathematics and Natural Sci- Science, Mathematics and Computer
ences, UL. 2009-27 Science, RU. 2010-08
C.J. Boogerd. Focusing Automatic J.S. de Bruin. Service-Oriented Discov-
Code Inspections. Faculty of Electrical ery of Knowledge - Foundations, Imple-
Engineering, Mathematics, and Com- mentations and Applications. Faculty
puter Science, TUD. 2010-01 of Mathematics and Natural Sciences,
UL. 2010-09
M.R. Neuhäußer. Model Checking
Nondeterministic and Randomly Timed D. Costa. Formal Models for Com-
Systems. Faculty of Electrical En- ponent Connectors. Faculty of Sci-
gineering, Mathematics & Computer ences, Department of Computer Sci-
Science, UT. 2010-02 ence, VUA. 2010-10
J. Endrullis. Termination and Pro- M.M. Jaghoori. Time at Your Ser-
ductivity. Faculty of Sciences, De- vice: Schedulability Analysis of Real-
partment of Computer Science, VUA. Time and Distributed Services. Faculty
2010-03 of Mathematics and Natural Sciences,
UL. 2010-11
T. Staijen. Graph-Based Specification
and Verification for Aspect-Oriented R. Bakhshi. Gossiping Models: Formal
Languages. Faculty of Electrical En- Analysis of Epidemic Protocols. Faculty
gineering, Mathematics & Computer of Sciences, Department of Computer
Science, UT. 2010-04 Science, VUA. 2011-01

Gossiping Theory

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Gossiping Theory

Загружено:

Авторское право:

Доступные форматы

Gossiping Models

Formal Analysis of Epidemic Protocols

ter verkrijging van de graad Doctor aan

geboren te Baki, Azerbeidzjan

3 Pairwise Exchange Model without Failures 23

4 Pairwise Exchange Model in Lossy Networks 45

5 Comparison of Performance Models 59

7 Automating the Mean-Field for Dynamic Networks 117

8 Concluding Remarks 145

9 Summary of the Thesis 151

1.1 Requirements for Gossip Protocols

1.1.1 General Requirements

1.1.2 Functional Requirements

1.1.3 Nonfunctional Requirements

1.2 Formal Analysis Techniques

which formal analysis techniques can, in principle, be employed efficiently in

Figure 1.1: Spectrum of validation

1.3 Experimental Evaluation

1.4 Model-Based Analysis Techniques

numerical approximations. On the other hand, theorem prover requires a lot of

1.4.1 Rigorous Mathematical Analysis

Hand-crafted Markov chains

1.4.2 Model Checking

probabilistic systems). The system specification language can, for example, be

Probabilistic and Stochastic Model Checking

Approximate and Statistical Probabilistic Model Checking

tems against specifications given in a sub-logic of CSL.

1.7 Organization of the Thesis

Figure 1.2: Pictorial representation of the thesis structure

Chapter 2: Background on Gossip Protocols. The next chapter introduces a

Chapter 3: Pairwise Exchange Model without Failures. This chapter ana-

Chapter 4: Pairwise Exchange Model in Lossy Networks. This chapter ex-

Chapter 5: Comparison of Performance Models. This chapter presents and

Chapter 6: Mean-field Framework. This chapter demonstrates that the exist-

Chapter 7: Automating the Mean-Field for Dynamic Networks. This chap-

Finally, Chapter 8: Concluding Remarks summarizes the main results of this

study in this thesis, is presented in Chapter 2. Together with Chapters 1 and 2,

1.8 Origins of the Chapters

2.1 A Generic Gossip Protocol

is based on the set of neighbours as determined by a membership protocol (e.g.,

2.2 A Gossip Protocol for Information Dissemination

2.2.1 Protocol Assumptions

2.2.2 Protocol Description

while true do while true do

(a) initiating node (b) contacted node

Figure 2.3: Illustration of the shuffle algorithm with n = 8, c = 5 and s = 3.

3.3 Experimental Observations

random selection of items to gossip and to keep in local storage. By applying

round 100 round 150

round 200 round 25

round 300 round 350

3.4 An Analytical Model of Information Dissemination

3.4.1 Probabilities of State Transitions

P (a2 b2 |a1 b1 ) = P (b2 a2 |b1 a1 ).

P (10|01) = Pselect · Pdrop

P (11|01) = Pselect · P¬drop

a2 b2 = 00: cannot occur as completely discarding d is not possible in the pro-

P (a2 b2 |10) = P (b2 a2 |01).

P (01|11) = Pselect · Pdrop · P¬select

a2 b2 = 10: this outcome is symmetric to the previous one:

P (10|11) = P¬select · Pselect · Pdrop

a2 b2 = 11: after the shuffle both nodes A and B have d, because:

P (11|11) = P¬select · P¬select + Pselect · Pselect + 2 · Pselect · P¬select · P¬drop