Вы находитесь на странице: 1из 31

application VMs p2p

?
?

network SDR NFV

Enabling Innovation in Networking


Can we escape a future of unstructured applications unable
to move beyond the abstract black-box ‘internet’?

A report for the course Advanced Topics in Computer Networks,


taught by Robert Meijer at the University of Amsterdam.

July 2015
Enabling Innovation in Networking

Contents
1 Introduction 2

2 Families of innovations 3
2.1 Network protocol innovations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Distributed applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3 Distributed systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Qualities of networks 10

4 Network programming languages 12

5 Mobile network operator challenges 14

6 Conclusion 19

A Implementing an OpenFlow controller 20

B Bibliography 30

1
Enabling Innovation in Networking

1. Introduction
Current innovations in networking occur largely at two distinct layers; the application layer, and
the network layer. Applications simply use the internet as an abstract ‘black-box’, depending
on the location transparency and global address plan it provides, and trusting the network to
handle the details.
This abstraction limits the ability of network providers, and in particular mobile network oper-
ators, to innovate; technologies like software-defined networking and network function virtual-
ization allow them to run their networks more efficiently, but most applications are unable to
take advantage of the further opportunities these might provide.
How can we improve upon this situation? This report discusses how we might approach the
design of systems and technologies which would enable ‘cross-layer’ innovations, ideally allowing
application engineers to break free of their abstraction-induced bonds.
We’ll start by discussing families of innovations which currently exist at three different levels –
network protocols, distributed applications, and distributed systems. Then, with an eye to the
future, we’ll discuss the qualities we might want to optimise for, and discuss an innovation (a
network programming language) which might reach across these layers.
Finally, we’ll consider the situation in the context of a mobile network operator; such operators
must continuously innovate to keep up with advancing technological requirements driven by user
demands, while chasing a diminishing revenue stream due to abstraction driving them towards
the role of merely being a ‘dumb pipe’.
In the appendix, you’ll find a report on the first practical assignment of the course, discussing the
development of an OpenFlow network controller, as a simple demonstration of the possibilities
of software defined networking.

2
Enabling Innovation in Networking

Figure 2.1: What is innovation? Figure 2.2: Families of “network” innovations

2. Families of innovations
Before we can try to organise network protocols and systems into families of innovations, we
must first decide what we mean by ‘innovation’. Can we simply declare newer, more modern
protocols as innovative?
A further question which immediately springs to mind is to ask whether “innovation” is neces-
sarily a good thing. Looking at Figure 2.1, consider the question of centralised vs de-centralised
name lookup (/directory) services. DNS provides a far more practical model in terms of the
modern internet, one which can be distributed and load-balanced and allows for geographical
replies. But it remains centralised, at least administratively; there is one “true” set of roots.
Could P2P be an answer, in the form of something like Namecoin [12]? While this system may
be innovative, it adds little practical value, while detracting from factors such as security which
rely on users being able to trust name services.

2.1 Network protocol innovations


Looking at network protocols specifically, many ‘innovations’ are counterproductive for higher
layers; for example, Network Address Translation is a technology which has become popular
primarily due to shortages in the address space of the internet. It both acts as a barrier to
more rapid uptake of a protocol allowing a larger address space (IPv6), and an obstacle to many
applications, which must use complicated workarounds in order to talk to other nodes on the
network (or, worse, route all traffic through an intermediary node).
However, if we look at the very technical development of physical network layers, as shown in
Figure 2.3, then we see some evidence that innovations being on the basis of time is a concept
which might well make sense, as a rough guide.
We can feel a clear pattern of innovation as we “spiral out”, easily grouped into families (perhaps
here primarily rather “genera”, falling into the “family” of physical modulation); starting from
the simple wired Manchester encoding of 10BASE-T’s low-speed wired connections, through
the use of DSSS in 802.11b and the move to the more pragmatic OFDM (essentially sacrificing
performance for diversity) with 802.11a, up to the MIMO innovations of 802.11n.

3
Enabling Innovation in Networking

Improvements to network robustness are (arguably) primarily due to advances at such low levels.
Obviously, improvements to capacity and latency truly are due to such advances.
Network protocols are tightly wound around many of these technologies, meaning many of them
suffer due to choices made at the time of their development. Even when there is theoretical
flexibility, practical considerations present problems; for example, ADSL remains often tied to
ATM, and that DVB-T in many countries cannot make use of modern video codecs such as
H.264, due to the limitations of the equipment in use [14].
How do we group “innovations” which are really just incremental improvements enabled by
advances in this physical embodiment of the protocols?
With these factors in mind, we present 4 example families in Figure 2.2. Two of these (security
and mobile) are primarily protocol innovations. The innovations for security are (hopefully)
fairly simple and obvious; TLS/SSL is somewhat along the lines of the telnet-to-ssh situation
seen in Figure 2.1, where we’re adding secure encapsulation, allowing an existing protocol (such
as HTTP) to be used in a secure way.
These kinds of security layers are slowly becoming a mandatory part of newer versions of proto-
cols. In the future, technologies such as IPSec might make this such a standard part of protocols
that this becomes universal. However, thanks to abstraction, this is largely transparent both to
applications and users.
Note that both of our two example innovations for accessibility are seemingly very non-innovative
moves. They involve taking complex protocols such as NNTP and SMTP and eliminating them
entirely; instead encapsulating their use cases (group discussion and e-mail) in websites – and
in fact turning them into distributed applications, as we’ll discuss further later.

Figure 2.3: Genera of innovations in physical modulation schemes

4
Enabling Innovation in Networking

The most exciting innovations in mobility and distributed networking can depend on protocol
innovations. Innovations in the area of mobility are a young family, with many potential innova-
tions only just becoming visible; for example, in the future, services might well migrate between
cells or individual devices, providing both mobility and distribution. Right now, the “internet
of things” makes use of protocols such as ZigBee (IEEE 802.15.4) to provide low-power mobility
applications (the most illustrative common use case is perhaps lightbulbs [17]).
But perhaps the most important family of network protocol innovations involves the protocols
controlling the administration and routing of the networks themselves, in Software Defined
Networking.
OpenFlow, for example, gives access to network switches and routers over the network itself,
allowing networks to be reconfigured (or, indeed, to reconfigure themselves) on-the-fly. These
innovations allow us to stop treating networks as black-boxes, like the generic ‘internet’, and
instead enables us to treat them as flexible components of the distributed systems which use
them.

2.2 Distributed applications


Categorisation of innovations into categories is considerably more difficult for distributed ap-
plications. We’ll consider the general trend of innovations, and then identify a few families to
which some example innovations can belong, but many developments at the application level
are driven by either lower-level network or higher-level systems advances.
As we discussed earlier, applications have moved from ‘dumb terminals’ (accessing applications
on remote machines) to being run locally on computers (often not connected to a network at
all). More recently, there has been a ‘backwards’ trend where applications were running on
remote servers and accessed through the network using web browsers, and finally, JavaScript
has meant that many parts of these applications are now run inside the web browsers, on the
local machines.

(dumb) local
terminals applications

Figure 2.5: SDN (Software Defined


Networking) allows software control
of all layers of a network, from radios
distributed web and network cards, to routers and
applications? browsers clouds of virtual machines.

Figure 2.4: Trending towards distributed applications

5
Enabling Innovation in Networking

This has allowed applications to be used by a wider audience, at first simply due to the lack of a
need for custom software – the logic was simply moved to the server. More recently, however, the
advent of JavaScript applications has meant that applications effectively distribute themselves
onto the devices being used, over the network, while they’re in use. So, one ‘family’ of innovation
in distributed applications is code migration, featuring these web applications.
It could be argued that the web is too specific a single technology for web applications themselves
to be a family of innovations, but the web browser and JavaScript model have enabled such
applications to take over to such an extent that there are now devices – for example, laptops, in
the form of ChromeBooks, and mobile phones, in the form of Firefox OS devices (and when it was
first launched, even the original iPhone) – which are designed solely to run these applications.
New technology, such as “web workers”, allow web applications to become more and more
complex, while still able to run on a wide variety of clients. As well as desktop machines and
mobile phones, consider also ‘smart’ watches and ‘entertainment’ systems in cars and airplanes.
As discussed, the examples given for accessibility in Figure 2.2 illustrate how innovation may
involve moving applications even further away from the protocol layer. The same applies as the
motivation behind the distributed ‘innovations’ given; for example, VoIP applications have been
replacing telephony (and telephony networks), and distributed applications such as Skype have
been replacing those in turn.
Relatedly, another recent enabling innovation in web browsers has been WebRTC [13], which
allows JavaScript applications running in browsers to support voice and video calls, either via a
third server or directly in ‘peer-to-peer’ fashion.
The three examples we saw in Figure 2.2 – BitTorrent, Skype and Bitcoin – for innovations in
the area of distribution pointedly all revolve around a peer-to-peer model. This is certainly a
popular ‘buzz-word’, but are these applications really distributed? Often, they aren’t; Skype
used central servers [18] (and has recently moved back to a completely traditional model) and
BitTorrent used central tracking servers for most of its lifetime.

Figure 2.6: Peer-to-peer, from a


naive viewpoint

Figure 2.7: Peer-to-peer, including the network

6
Enabling Innovation in Networking

geographically migrating
distributed between
network
location-aware elements

Figure 2.8: Innovations enabling locality

But we can certainly claim that peer-to-peer encompasses a family of innovations in distributed
applications; they may not (yet) be self-organising, but it has enabled more efficient use of
networks. One real obstacle is that such applications lack any real awareness of the network –
the chosen “peers” often end up being on the other side of the world, being unable to discover
that there are more appropriate systems (e.g. with better capacity and latency) physically close
to them.
For example, consider Figure 2.6; while the connections between the systems here might make
sense to the application, if we take a look at the real networking situation which might look like
2.7, we can see that the application is going to cause large amounts of unnecessary load on the
network, suffering from decreased capacity and latency as a result.
But a more serious problem is that these applications are all developed in an ad-hoc way. The
OSI (application) layer model and broker systems like CORBA have failed to match the needs
of modern applications, but nothing has replaced them. Applications have no structure to rest
upon for their needs, and so everyone must “reinvent the wheel” for their needs.
So, while we could claim transparency as a family of innovations, only some kinds of transparency
seem to still be realistic in current distributed applications. For example, CDNs (content distri-
bution networks) provide geographic and/or failure transparency – often working closely with
network providers, installing their own equipment in the networks of others – and automated
replication of data can be a key part of their offerings. But there no solutions for transparent
concurrency have emerged, once-popular broker-based models of object access have fallen by
the wayside, and code replication (other than in a very coarse-grained manner using ‘clouds’) is
rare.
One last family of innovations in distributed applications could instead involve interoperability.
An obvious example is web services; their interfaces are described in some formal way (such as
WSDL), and calls between services are performed in a standardised fashion (SOAP on HTTP).
Protocols such as SIP – while not innovative in themselves – also lead to creative new applications
being enabled simply by being able to communicate with a variety of different applications.
But this isn’t enough – real advances in distributed applications need a metamorphosis of the
way they form part of distributed systems.

7
Enabling Innovation in Networking

Figure 2.9: Netflix Open Connect [5]

2.3 Distributed systems


Innovations in distributed systems are inevitably intertwined both with innovations in applica-
tions, and innovations in the networks they use.
The growth of cloud services (and of more geographically local services) has definitely been
a start in modern innovation in distributed systems, demonstrating demand for alternative
solutions to traditional models, but generally being very limited in what they offer. For example,
virtual machines are only available in a limited number of locations, and access to networking
capacity is generally limited to, at best, some basic load-balancing.
As discussed, CDNs have been trying to push their equipment closer and closer to end users.
A good example of this being driven by necessity is provided by Netflix’s Open Connect CDN,
where Netflix make caching network appliances freely available to ISPs who are willing to place
them closer to end-users on their networks.
It doesn’t take a large leap of imagination to suggest an obvious next step in this progression:
we could simply place clouds in a wider variety of locations, allowing applications to be deployed
closer to end-users. Eventually, these applications could be running at every cell tower and WiFi
AP – activated and deactivated according to local demand. This kind of shared infrastructure
almost seems like a glaringly obvious theme for the future of distributed systems.
At that point, the difference between ‘networking’ and ‘computing’ nodes would become less
and less clear, with network equipment running applications. In fact, the boundaries between
distributed systems and the networks themselves would also become blurred.
If protocol innovations allow us to control the network, why should we allow ourselves to be lim-
ited by client-server or peer-to-peer models? We can simply switch between different topologies
of the network as appropriate for the applications being used, with nodes organising themselves
to suit their current needs, and with the code and data for applications migrating between
different network elements.

8
Enabling Innovation in Networking

Figure 2.10: Components of a city-wide network system

This could be done at a far more fine-grained level than is currently typical, with networks
inside computers themselves (connecting cores and peripherals) also dynamically customising
themselves as necessary for the demands of the applications using them.
Developments along these lines could be argued to form two families of innovations; first, inno-
vations about migrating systems, where applications are run when and where they are needed
(thus inevitably making use of third-party infrastructure), and second, the revolution of network-
aware systems, which are not only aware of the state of the system, but actively both control it
and take advantage of it.
Once we have such new methods available to us, further options would open up: one popular
example is the development of city-wide networks, allowing high-bandwidth and low-latency
applications to be deployed as part of a large-scale distributed system.
For example, video from cameras could be processed locally and used to develop a real-time
view of the situation on roads, which could be provided to self-driving vehicles, integrated into
transport planners, as well as being used for longer-term planning. Adding other kinds of sensors
to the mix would allow even more possibilities.
While security implications would probably make it undesirable to make this a completely open
system, it’s possible that such systems could grow to cover whole countries.
Of course, most applications running on such systems would still desire a high level of abstraction,
perhaps very similar to today’s virtual machines running inside clouds. Such abstraction could
be provided by some kind of custom programming languages and/or frameworks (as we’ll discuss
later), but on a more abstract level still, by considering the qualities of networks.

9
Enabling Innovation in Networking

3. Qualities of networks
When considering the future of networking, an important early step is to find a way to evaluate
networks. Obviously, different viewpoints lead to different wishes, goals and requirements. We
can view a network as having different qualities, and use those as our building blocks for finding
new ways to analyse and improve networks.
On a technical level, some very obvious qualities come to mind: capacity (bandwidth), latency
and jitter together provide the traditional “Quality of Service”. These have been available in
managed form using ATM (Asynchronous Transfer Mode) networks, which provided virtual
circuits with guaranteed QoS parameters.
These are simply not possible to provide on the current internet. Even in completely managed
networks, where it would theoretically be possible, any QoS guarantees are generally contractual,
backed by an SLA (Service Level Agreement) rather than technical means. However, it should
be possible to provide a reasonable approximation on our own networks (whether or not they
are truly part of a third-party cloud).
Ideally, we’d like to be able to plot qualities for different possible configurations of our network
exactly, along the lines of Figure 3.1, and then we can simply pick the optimal point and
reconfigure our network appropriately. Of course, in reality, things are not so simple. Assuming
we have some way to measure these qualities, there are two obvious (and complementary)
approaches to optimising the network.
First, we could optimise a theoretical model of the network. If possible, our qualities would be
expressed in a manner such that we can numerically optimise them, ideally equations (which we
could then solve differentially). Other qualities would need to be optimised in a more general
fashion; techniques such as simulated annealing could be appropriate.
However, this kind of approach will inevitably fail to capture many details of the actual network.
As such, the second approach involves a ‘brute-force’ approach; we can instantiate a network
with the parameters we’ve discovered, deploy a real copy of our distributed system, and simply
measure the qualities.

Optimal?

Avg. Latency

Reliability

Security

Figure 3.1: Idealised view of optimising network qualities

10
Enabling Innovation in Networking

modelling experimenting
adjust instantiate
parameters network

estimate evaluate
qualities qualities

Figure 3.2: Model-based and ‘real’ feedback


loops
Figure 3.3: Loop from Cisco marketing [3]

This could theoretically be done to a very limited extent in a static network environment, but
thanks to software-defined networking and virtual machines, this has become more feasible.
If our network can be dynamically reconfigured on-the-fly, we can perform these experiments
with a single network – and, in fact, we can even continue this process as part of deployed, live
systems, continuing to optimise the network as appropriate for the current situation. Figure 3.2
provides a high-level view of the stages of this process.
Unfortunately, there is a fundamental problem in our scheme: how do we measure the qualities
of certain configurations of our networks? For the simple QoS qualities we’ve discussed, this is
(relatively) simple, but there are many other qualities which could be considered when modelling
and/or evaluating networks.
As an example, along the lines of reliability, scalability, security and maintainability. Can we
define what these qualities might mean in a more precise manner? (In any case, feedback loops
are a conceptually convenient way to illustrate iterative improvement of qualities; for example,
see Figure 3.3.)
Reliability is perhaps most easily defined as the resistance of the network to the failure of nodes;
if we remove random network elements, does the system fail? To take a real-world example,
Netflix’s “Chaos Monkey” software [2] attempts to verify this property by intentionally creating
failures, but ideally it would be possible to measure this in a more abstract way.
Similarly, scalability can be tested by simply increasing the load on the network (and allowing
dynamic reconfiguration/activation to take place) and seeing what happens, or by developing a
model which allows approximation of this.
Defining security is more difficult. A naive approach could be to score features such as firewalls
and intrusion detection systems. Of course, dynamically reconfigurable networks introduce
security risks themselves; often the optimal environment for security might be a static network.
Our last example – maintainability – is even more difficult to measure. While many metrics are
available for evaluating the theoretical maintainability of software systems, none of them have
been shown to be particularly accurate.
Clearly, putting the idea of using qualities to optimise a network into practice is non-trivial.
However, it is definitely a promising concept, which will hopefully improve over time as more is
learnt about appropriate models and metrics.

11
Enabling Innovation in Networking

data

system
configuration

interface firewall

database
switch

computing
elements

Figure 4.1: Visual distributed system description

4. Network programming languages


As valuable as many of the innovations we’ve discussed may sound, the practical limitations of
complexity still seriously limit our ability to use them in our distributed systems. As discussed,
the lack of structure in both applications and networks makes it difficult to do anything about
this.
One idea could be to use so-called internet factories [15] to create a program which can create and
manage an appropriate (virtual) network for the application. This approach involves describing
the necessary infrastructure and interfaces, then having such a factory compile it.
The obvious question is: what kind of programming language would be best for describing such
a program? While we can’t develop a whole new language in this report, we can report on some
existing languages, and consider what kind of elements we might need in an internet factory
language.
The most obvious approach is to simply use a combination of (JavaScript and shell) scripts and
web services, as the original Internet Factories paper describes. The JavaScript can present
a user-friendly interface and call web services to perform high-level application management
(such as managing network elements and resources), while the shell scripts (running on network
elements) can perform the more complicated networking tasks.
However, these require a deep level of knowledge to write (although often this is unnecessary),
and fail to abstract away many of the details. What other alternatives are available?

12
Enabling Innovation in Networking

Some specialised domain-specific languages for networks already exist. For example, Frenetic
[9] and Nettle [16] are languages designed to make it easier to write (switch) controllers for
OpenFlow. Obviously, these are by their nature aimed at a very low level of the networking
architecture, and you can see this in an example snippet of Nettle code (from [16]):

net = routingNetwork bgpNet staticNet redistPolicy


where bgpNet = bgpNetwork home conns prefs usefilter adFilter adMod
prefs = cond (nextHopEq (peerAddr conn A OC3 )) high
(always low )
usefilter = const $ reject $ destInSet martians
high = 120
low = 100

Another active research area is languages which try to hide the network from distributed appli-
cations entirely, allowing them to simply code parallelizable processes and leave the details of the
network distribution to the language and runtime. One such language is Chapel [7]. Some trivial
example Chapel code, which illustrates how it automatically distributes computation between
nodes, is shown below:

use CyclicDist;
config const n = 100;

forall i in {1..100} dmapped Cyclic(startIdx=1) do


writeln("iteration ", i, " runs on node ", here.id);

However, by their nature, these languages are aimed at computational work, rather than focusing
on interactions/interfaces between systems.
However, what we really want is a language which somehow provides a balance between these
two extremes. We’d like to have access to the low-level details of a network if necessary, allowing
us to reconfigure the networking switches and topology on-demand. In most situations, in order
to actually run our applications, we will need to able to automatically activate and deactivate
virtual machines in a cloud.
Importantly, we’d like the details of these tasks abstracted away as much as possible. Taken
to an extreme, visual interfaces (such as that seen in Figure 4.1) can allow quick designing of
networks, offering a high-level alternative approach to textual programming.
On the other hand, we’d like to keep our systems as high-performance and fully-featured as
possible, which often conflicts with abstraction – for example, systems which make real-time
use of local sensor networks may well need the ability to communicate directly without going
through layers such as brokers!
There are also many tiny details; for example, since features such as transparent redundancy
(and fault tolerance) are impossible to do perfectly in a distributed system, we need to make sure
that the failure cases (such as systems becoming inconsistent) can be recognised and handled
by the programmer.

13
Enabling Innovation in Networking

Figure 5.1: The ever-increasing


variety of devices served by modern
Figure 5.2: The services provided by KPN [4]
mobile networks

5. Mobile network operator challenges


Network innovations are especially important to mobile network operators, who invest large
amounts of money into their network infrastructure, and yet often find themselves used as a
“dumb pipe” to the internet, with legacy specialised services (such as telephony and SMS)
becoming rapidly irrelevant.
One approach for improving profitability is to broaden the range of services offered; for example,
KPN offer a variety of different services, as can be seen in Figure 5.2 (including, importantly,
consultancy).
Obviously, virtualisation approaches can be even more advantageous if they can also be used for
providing services such as IPTV, and if fixed-line telephony and home networks can be integrated
into dynamic networks. However, in this paper, we will only discuss possibilities which focus on
activities performed as a mobile network operator.
Operators have been confronted with ever-evolving demands from customers; after GSM, there
was EDGE, an intermediate step attempting to help satisfy the demands of customers who
needed lower-latency, higher-bandwidth data, without the need for separate network infrastruc-
ture and frequencies. However, since then, operators have been required to deploy a whole new
infrastructure for UMTS (and then upgrades, for HSDPA), and most recently, yet more huge
investments have been required for the latest LTE (4G) technology.

2G 3G 4G
Figure 5.3: Evolution of mobile standards

14
Enabling Innovation in Networking

Figure 5.4: Typical architecture of a modern mobile network (from Spirent marketing poster)

Operators who deployed the ‘wrong’ technology (such as WiMAX) have found themselves scram-
bling with even more time and costs to replace it, and even now the spectre of 5G in the future
is something to be wary of.
Customers have also been quickly moving away from operator-controlled services which might
have provided extra revenues to help fund these changes. For example, SMS has been largely
superceded not by MMS (as operators had planned for) but rather by third-party communication
systems such as WhatsApp or Apple’s iMessage.
Furthermore, these mobile protocol innovations themselves often require substantial changes
in the networks of operators, and the greatly increased need for higher-capacity, lower-latency
communications means that not it is not only necessary to upgrade cell towers themselves, but
also the national networks with which they connect to core networks and the internet.
And finally, operators also need to manage their costs. More imaginative approaches to their
networks have paid dividends for some. One easy method to reduce load on networks – and thus
the cost of maintaining sufficient capacity – has been to create parallel WiFi networks. Both
telephony and data can be transparently migrated to these networks when available, unnoticed
to the end-user.
One major challenge, which is often worsened by this kind of innovation, is that the core networks
of these operators have grown organically, making them difficult to understand and as such,
difficult to change.
Figure 5.4 illustrates – at a very basic level – the necessary parts of the core network of a
mobile operator, and it is not difficult to imagine that even moderately-sized networks become
extremely complex even without any problems caused by legacy growth.

15
Enabling Innovation in Networking

Figure 5.5: The Netherlands (left) compared to China (right)

Modern innovation depends on the ability to react quickly to customer needs and offering up-to-
date infrastructure and services. Building large networks and relying on the resulting economies
of scale to make the design and manufacture of custom equipment possible is a viable option for
some large operators.
However, operators in small countries (such as the Netherlands) cannot compete in this way;
see Figure 5.5 for an idea of the differences in scale involved – China has a population (and thus
a potential market) of more than 1.3 billion people, while the Netherlands has far less than 20
million.
Network Function Virtualization is an alternative modern approach to helping cope with the
complexity; this encompasses the various methods of implementing network functions in soft-
ware, including both Software-Defined Networking and virtualisation.
Innovative network functionality can then be deployed using commodity hardware and comput-
ing ‘clouds’, which are affordable even for the smallest operators. Furthermore, thanks to the
software-based approach, major changes can be deployed in a matter of days or weeks, allowing
new services to be launched as soon as possible.
It also makes it easier for operators to share more of their networks with other companies,
reducing costs. This “mobile network sharing” is becoming more and more popular, as many
telecommunications companies move to differentiating themselves primarily with their services
rather than their infrastructure.

Figure 5.6: Expanding the scope of software-defined networking [8]

16
Enabling Innovation in Networking

core computing cloud

legacy
backbone

cell towers cloud


with local backbone
"cloudlets"

Figure 5.7: Cloud-oriented mobile networks

To be more concrete, we can consider virtualization of the two major portions of a typical mobile
provider network: the cell towers (base stations), and the “core networks” (see Figure 5.7).
For example, [10] discusses how “Cloud RAN” (Radio Access Networks) is the next step in the
ongoing process of separating radio functionality from the remaining logical functionality. Fig-
ure 5.8 illustrates a typical modern architecture for this, with so-called “Remote Radio Heads”
providing the physical layer, while virtualised base stations provide the rest of the base station
functions.

Figure 5.8: Virtual base stations and “Remote Radio Heads” [11]

17
Enabling Innovation in Networking

Similarly, “Cloud EPC” (the so-called “Evolved Packet Core”, the term for an LTE core network)
for the virtualiation of telco core networks is also discussed in [10]. Core networks can be
virtualised in a gradual fashion, as legacy functionality is replaced, with each component being
moved into clouds only when it is cost-effective and efficient to do so.
Many further discussions of the (technical) details involved for such virtualisation (as well as
further possibilities, such as virtualising the home equipment provided to customers) can be
found in the literature.
Most importantly, this is definitely not just an academic pipe dream, but rather a collection of
practical solutions to real challenges.
Research into NFV has been industry-led; the NFV working group of ETSI (the European
Telecommunications Standards Institute) has existed since November 2012 and currently in-
cludes many large mobile operators and equipment providers such as China Telecom, KPN,
Vodafone, Sprint, Samsung, Cisco and ZTE [1].
Many companies (such as Intel, ZTE, Cisco, Nokia Siemens, Juniper and Alactel-Lucent, to
name just a few) are already offering hardware or full mobile operator solutions based on SDR
and NFV, advertising the flexibility and cost-effectiveness of migrating networks to cloud-based
operations.
While mobile network operators all have many significant challenges to deal with, the possibilities
offered by the programmable nature of modern networks give them the flexibility they need to
develop new, innovative services and distinguish themselves from their competitors.
Other advantages include energy efficiency, both on the side of the cloud (servers can be used at
optimal efficiency, and turned off if not required) and the physical layer (not only can overlapping
base stations can be turned off when the extra capacity is not required, but other parts of the
network infrastructure such as WiFi access points can assist).
Meanwhile, as we can see in Figure 5.9, the demand from customers on the mobile networks
will continue to grow, requiring more and more investment into network capacity, meaning that
standing still is not an option.

Figure 5.9: Total monthly mobile voice/data [6]

18
Enabling Innovation in Networking

6. Conclusion
The theme is clear: network innovations serve to enable innovations in network systems. The
modern world has networking nodes everywhere - in people’s pockets, on their wrists, in their
cars, even in lightbulbs. The challenge has become how to make effective use of these nodes, and
to do this, we first need to turn our attention to taking advantage of the improvements enabled
by advances in the networks themselves.
To do this, we need to find new ways of evaluating such networks, such as the qualities we
discussed, and new ways of making use of the networks. We’ve considered these things at both a
conceptual level – such as with the families of innovations we discussed – and in a more concrete
fashion, by considering the potential of specialised programming languages.
But most importantly, the message is that the network itself has to become a first-class part of our
distributed systems, being dynamically reconfigurable, with applications migrating where they
are convenient, and with the boundary between networking elements and computing elements
becoming blurred.
This future definitely isn’t yet possible; the highly interoperable internet has enabled many
new applications, but inevitably presents the network solely as an abstraction. While most
infrastructure operators and end-users might not yet be able to dynamically reconfigure the
topologies of our networks nor run software on mobile network cells, the more forward-looking
telecommunications providers are starting to do so, and it is a growing field.
And in the meantime, for the rest of us, large-scale cloud infrastructures have meant that
dynamic activation and reconfiguration of geographically-distributed network nodes has not only
become a possibility, but a vital part of modern many distributed applications and systems.
And while we wait for the real revolutionary transformations in networking, not only cloud
computing but also technologies such as OpenFlow allow us to further experiment with these
concepts, and offer a stepping-stone which can be built upon to make truly innovative networked
systems a reality.

19
Enabling Innovation in Networking

A. Implementing an OpenFlow controller


The practical lab assignment for the course involved implementing an OpenFlow controller,
using Python, which would calculate shortest path routes for arbitrary networks using Dijkstra’s
algorithm.
I implemented such a controller using POX, and the code can be found later in the report. It
makes use of POX’s discovery module. The code waits for the discovered topology to stabilise,
and then one second after the last-seen change, it applies Dijkstra’s algorithm to the entire
discovered topology. If the topology changes (for example, due to links failing), then after the
one-second stabilisation period, it will recompute the routes for the network. The controller
assumes equal-cost paths (alternatives are discussed later).
A key component is the handling of broadcast packets, which is (for example) vital for IP
networking due to ARP packets (in the absence of static ARP entries on every involved host).
The simplest approach of simply re-broadcasting them (on all ports except the one which received
the packet) fails in topologies which contain cycles. Conveniently, our Dijkstra algorithm results
can also be used to provide a minimum spanning tree, and so we simply use the spanning tree
provided by the most recent computation of Dijkstra for broadcasting.
The provided code can route all packets manually, allowing the controller to verify all routing
for correctness on a per-packet basis, which is convenient when debugging during development
(this is controlled by the install rules variable). The processing done for incoming packets
in this situation is relatively simple:
• Incoming packets from edge ports (connected to hosts) have their MAC addresses recorded.
• Broadcast packets are sent out on the spanning tree (if present).
• Other packets are sent on the calculated shortest path (if present).
• Finally, packets are discarded if the spanning tree and/or shortest path information is not
yet available.
However, this is extremely inefficient (both in terms of bandwidth and latency). A better
solution is to instead add flow table entries to the switches (which would be the correct action in
a real-world environment), which is the default behaviour for this code. To illustrate, if we use
the built-in ‘linear’ mininet topology with 4 nodes, the ‘iperf’ test reports 452 Kbits/sec of TCP
performance between the furthest-apart nodes when using manual routing, and 4.8 Gbits/sec
when using the rules.
So, to do this, the code clears all the rules at the start of recalculateDijkstra (i.e. simply
sends a OFPFC DELETE to every switch), and then adds new rules one-by-one in doDijkstra (for
each host individually), where we calculate the ports anyway.
This can also be extended to the broadcasts; we configure the ports which are not part of
the spanning tree with the OFPPC NO FLOOD flag, and then we add a flow table entry using the
OFPP FLOOD action. Obviously we need to make sure that the hosts and any unknown ports are
included in our spanning tree.
I tested the controller using several different styles of topologies; random topologies, disconnected
topologies, ring, line and hub topologies (see Figure A.1). I also verified that the controller
performs correctly for edge cases (in particular that it can cope with cycles in the network, and
multiple distinct sub-networks). As an example, the portion of the Python code I wrote (based

20
Enabling Innovation in Networking

on the skeleton provided) for creating ring, star (hub) and ‘two rings’ (disconnected) topologies
is provided at the end of this appendix.

Figure A.1: Star, ring and line topologies.

The testing was done by pinging between the hosts in the topology in question manually, using
the mininet command line; I also tested (manually) dropping random links. It would be possible
to automate this testing, but for the purposes of this assignment, I felt manual testing sufficed.
The discovery module which we’re using makes use of OFDP, an OpenFlow-specific version of
the LLDP (the Link Layer Discovery Protocol). We avoid much of the complexity of designing
and implementing a routing protocol (specifically, the topology discovery) in this way. Our
controller is fairly similar (although much simpler) in design to the ‘real-world’ routing protocol
IS-IS (Intermediate System to Intermediate System), in which each router independently builds
topology information using Dijkstra, if we consider only intra-area routing.
OSPF (Open Shortest Path First) also makes use of Dijkstra’s algorithm, but operates at a
different level (on top of IP) and is considerably more complex (for example, it splits the network
into different areas, and routers communicate with neighbour routers to establish links between
neighbours). All of these systems are designed for use internally to an autonomous system,
including our controller.
There are, of course, further improvements which could be made. For example, the controller
could determine the weights of the edges in the Dijkstra tree by performing some measurements
(such as latency, using ping, or bandwidth, using some test packets) rather than simply weighting
all edges with the same value. And finally, there are many optimisations which could be made for
efficiency, such as not recomputing paths for the entire network when a single path is dropped.
However, the controller as it is demonstrates the advantages which OpenFlow brings to proto-
typing and design of innovative networking algorithms, allowing us to quickly and easily develop
systems which can be used for experiments with an emulated network but can also be applied
to real-world networking equipment.
The command-line output for some example results is provided below. The controller was started
remotely using pox.py dijkstra controller.
First, the performance measurements discussed above (first with manual routing, with all packets
being processed by the controller, and then with rule-based routing, with routing being done
using flow rules added to the switches):
# mn --topo=linear,4 --controller remote --test iperf
Waiting for iperf to start up...*** Results: [’452 Kbits/sec’, ’921 Kbits/sec’]

# mn --topo=linear,4 --controller remote --test iperf


Waiting for iperf to start up...*** Results: [’4.80 Gbits/sec’, ’4.81 Gbits/sec’]

21
Enabling Innovation in Networking

Then, a demonstration that the routing works for two non-trivial topologies:
# mn --custom topo.py --topo=myringtopo --controller remote

mininet> pingall
*** Ping: testing ping reachability
h0 -> h1 h2 h3 h4 h5 h6 h7 h8
h1 -> h0 h2 h3 h4 h5 h6 h7 h8
h2 -> h0 h1 h3 h4 h5 h6 h7 h8
h3 -> h0 h1 h2 h4 h5 h6 h7 h8
h4 -> h0 h1 h2 h3 h5 h6 h7 h8
h5 -> h0 h1 h2 h3 h4 h6 h7 h8
h6 -> h0 h1 h2 h3 h4 h5 h7 h8
h7 -> h0 h1 h2 h3 h4 h5 h6 h8
h8 -> h0 h1 h2 h3 h4 h5 h6 h7
*** Results: 0% dropped (72/72 received)

# mn --custom topo.py --topo=myhalftopo --controller remote

mininet> pingall
*** Ping: testing ping reachability
h0 -> h1 h2 h3 X X X X X
h1 -> h0 h2 h3 X X X X X
h2 -> h0 h1 h3 X X X X X
h3 -> h0 h1 h2 X X X X X
h4 -> X X X X h5 h6 h7 h8
h5 -> X X X X h4 h6 h7 h8
h6 -> X X X X h4 h5 h7 h8
h7 -> X X X X h4 h5 h6 h8
h8 -> X X X X h4 h5 h6 h7
*** Results: 55% dropped (32/72 received)
And finally, an example demonstrating that the controller can cope with links disappearing.
When a link in the ring topology is removed, packets fail to be routed for a few seconds, but
then the network reconfigures itself with a new route and packet routing works again:
# mn --custom topo.py --topo=myringtopo --controller remote
mininet> h0 ping -c 1 h8 | grep packets
1 packets transmitted, 1 received, 0% packet loss, time 1001ms
mininet> link s0 s8 down
mininet> h0 ping -c 1 h8 | grep packets
1 packets transmitted, 0 received, 100% packet loss, time 1003ms
... wait ...
mininet> h0 ping -c 1 h8 | grep packets
1 packets transmitted, 1 received, 0% packet loss, time 1007ms

22
Enabling Innovation in Networking

Controller
import pox.openflow.libopenflow_01 as of

from pox.lib.addresses import EthAddr


from pox.core import core
import pox.openflow.discovery

from pox.lib.recoco import Timer

import pox.lib.packet as pkt

install_rules = True

class DijkstraController:
def __init__(self):
# maps host MAC addresses to switch connections
# (this tells us the destination switch which is important)
self.macs = {}
# and MAC addresses to the switch ports, too
self.ports = {}

# paths give us the next port for each target dpid, *for each dpid*
self.paths = {}

# spanning tree contains the *ports* of the spanning tree, for each dpid
# i.e. spanning_tree[my_dpid] tells you which ports are involved (not
# including a connected host)
self.spanning_tree = {}

core.openflow.addListeners(self)
core.openflow_discovery.addListeners(self)

self.recalculate_timer = None

def _handle_PacketIn(self, event):


packet = event.parsed

handle_manually = not install_rules

# (the hosts in our own topo are on port 1, but


# we check manually since it’s more generic)
if core.openflow_discovery.is_edge_port(event.dpid, event.port):
# this is from an edge (->host) port, record the MAC
mac = packet.src
if not mac in self.macs:
print "discovered new host: %s (%s)" % (mac, str(event.connection))
self.macs[mac] = event.connection
self.ports[mac] = event.port
if install_rules:
# add a rule to (locally) route to this host

23
Enabling Innovation in Networking

self.addHostRule(mac)
for connection in core.openflow.connections:
self.addFloodRule(connection, mac)
handle_manually = True
# force recalculation of routing
if self.recalculate_timer:
self.recalculate_timer.cancel()
self.recalculate_timer = Timer(1, self.recalculateDijkstra)

dest = packet.dst
if dest == EthAddr("ffffffffffff"):
print "%s: flooding from %s" % (str(event.connection), packet.src)

if not handle_manually:
# we should already have a rule..
return

output_ports = set()
if event.dpid in self.spanning_tree:
output_ports = self.spanning_tree[event.dpid].copy()
for i in event.connection.ports:
if core.openflow_discovery.is_edge_port(event.dpid, i):
output_ports.add(i)
output_ports.remove(event.port)
if len(output_ports) == 0:
print ".. but spanning tree doesn’t include us, and no other ports!"
return

msg = of.ofp_packet_out()
msg.buffer_id = event.ofp.buffer_id
msg.in_port = event.port
for p in output_ports:
action = of.ofp_action_output(port = p)
msg.actions.append(action)
event.connection.send(msg)
return

if not handle_manually:
# we should already have a rule, why are we seeing this?
# (non-existant destinations, or rules not ready yet..)
print "! no rule for: " + str(packet)

if not (dest in self.macs):


print "%s: dropping packet to unknown host %s" % (str(event.connection), dest)
return

print "%s: routing from %s to %s (target %s)" % \


(str(event.connection), packet.src, dest, self.macs[dest])

dest_port = None
if self.macs[dest].dpid == event.dpid:

24
Enabling Innovation in Networking

dest_port = self.ports[dest] # host port


else:
dest = self.macs[dest].dpid
if not dest in self.paths[event.dpid]:
print "dropping packet due to no path"
return
dest_port = self.paths[event.dpid][dest]

print "sending on port %d" % dest_port


msg = of.ofp_packet_out()
msg.buffer_id = event.ofp.buffer_id
action = of.ofp_action_output(port = dest_port)
msg.actions.append(action)
event.connection.send(msg)

def addFloodRule(self, connection, mac):


entry = of.ofp_flow_mod()
entry.priority = 100
entry.match.dl_src = mac
entry.match.dl_dst = EthAddr("ffffffffffff")
action = of.ofp_action_output(port = of.OFPP_FLOOD)
entry.actions.append(action)
connection.send(entry)

def addHostRule(self, mac):


connection = self.macs[mac]
port = self.ports[mac]
entry = of.ofp_flow_mod()
entry.priority = 100
entry.match.dl_dst = mac
action = of.ofp_action_output(port = self.ports[mac])
entry.actions.append(action)
connection.send(entry)

def _handle_ConnectionUp(self, event):


print "new connection: %s" % event.connection
pass

def recalculateDijkstra(self):
self.recalculate_timer = None
print "recalculating the shortest paths"
if install_rules:
# remove all rules, install only the flooding one
msg = of.ofp_flow_mod(command = of.OFPFC_DELETE)
for connection in core.openflow.connections:
connection.send(msg)

# we delete the discovery rules above and we can’t use


# cookie matching on this old OpenFlow version, so let’s
# sneakily recreate it as the easiest fix...
msg2 = of.ofp_flow_mod()

25
Enabling Innovation in Networking

msg2.match = of.ofp_match(dl_type = pkt.ethernet.LLDP_TYPE,


dl_dst = pkt.ETHERNET.NDP_MULTICAST)
msg2.actions.append(of.ofp_action_output(port = of.OFPP_CONTROLLER))
connection.send(msg2)

# enable flooding on all ports (we’ll disable again below)


for p in connection.ports.keys():
enableflood = of.ofp_port_mod(
port_no = p,
hw_addr = connection.ports[p].hw_addr,
config = 0,
mask = of.OFPPC_NO_FLOOD)
connection.send(enableflood)
for mac in self.macs.keys():
for connection in core.openflow.connections:
self.addFloodRule(connection, mac)
self.addHostRule(mac)

# first: create list of all dpids


edges = {}
dpid_set = set()
for p in core.openflow_discovery.adjacency:
dpid_set.add(p.dpid1)
dpid_set.add(p.dpid2)
if p.dpid1 not in edges:
edges[p.dpid1] = []
edges[p.dpid1].append(p)
if install_rules:
# we route all the way to targets
for m in self.macs.keys():
class NotALink: pass
mylink = NotALink()
# link from switch to host
mylink.dpid1 = self.macs[m].dpid
mylink.dpid2 = m
mylink.port1 = self.ports[m]
mylink.port2 = None
dpid = self.macs[m].dpid
if dpid in edges:
edges[dpid].append(mylink)
else:
print "WARNING: switch for host %s not yet available?" % m
print dpid_set
# then, calculate dijkstra from each of them
for d in dpid_set:
self.paths[d] = self.doDijkstra(d, dpid_set, edges)
for d in dpid_set:
connection = core.openflow.connections[d]
for p in connection.ports.keys():
# disable flooding on ports which aren’t in the spanning tree
# (but *do* flood to as-yet-unknown ports)

26
Enabling Innovation in Networking

do_flood = p in self.spanning_tree[d]
do_flood = do_flood or core.openflow_discovery.is_edge_port(d, p)
goes_to_switch = False
for pp in edges[d]:
if pp.port1 == p:
goes_to_switch = True
if do_flood or (not goes_to_switch):
continue
disableflood = of.ofp_port_mod(
port_no = p,
hw_addr = connection.ports[p].hw_addr,
config = of.OFPPC_NO_FLOOD,
mask = of.OFPPC_NO_FLOOD)
connection.send(disableflood)

# calculate paths *from* switch with this dpid


def doDijkstra(self, dpid, nodes, edges):
visited = {dpid: 0}
path = {}
remaining_nodes = nodes.copy()

while remaining_nodes:
closest_node = None
for n in remaining_nodes:
if not (n in visited):
continue
if (not closest_node) or (visited[n] < visited[closest_node]):
closest_node = n
if not closest_node:
# no more connected nodes?
break

# closest_node now gets added to our tree


remaining_nodes.remove(closest_node)
weight = visited[closest_node]

if not closest_node in edges:


# this can happen temporarily when the topology we "see" is asymmetric
# (it’ll resolve itself as soon as we notice the other half of the link)
print "oh-oh, closest_node %d wasn’t in edges" % closest_node
continue

for e in edges[closest_node]:
d = e.dpid2
new_weight = weight + 1
if not (d in visited) or (new_weight < visited[d]):
visited[d] = new_weight
path[d] = e

# we cheat and make a spanning tree here, because sigh


for p in path.values():

27
Enabling Innovation in Networking

# first: clear out the old values (for connected nodes only!)
self.spanning_tree[p.dpid1] = set()
self.spanning_tree[p.dpid2] = set()
for p in path.values():
# then: add the ports
self.spanning_tree[p.dpid1].add(p.port1)
# (dpid2 might be a host, they don’t participate)
if p.port2 != None:
self.spanning_tree[p.dpid2].add(p.port2)

# we only want the FIRST hop


for p in path.keys():
while path[p].dpid1 != dpid:
path[p] = path[path[p].dpid1]
# in fact, all we want is the port to send on :)
for p in path.keys():
if install_rules and (p.__class__ == EthAddr):
entry = of.ofp_flow_mod()
entry.priority = 100
entry.match.dl_dst = p
action = of.ofp_action_output(port = path[p].port1)
entry.actions.append(action)
core.openflow.connections[dpid].send(entry)
path[p] = path[p].port1
return path

def _handle_LinkEvent(self, event):


print "link event: " + str(event)
if event.removed:
# check for a link between a switch and a connected involved host going down
# we kind of assume this won’t happen though
for m in self.macs.keys():
dpid = self.macs[m].dpid
if dpid == event.link.dpid1 or dpid == event.link.dpid2:
print "WARNING: removing link for %s" % str(m)

# the dijkstra recalculation will remove the old rules


if self.recalculate_timer:
self.recalculate_timer.cancel()
self.recalculate_timer = Timer(1, self.recalculateDijkstra)

def launch():
pox.openflow.discovery.launch()
#pox.host_tracker.launch()
core.registerNew(DijkstraController)

28
Enabling Innovation in Networking

Topology Tests
from mininet.topo import Topo

class MyTopo(Topo):
def __init__(self, n=8, **opts):
Topo.__init__(self, **opts)
self.myswitches = []
self.myhosts = []

# add n switches, and a host attached to each one


for i in range(n):
switchName = ’s’ + str(i)
self.myswitches.append(self.addSwitch(switchName))
for i in range(n):
hostName = ’h’ + str(i)
self.myhosts.append(self.addHost(hostName))
for i in range(n):
self.addLink(self.myswitches[i], self.myhosts[i])

class MyRingTopo(MyTopo):
def __init__(self, n=9, **opts):
MyTopo.__init__(self, n, **opts)
for i in range(n):
self.addLink(self.myswitches[i], self.myswitches[(i+1)%n])

class MyStarTopo(MyTopo):
def __init__(self, n=9, **opts):
MyTopo.__init__(self, n, **opts)
for i in range(n-1):
self.addLink(self.myswitches[i+1], self.myswitches[0])

# two rings
class MyTwoHalfTopo(MyTopo):
def __init__(self, n=9, **opts):
MyTopo.__init__(self, n, **opts)
firsthlf_top = n/2
for i in range(firsthlf_top):
self.addLink(self.myswitches[i], \
self.myswitches[(i+1)%firsthlf_top])
secondhlf_size = n - n/2
for i in range(n-firsthlf_top):
self.addLink(self.myswitches[firsthlf_top+i], \
self.myswitches[firsthlf_top+((i+1)%secondhlf_size)])

topos = {
’myringtopo’: ( lambda: MyRingTopo() ),
’mystartopo’: ( lambda: MyStarTopo() ),
’myhalftopo’ : ( lambda: MyTwoHalfTopo() )
}

29
Enabling Innovation in Networking

B. Bibliography
[1] ETSI NFV WG member list. https://portal.etsi.org/TBSiteMap/NFV/NFVMembership.aspx.
[2] Resilience engineering: Learning to embrace failure. Queue, 10(9):20:20–20:28, September
2012.
[3] Software-Defined Networking: Why We Like It and How We Are Building On It.
https://www.cisco.com/web/strategy/docs/gov/cis13090 sdn sled white paper.pdf, 2013.
[4] KPN Annual Report 2014, February 2015.
[5] Netflix OpenConnect Appliance deployment guide. http://oc.nflxvideo.net/docs/OpenConnect-
Deployment-Guide.pdf, April 2015.
[6] akamai. State of the internet, 2014.
[7] David Callahan, Bradford L Chamberlain, and Hans P Zima. The cascade high productivity
language. In High-Level Parallel Programming Models and Supportive Environments, 2004.
Proceedings. Ninth International Workshop on, pages 52–60. IEEE, 2004.
[8] Ericsson. The real-time cloud. http://www.ericsson.com/res/docs/whitepapers/wp-sdn-
and-cloud.pdf, February 2014.
[9] Nate Foster, Rob Harrison, Michael J Freedman, Christopher Monsanto, Jennifer Rexford,
Alec Story, and David Walker. Frenetic: A network programming language. In ACM
SIGPLAN Notices, volume 46, pages 279–291. ACM, 2011.
[10] Bo Han, Vijay Gopalakrishnan, Lusheng Ji, and Seungjoon Lee. Network function virtual-
ization: Challenges and opportunities for innovations. Communications Magazine, IEEE,
53(2):90–97, 2015.
[11] Intel. Realising the Benefits of Network Functions Virtualisation in Tele-
coms Networks. http://www.qosmos.com/wp-content/uploads/2014/02/ In-
tel Tieto Qosmos TelecomNetVirtualisationFeb2014.pdf, February 2014.
[12] Frederic Jacobs. Providing better confidentiality and authentication on the internet using
namecoin and minimalt. CoRR, abs/1407.6453, 2014.
[13] Alan B Johnston and Daniel C Burnett. WebRTC: APIs and RTCWEB protocols of the
HTML5 real-time web. Digital Codex LLC, 2012.
[14] Ulrich H Reimers. DVB – the family of international standards for digital video broadcast-
ing. Proceedings of the IEEE, 94(1):173–182, 2006.
[15] Rudolf Strijkers, Marc X Makkes, Cees de Laat, and Robert Meijer. Internet factories:
Creating application-specific networks on-demand. Computer Networks, 68:187–198, 2014.
[16] Andreas Voellmy and Paul Hudak. Nettle: Taking the sting out of programming network
routers. In Practical Aspects of Declarative Languages, pages 235–249. Springer, 2011.
[17] Jianfeng Wang. Zigbee light link and its applicationss. Wireless Communications, IEEE,
20(4):6–7, 2013.
[18] Zhenlong Yuan, Cuilan Du, Xiaoxian Chen, Dawei Wang, and Yibo Xue. Skytracer: To-
wards fine-grained identification for skype traffic via sequence signatures. In Computing,
Networking and Communications (ICNC), pages 1–5. IEEE, 2014.

30

Вам также может понравиться