Академический Документы
Профессиональный Документы
Культура Документы
net/publication/304457462
CITATIONS READS
20 8,098
4 authors, including:
Ali Chehab
American University of Beirut
264 PUBLICATIONS 1,292 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Ali Chehab on 05 October 2017.
Abstract— The control plane is an essential part of approaches, languages, architectures, application
the SDN architecture, so it is very important to give program interfaces (APIs) and protocols have been
proper attention to any proposal or design of an SDN implemented. In this paper, we will try to compare the
controller. During the past few years, several most popular controllers and to test their performance
controllers have been developed and several studies using Cbench, which is an OpenFlow testing tool.
have been done to evaluate, compare and test the
performance of these controllers. In this paper, new This paper is structured as follows: section II
discusses the previous work related to SDN controller’s
controllers are tested, such as ONOS and Libfluid-
based controllers (raw, base), using Cbench, an comparison, section III describes the basic controller’s
modules and the characteristics of the most popular
OpenFlow testing tool. Even though the results show
that MUL and Beacon are the best performing controllers, section IV shows and analyzes the results of
our controllers’ performance test and finally we conclude
controllers; however, the selection of the best-
performing controller should be based on several in section V.
criteria, per the user requirements. II. RELATED WORK
Keywords— SDN; controller; Cbench; Several studies have been done in the aim of
performance. comparing SDN controllers. Tootoonchian et al in [2]
were among the first to provide the comparative analysis
I. INTRODUCTION of controllers. They covered a limited number of
Software Defined Network (SDN) is a new controllers (NOX-MT, Beacon and Maestro) and
networking paradigm where the architecture moves from focused on controller performance. These controllers
the traditional fully distributed model to a more have already been superseded by other controllers like
centralized approach. This approach is also characterized POX, Ryu, FloodLight and OpenDaylight.
by the separation of the data and control planes. The data
plane includes the forwarding element (switches and The study conducted in [3] provides a set of
requirements that are at the base of the comparison
routers) and the control plane includes the controller. The
controller provides a high abstraction level of the between the controllers: TLS Support, virtualization,
open source, interfaces, GUI, RESTful API,
forwarding elements management which is absent in
today’s networks. Therefore, the controller is a productivity, documentation, modularity, platform
support, age, OpenFlow support, and OpenStack
fundamental component of the SDN architecture that
Neutron support. The comparison was done using a
will contribute to the success or failure of SDN.
Therefore, there is a need to assess and compare the Multi-Criteria Decision Making (MCDM) method
named Analytic Hierarchy Process (AHP) adapted by a
different existing controllers in the market and research
domains. monotonic interpolation/extrapolation mechanism which
maps the values of the properties to a value in a pre-
We are far from a controller (in some cases referred defined scale. By using the adapted AHP, the top five
to as network operating system) which is hardware and controllers have been compared, and “Ryu” is selected to
language independent. Yanc, proposed in [1], can be be the best controller based on their requirements.
considered as an example of such a system. However,
An advanced study of SDN/OpenFlow Controllers was
today’s controllers run as monolithic applications and
they are highly tied to their programming languages performed in [4]. The authors claim to do an impartial
comparative analysis of the effectiveness of the widely
(java, python, C, C++, etc.) Therefore, SDN specific
languages such as Pyretic and Frenetic offering high- used SDN controllers: NOX, POX, Beacon, Floodlight,
MuL, Maestro and Ryu. They used a tool named
level abstraction languages were proposed to allow for
application portability; however, they are in reality hcprobe. They concluded that there were a number of
possible security vulnerabilities of the tested controller
linked to a specific controller platform (POX).
that most of the controllers were capable of coping with
Due to the importance of the controller within the an average workload, and that the maximum throughput
SDN architecture and the diversity of architectures and was demonstrated by Beacon.
implementations in the market and research fields, there
In [5] two operating modes are compared, proactive and
is a need to assess and benchmark all these choices
against different performance indicators. reactive. The proactive mode has better performance
than the reactive one due to the fact that the rules are
From controllers processing few thousands of flows/s
to ones processing few millions of flows/s, multiple
loaded to the switch at the start not as in the reactive mode controller and calculates the routes in the network. This
where the rules are loaded to the switch each time the module uses the neighbor database to compute the
switch receives a packet with no matching rule in its flow network topologies based on the received information
table. from the Link Discovery module. The Topology Manager
builds the global Topology Database at the controller,
Another study is done in [5] giving further considerations
which contains the shortest (and alternate) path
when designing a new controller. Two types of information to any OpenFlow node or host.
architectures are considered: static partitioning with static
batching, and shared queue with adaptive batching. IV. SDN CONTROLLER FEATURES
Beacon which uses static batching presented the highest
performance in the experiment conducted with Maestro, 1) Programming Language
NOX-MT and Floodlight. However, Maestro that uses the Running cross-platform, allowing multithreading,
adaptive batching architecture presents the best latency being easy to learn, allowing fast memory access and good
records. memory management are essential programming
The fact that the controllers are software-based, the languages’ characteristics. When choosing a certain
programming languages used to develop them are key. In controller, we have to take these factors into consideration
[6], the choice of the programming language is because they affect the controller’s performance and
demonstrated to have an impact on the portability and the development speed. Python, C++, and Java are the most
performance of the controller. The authors claim that Java used languages for SDN controllers programming. In
is the best choice given that it supports multithreading and general, the Java coded controllers have the characteristic
it is a cross-platform language. Whereas Python presents to run cross-platform and present good modularity, the C
issues with multithreading on the performance level and coded controllers provide high performance but lack high
C, C++ have problems with memory management and modularity, good memory management and good GUI
the .Net languages are dependent of the runtime platform and the Python coded ones lack real multi-threading
(compatibility with Linux is not supported). Thus, they handling.
show that Beacon, which is java- based, has the best 2) OpenFlow Support
performance among several controllers (NOX, POX,
Maestro, Floodlight, Ryu). Therefore, in [7] the issue of The OpenFlow protocol is a key enabler for software-
software aging is highlighted. Memory leak is the main defined networks. It was the first standardized southbound
issue studied and the comparison was done between two interface. It allows direct manipulation of the forwarding
java based controllers (Floodlight and Beacon) to ensure plane of OpenFlow switches [9]. When choosing an
the fairness of the study. The results showed that Beacon OpenFlow controller, we need to understand the
outperforms Floodlight with less memory consumption. OpenFlow functionality that the controller supports as
well as the development roadmap to implement newer
In this work, we will compare the most common open versions of OpenFlow, such as v1.3 or v1.4. One reason
source controllers considering multiple criteria. for needing to take this into consideration is that important
Therefore, the choice of the controller will be a subjective functionality such as IPv6 support, for example, is not part
issue based on the needed requirements. of OpenFlow v1.0 but is part of the OpenFlow v1.3
III. SDN CONTROLLER BASIC FUNCTIONS standard.
Multithreadin
Distributed/
Centralized
Modularity
Productivity
Northbound
Southbound
Openstack
Partener
g Support
Language
Program-
Platform
Entation
Support
Support
Docum-
GUI
Ming
Apis
Apis
ONOS Java Web Good High D Linux,MAC Fair OF1.0, REST ON.LAB, At&T, Y N
Based OS, And 1.3, API Ciena,Cisco,
Windows NETCO Ericsson,Fujitsu,
NF Huawei,Intel,
Nec,Nsf.Ntt
Comunnication,
Sk Telecom
Open- Java Web Very High D Linux,MAC Fair OF1.0, REST Linux Y Y
Based Good OS, And 1.3, 1.4, API Foundation With
Day- Windows NET- Memberships
Light CONF/ Covering Over
YANG, 40 Companies,
OVSDB, Such As Cisco,
PCEP, IBM, NEC
BGP/LS,
LISP,
SNMP
NOX C++ Python Poor Low C Most Fair OF 1.0 REST Nicira NOX_ N
+ QT4 Supported On API MT
Linux
POX Python Python Poor Low C Linux,MAC High OF 1.0 REST Nicira N N
+ QT4 OS, And API
Windows
RYU Python Yes Fair Fair C Most High OF 1.0, REST Nippo Telegraph Y Y
Supported On 1.2, 1.3, For And Telephone
Linux 1.4, South Corporation
NETCO bound
NF, OF-
CONFIG
Beacon Java Web Fair Fair C Linux,MAC Fair OF 1.0 REST Standford Y N
Based OS, And API University
Windows
Maestro Java - Poor Fair C Linux,MAC Fair OF 1.0 REST RICE, NSF Y N
OS, And API
Windows
Flood- Java Web/ Good Fair C Linux,MAC Fair OF 1.0 , REST Big Switch Y N
Java OS, And 1.3 API Networks
Light Based Windows
Iris Java Web Fair Fair C Linux,MAC Fair OF 1.0, REST ETRI Y N
Based OS, And 1.3, API
Windows OVSDB
MUL C Web Fair Fair C Most Fair OF 1.4, REST Kulcloud Y Y
Based Supported On 1.3, 1.0, API
Linux OVSDB,
OF-
CONFIG
Runos C++ Web Fair Fair D Most Fair OF 1.3 REST ARCCN Y N
Based Supported On API
Linux
Lib- C++ - Fair Fair - Most Fair OF 1.0, - ONF Y N
Supported On 1.3
Fluid Linux
and waits for the reply to compute the time taken to
process a single packet by the controller. Cbench and the
Throughput Mode
tested controller can be run on the same machine or on 6
different machines. Thus, due to the limitation of having
a 10 Gbps link, we run our tests on a single machine. The
machine used to perform the experiments has an Intel® 5
Core™ i7-3630QM CPU @ 2.40GHZ x 8 (8 cores). 16
Million responses/s
GB of memory was available. Operating system was
4
Ubuntu 14.04 LTS-64 bit. We used the command below
for running tests:
./cbench –c localhost –p 6633 –l 14 -m 10000 –M 3
1000 –s 8 –t
Where: 2
-c controller (IP or hostname)
-p controller port number 1
-l loops per test
-m test time per s
-M Nbr of mac addresses per switch 0
-s Nbr of switches 0 8 16 64 100
-t throughput mode Number of switches
ONOS POX
A. Analysis of Results NOX MUL
BEACON FLOODLIGHT
The tests were conducted for the listed controllers MAESTRO IRIS
except Runos, which does not function with the current RYU LIBFLUID_MSG
Cbench version even though it supports OpenFlow 1.3. LIBFLUID_RAW OPENDAYLIGHT
It is important to note that OpenDaylight, as stated in Figure 1: Cbench test in throughput mode with different number
[22], has problems with Cbench. So we have done the of switches-
OpenDaylight’s tests using Wcbench, which is a Phyton
wrapper around Cbench. Latency Mode
Performing the tests in the two modes, varying the 1000
number of switches, and in the throughput mode varying
the number of threads binding to the controller instance,
we have obtained the results shown in Figures 1, 2, and
3. In Fig. 1, the results show that the controllers coded by
time in µs