Testing Challenges for Modern Networks Built Using

SDN and OpenFlow

July 2013

Rev. A 07/13

SPIRENT
1325 Borregas Avenue
Sunnyvale, CA 94089 USA
Email: sales@spirent.com
Web: www.spirent.com

AMERICAS 1-800-SPIRENT +1-818-676-2683 sales@spirent.com

EUROPE AND THE MIDDLE EAST +44 (0) 1293 767979 emeainfo@spirent.com
ASIA AND THE PACIFIC +86-10-8518-2539 salesasia@spirent.com

2013 Spirent. All Rights Reserved.

All of the company names and/or brand names and/or product names referred to in this document, in particular,
the name Spirent and its logo device, are either registered trademarks or trademarks of Spirent plc and its
subsidiaries, pending registration in accordance with relevant national laws. All other registered trademarks or
trademarks are the property of their respective owners.
The information contained in this document is subject to change without notice and does not represent a
commitment on the part of Spirent. The information in this document is believed to be accurate and reliable;
however, Spirent assumes no responsibility or liability for any errors or inaccuracies that may appear in the
document.

Testing Challenges for Modern Networks

Built Using SDN and OpenFlow

CONTENTS
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Understanding SDN and OpenFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
A new Perspective on the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
The Benefits of SDN and OpenFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Common Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Enterprise Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Service Provider Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Summing up the Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Moving SDN Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Emerging Test Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
SDN/OpenFlow Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

SPIRENT WHITE PAPER i

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

INTRODUCTION
Networks must continue to evolve for a variety of reasons. For example, network traffic
continues to see very high rates of growth. According to Internet World statistics, 2.4
billion individualsor over one third of the Earths entire populationused the Internet
in 2012. The Cisco Visual Networking Index indicates that global IP traffic will surpass
the zettabyte threshold by the end of 2016, with a forecast of 1.3 zettabytes for the year
or 110.3 exabytes per month.
There are also ongoing shifts in the type of traffic crossing networks. The Cisco Visual
Networking Index reveals that the number of mobile devices connected to IP networks
in 2016 will be nearly three times as high as the global population at that time. The
index also shows that video traffic will represent 55% of all consumer Internet traffic
that year and that video on demand will grow to the equivalent of 4 billion DVDs
per month.
Emerging network deployment scenarios, such as virtualized and cloud environments
inside hyper-scale data centers, are also straining existing network technologies and
architectures. Dan Pitt, Executive Director of the Open Networking Foundation has
described three key challenges faced by traditional networkingcost, agility
and design.
A closer look at each of these challenges sheds light on the scope of change required to
keep up with evolving network demands:
CostAs with other types of infrastructure, significant up-front capital spending
(CapEx) is required in order to build a large network. Additionally, ongoing
operating expenditures (OpEx) in the form of management tools and staffing
also add to the cost.
AgilityTodays globally competitive world not only demands cost-effective
solutions, it requires fast time-to-market so that new services can be made
available quickly. Once these new services are available, they must also support
rapid, on-demand provisioning and re-provisioning.
DesignTraditional networks were not designed for virtualized and cloud
environmentsor for other on-demand service models. Unfortunately this
means they can become a bottleneck for server virtualization. They may
also fall short when it comes to east-west traffic performance.
It turns out that a new approach to networking, called software-defined networking
(SDN), and an associated protocol, OpenFlow, address each of these challenges and
more. Yet, while SDN and OpenFlow hold tremendous promise, they are also disruptive
to several aspects of the network ecosystem, including network testing.

SPIRENT WHITE PAPER

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

UNDERSTANDING SDN AND OPENFLOW

Understanding SDN and OpenFlow is helped by first considering how traditional
network devices operate. Most of todays switches and routers are responsible for both
the control and data planes. The control plane determines which packets are forwarded
where, and the data plane actually forwards them. However, as server virtualization
and cloud computing gain broader adoption, limitations in this approach have become
more severe.
Since each network element in these traditional IP networks is an autonomous system
with a view of the network mostly limited to the next hop, there is no end-to-end view
of traffic flows. This introduces a few challenges. For example, in heavily virtualized
data centers and cloud computing environments, it is difficult to ensure that traffic is
properly routed as virtual machines migrate from host to host.
As shown in the figure below, SDN changes the traditional approach by separating
the control and data planes and centralizing control for all network devices in a single
controller that often runs on a general-purpose server. SDN also allows applications, to
be written above the controller so that complex, end-to-end network configurations can
be made through API callsthus the software defined in SDN.

Applications

API

Control
Plane

API

Network Operating System

Data Plane
Hardware Abstraction Layer

Switching Silicon/HW

SPIRENT WHITE PAPER

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

OpenFlow, shown between the control plane and the data plane in the diagram, is a
Layer 2 communications protocol that enables SDN. OpenFlow-enabled controllers and
switches communicate with each other through the OpenFlow protocol that supports a
small number of primitives such as modify forwarding table and get stats.
The OpenFlow protocol ensures that network configuration changes made in the
controller are quickly distributed to all appropriate switches and routers. Note that other
protocols beyond OpenFlow can be used to distribute forwarding rules.

A new Perspective on the Network

Traditional switches and routers are typically monolithic, tightly integrated, proprietary
closed appliances built on custom silicon. While devices from multiple vendors can
interoperate within an IP network, they generally have custom control planes that
lead to vastly different administrative interfaces, management tools and value-added
features. To ensure consistency throughout a network, it is common for equipment from
a single vendor to be used.
SDN-based networks instead take an open and modular approach. Intelligent
switches and routers become a set of simplified, distributed traffic forwarders, taking
their guidance from a centralized controller such as an OpenFlow controller. Rather than
relying on custom silicon with proprietary control plane logic, the distributed traffic
forwarders can be built on more generic devices such as x86 servers. Applications
are then written to guide the SDN controller in making end-to-end switching and
routing decisions.
SDN applications open up a far greater range of possibilities when it comes to
controlling the network. For example, SDN applications may deliver firewall, load
balancer, intrusion detection and other network capabilities. Various types of network
virtualization applications will also be written. In fact, network virtualization is so
closely associated with SDN that many in the industry incorrectly equate the two. The
reality is that network virtualization in the SDN world is really just another
SDN application.

SPIRENT WHITE PAPER

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

THE BENEFITS OF SDN AND OPENFLOW

Both enterprise organizations and service providers alike have plenty to gain from
the adoption of SDN and OpenFlow. Some of these benefits are unique to enterprise
networks while others are unique to service provider networks. There are also a number
of benefits that both types of networks can enjoy.

Common Benefits
The greatest driver of disruptionas well as benefitsfrom SDN and OpenFlow is
likely to come from opening up the developer ecosystem, enabling a larger number of
developers to contribute to the advancement of networking. Prior to SDN, nearly all
developers of switch and router technology were employees of network equipment
vendors. While these vendors have many of the best and brightest in the industry,
separating out the network control plane means that just about any innovative
developer can now create an SDN application to control the network.
Would you like to create service paths and tunnels? There will be an app for that. Would
you like an easy way to configure quality of service (QoS) for a particular application
or set of users? There will be an app for that too. This is not to suggest that SDN
applications will install for $.99 from an app store. It is really meant to illustrate that
switch and router functionality will be developed and made available separately from
underlying hardware. Rather than waiting for your network equipment vendor of choice
to deliver a given capability, an OpenFlow-based SDN application will typically run on all
OpenFlow enabled devices just as soon as it is released.
Hardware advancements are also expected to happen more quickly as OpenFlow and
SDN are more broadly adopted. For example, more switches and routers that are only
responsible for the data plane will be implemented on general-purpose servers. As
soon as Intel or AMD release a new processor, new serverspresumably with higher
performancewill immediately be available to serve as faster switches and routers.

Enterprise Benefits
One of the more common trends in enterprise IT in recent years has been the
virtualization and consolidation of data centers. While most of the focus has been on
server virtualization, network virtualization has an important role to play as well. When
virtual machines (VMs) move from one host to another all related network traffic must
follow immediately. A network virtualization application on an OpenFlow network will
enable immediate and automated reconfiguration of forwarding rules. This keeps traffic
flowing to the right VMs and their applications. It also avoids sending traffic to a host
that should no longer receive it.

SPIRENT WHITE PAPER

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

Service Provider Benefits

Network service providers (NSPs) also have much to gain with SDN and OpenFlow. In
fact, SDN and OpenFlow can be used for much more than controlling packet forwarding.
As packets travel through the network, service providers can request OpenFlow-enabled
devices to log a variety of information. This can be used to get an end-to-end view of
their entire network, including firewalls, deep packet inspection (DPI) systems, switches
and routers. NSPs will likely use a variety of SDN applications to add or extend passive
capabilities such as traffic monitoring and active capabilities such as bandwidth
steering. They should also be able to gain better control over traffic flows from
individual customers and perhaps even put that control in the hands of their customers.

Summing up the Benefits

Earlier we discussed the three key challenges of traditional networking, including
cost, agility and design. Together, SDN and OpenFlow offer improvements in each of
those areas:
CostSDN networks can reduce CapEx by using lower cost traffic-forwarding
devices built with generic x86 servers. Centralized management and control
enabled by SDN also reduces ongoing OpEx
AgilitySDN networks can overcome the more static limitations of current
network devices, supporting faster rollouts of new services, as well as, more
rapid, on-demand provisioning and reprovisioning of existing services
DesignSDN networks support nearly every configuration and topology. This is
particularly helpful for virtualized and cloud environments and other on-demand
service models.

SPIRENT WHITE PAPER

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

MOVING SDN FOREWORD

SDN is in the process of moving into production networks. However, adoption is still
fairly limited. At the same time, there are already some good proof points to be found.
For example, Google has been vocal in its use of SDN. The company has rolled out an
SDN implementation across its WAN, saying the WAN is now higher performance, more
fault-tolerant, and cheaper.
While new network devices that support SDN are coming to market all the time, most
existing devices do not support OpenFlow or SDN. A number of organizations are
working to increase adoption and ensure interoperability. The two most commonly
known organizations are:
Open Network Foundation (ONF)The mission of ONF is to commercialize and
promote SDN and the underlying technologies as a disruptive approach to
networking that will change how virtually every company with a
network operates
InCNTREThe Indiana Center for Network Translational Research and
Education (InCNTRE) at Indiana University is a hub of education, research,
training and development for the adoption of OpenFlow and other standardsbased Software-Defined Networking (SDN) technologies
Keep each of them in mind as you plot your course toward SDN.

SPIRENT WHITE PAPER

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

EMERGING TEST CHALLENGES

Software defined networks do not always behave intuitively. In fact, depending on
the configuration, they may seem to break the rules of traditional IP networks. Robust
testing is the only reliable way to ensure they are working properly. Fortunately, some
testing methods remain the same for SDN and OpenFlow networks. For example, end-toend tests in these networks are quite similar to tests in more traditional networks. What
has changed is the internal plumbing.
One significant change in the plumbing is the shift from hardware-only implementations
to a combination of hardware and software. SDN and OpenFlow networks rely much
more on software running on general-purpose servers. This means a certain amount
of fundamental testing must be revisited. Do MPLS rules and policies still behave as
expected? How do these nodes handle line rate traffic? What happens under extreme
load conditions?
The simple fact that SDN and OpenFlow are relatively new also drives additional testing
challenges. APIs, protocols and vendor implementations must all be thoroughly tested
to ensure compliance with standards, as well as, interoperability between various
implementations. The addition of new SDN applications also drives the need for more
testing. Since these applications change the behavior of the network, validation testing
should take place for each new application and each new application revision.
Other forms of testing are also critical. These include performance, availability, security
and scale testingalso known as PASS.
PerformanceThe addition of more software-based components has the
potential to impact performance on the network. This may include throughput
and latency during normal network conditions and under stress conditions.
AvailabilityWith the centralization of the control plane, SDN controllers
become an important aspect of network availability. They must keep up with
changes communicated by applications and devices, even during periods of
rapid change.
SecurityUnapproved applications should not be able to change network
configurations through the SDN controller. Similarly, rogue entities should
not be able to change individual device configurations. Fuzz testing can be an
important element of security testing in the SDN world.
ScaleScale cuts across many dimensions in SDN and OpenFlow networks. Not
only must controllers scale to handle large networks, they must scale to keep
up with large numbers of requests that can arrive simultaneously from network
devices and SDN applications.

SPIRENT WHITE PAPER

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

SDN/OPENFLOW TESTING
For example, the main components when testing an OpenFlow 1.0 network are:
An OpenFlow 1.0 Controller
One or more OpenFlow 1.0 Switches
Secure Channels connecting each switch to the controller
Fortunately there are test methodologies. Although constantly growing, the existing
ones include:

Secure Channel Connection

The Secure Channel is the mechanism for establishing and maintaining communication
between an OpenFlow 1.0 Controller and one or more OpenFlow 1.0-enabled switches.
The proper operation of the Secure Channel by the switch is critical to the successful
deployment of an OpenFlow network because it is the channel through which the
OpenFlow Controller configures, manages, receives events and sends packets out
through the switches.

Flow Table Push

Each OpenFlow 1.0 switch uses a flow table to perform packet matching and forwarding.
For a switch to operate properly, it must be able to accept flows pushed down from the
OpenFlow Controller. It must also properly match incoming packets, modify the packets
correctly if specified, and forward them out of the proper switch port.

Flow Timeout Test

The Flow Entries pushed to an OpenFlow switch hove optional timeouts that can be
used to remove flows after a period of time expires. A flow entrys hard timeout is used
to remove a flow regardless of number or frequency of packet matches. When the hard
timeout expires, the flow is removed by the switch. A flow entrys idle timeout is used
to remove a flow after a period of time when there is no activity. Both timeouts are
important for efficient operation to ensure the switch has only the latest flows.

Barrier Request Message Response

Upon receipt of a Barrier Request, an OpenFlow-enabled switch must delay processing
subsequent OpenFlow protocol commands until all commands received prior to the
receipt of the Barrier Request have been completed. Barrier Request/Response is the
way an OpenFlow 1.0 Controller ensures that all inter-flow dependencies have been
satisfied, and that all commands sent to the switch have been processed.

Flow Table Scale Test

In an OpenFlow network of even moderate size, the number of unique flow entries can
be quite large. An OpenFlow-enabled switch, therefore, must be capable of properly
handling a large number of unique flow table entries
At the time of creating this white paper, SDN/OpenFlow 1.3 was released. The available
methodologies will increase to cover the enhancements offered by the latest release.

SPIRENT WHITE PAPER

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

CONCLUSIONS
Existing networks are under pressure and facing a number of challenges. These include
the continued growth of network traffic, the rise of mobile devices and the increased
consumption of streaming video. Emerging network deployment scenarios, such as
virtualized and cloud environments inside hyper-scale data centers, are also straining
existing network technologies and architectures.
Traditional network technologies are less than ideal when it comes to cost agility and
design flexibility. These gaps are driving the need for a new approach to the network.
While new and improved hardware is often the answer to existing hardware challenges,
software and APIs will be the answer this time. SDN and OpenFlow will enable lower
CapEx and OpEx, faster rollout and provisioning of on-demand services, and flexible
designs that better support virtualized and cloud environments.
At the same time, SDN and OpenFlow will introduce a number of new challenges, many
of them within the domain of network testing. The increased use of software in network
devices will require fundamental testing to be revisited. Rather than just using siliconbased switches and routers with clearly specified throughput and latency measures,
network engineers will also use general-purpose servers as traffic for workers. This
will require testing and certification using the PASS-methodology to ensure that
performance, availability, security and scale are all validated. APIs, protocols and
vendor implementations must all be thoroughly tested to ensure compliance with
standards as well as interoperability between various implementations.
In order to increase your odds of success, be sure to select and use test tools that
support SDN and OpenFlow. Chosen tools should also support all elements of the PASS
methodology to ensure healthy network operation from-end to-end.

At Spirent Communications we work behind the scenes to help the world communicate
and collaborate faster, better and more often. The worlds leading communications
companies rely on Spirent to help design, develop and deliver world-class network
devices and services.
Spirents lab test solutions are used to evaluate performance of the latest technologies.
As new communication services and applications are introduced in the market, Spirent
provides tools for service management and field test to improve troubleshooting and
quality. Spirent also enables enterprises, institutions and government agencies to
secure and manage their networks.
To learn more about SDN/OpenFlow testing and how Spirent can help with your testing
requirements, please visit: http://www.spirent.com/Networks-and-Applications/
OpenFlow.

SPIRENT WHITE PAPER

Testing Challenges for Modern Networks Built

Using SDN and OpenFlow

SPIRENT WHITE PAPER

10