Virtualization of Video Streaming Functions

Saarland University
Faculty of Natural Sciences and Technology I

Department of Computer Science
Master Thesis
Virtualization of Video Streaming Functions

Submmited by: Birhan Tadele Teklehaimanot
Advisor
Goran Appelquist
Supervisor
Prof. Dr.-Ing. Thorsten Herfet
Reviewers
Prof. Dr.-Ing. Thorsten Herfet
Prof. Dr. Dietrich Klakow
April 25, 2016

Eidesstattliche Erklärung
Ich erkläre hiermit an Eides Statt, dass ich die vorliegende Arbeit selbstständig verfasst und
keine anderen als die angegebenen Quellen und Hilfsmittel verwendet habe. Ich erkläre hiermit
an Eides Statt, dass die vorliegende Arbeit mit der elektronischen Version übereinstimmt.
Statement in Lieu of an Oath
I hereby confirm that I have written this thesis on my own and that I have not used any other
media or materials than the ones referred to in this thesis. I hereby confirm the congruence of
the contents of the printed data and the electronic version of the thesis.
Saarbrücken, on April 25, 2016
Birhan Tadele Teklehaimanot
Einverständniserklärung
Ich bin damit einverstanden, dass meine (bestandene) Arbeit in beiden Versionen in die Biblio-
thek der Informatik aufgenommen und damit veröffentlichtwird.
Declaration of Consent
I agree to make both versions of my thesis (with a passing grade) accessible to the public by
having them added to the library of the Computer Science Department.
Saarbrücken, on April 25, 2016
Birhan Tadele Teklehaimanot

Abstract
Edgeware is a leading provider of video streaming solutions to network and service operators.
The Edgeware Video Consolidation Platform(VCP) is a complete video streaming solution con-
sisting of the Convoy Management system and Orbit streaming servers. The Orbit streaming
servers are purpose designed hardware platforms which are composed of a dedicated hardware
streaming engine and a purpose designed flash as a storage system. The Orbit streaming server
is an accelerated HTTP streaming cache server which have up to 80 Gbps bandwidth and can
stream to 128000 clients from a single rack unit. In line with the new trend of moving more and
more functionalities towards a virtualized or software environment, the main goal of this thesis
is to make a performance comparison between Edgeware’s Orbit streaming server and one of the
best generic HTTP accelerators(reverse proxy severs) after implementing logging functionality
of the Orbit on top of it. This is achieved by implementing test cases for the use cases that can
help to evaluate those servers. Finally, after evaluating those proxy servers Varnish is selected
and then compared the modified Varnish and Orbit to investigate the performance difference.
iii
Acknowledgements
First and foremost, I would like to express my heartfelt gratitude to my supervisor Prof. Dr.-
Ing Thorsten Herfet for providing me an opportunity to write my thesis with him. My sincere
thanks to Göran Appelquist, for his patience and invaluable guidance. During our periodic
discussions, his constructive suggestions have helped me to gain such a wonderful experience on
doing my thesis. Besides, I have learned a lot while working with him. Furthermore, I would
like to thank my immediate family, specifically my father and my mother for helping me to
reach here even though the culture in our village is difficult to send girls to school. Last but
not least, my wholehearted gratitude to my brothers, sisters and my husband for their love,
encouragement, endless motivation and support.
Contents
Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
1 Introduction 1
1.1 Multimedia Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Streaming Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Traditional HTTP Download Technologies . . . . . . . . . . . . . . . . . . 2
1.2.2 True Streaming Technologies . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.3 HTTP Adaptive Bitrate Streaming Technologies . . . . . . . . . . . . . . 4
2 Background and Related Works 6
2.1 Content Delivery Networks(CDN) . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.1 Components of CDN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.1 Forward proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 Transparent proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.3 Reverse proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3 The Orbit streaming server 12
3.1 Edgeware solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.2 Edgeware Orbit server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.3 Proposed solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4 General comparisons of HTTP reverse proxy servers 15
4.1 Squid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.1.1 Pros and cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.2 Apache traffic server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
v
vi Contents
4.2.1 Pros and cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.3 Nginx . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.3.1 Pros and Cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.4 Varnish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.4.1 Pros and cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.5 Aiscaler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.5.1 Pros and cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5 Test methodology 21
5.1 Definition of test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.1.1 Live test case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.1.2 100% cache hit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.1.3 90% cache hit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.2 Implementation of test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.3 Configuration of proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.4 Test Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
5.4.1 Test setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
5.4.2 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6 Performance comparison of Orbit and proxy servers 29
6.1 Live test case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
6.2 100% cache hit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.3 90% cached . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.3.1 Nginx . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.3.2 Varnish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.5 Implementation of logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.6 Orbit and modified Varnish performance comparison . . . . . . . . . . . . . . . . 43
6.6.1 Varnish results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6.6.2 Orbit results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Contents vii
7 Conclusion 49
List of Figures 52
List of Tables 55
Bibliography 57
List of Abbreviations
CDN Content Delivery Network
HLS HTTP Live Streaming
HDS HTTP Dynamic Streaming
SSL Secure Socket Layer
ATS Apache Traffic Server
QoS Quality of Service
ASF Apache Software Foundation
FMS Flash Media Server
VoD Video on demand
RTMP Real Time Message Protocol
WMS Windows Media Services
TCP Transport Control Protocol
IIS Internet Information Services
ATS Apacche Traffic Server
CAPEX Capital Expenditure
OPEX Operational Expenditure
VCP Video Consolidation Platform
QoE Quality of Experience
GPL General Public License
VCL Varnish Configuration Language
MSE Massive storage engine
ix
Chapter 1
Introduction
1.1 Multimedia Streaming
Internet was originally designed to support data traffic transmission. Later in the early 1990’s,
the need of multimedia transmission emerges due to the growth of Internet in terms of users,
applications and nodes. Nowadays multimedia content is composing of large portion of the
internet traffic as more and more users are accessing multimedia content over the internet. For
instance, the share of mobile video traffic will reach up to 67% in 2017 [7] and it is estimated that
the multimedia transmission will cover up to 90% of the internet traffic within the next few years
[6]. Multimedia includes text, still images, audio, animation and video in an integrated manner.
Multimedia streaming refers to transmission of multimedia content from streaming sender to

a streaming receiver in a compressed form without downloading the whole content at the re-
ceiver device. The basic difference between multimedia streaming and textual data transfer
is that multimedia streaming requires real-time delivery but can tolerate certain amount of
data loss. Moreover, the main components of a multimedia streaming system are the encoder,
streaming server, streaming client, media transfer protocol and the underlying physical network.
The minimal set actions performed by the streaming system are, first camera captures and
produces either still images or video. The output from camera can be raw media without any
compression and this requires a very large bandwidth as it’s size is very large compared to
compressed version of it. Therefore, the media should be compressed using appropriate com-
pression techniques at the sender end to transmit over the internet. The compressed media
are then stored in the server together with its metadata which is the description of the media
such as location and timing information about the media. Hence, when the receiver requests
a certain media, the sender sends that media and its metadata. The media is received in the
form of packets and reassembled to the original compressed stream at the receiver end. Then,
1
2 Chapter 1. Introduction
the decoder takes this compressed stream and it decodes the media. Finally, the decoded media
is subsequently passed to the renderer for display.
In general, depending on the media, streaming can be classified into on-demand and live stream-
ing. In on-demand streaming media is pre-recorded, compressed and stored on the streaming
server and delivered to clients when requested. On the other hand, in live streaming media is
captured, compressed and transmitted on the fly.
1.2 Streaming Technologies
In this section the background of the most widely used streaming technologies is described in
short.
1.2.1 Traditional HTTP Download Technologies
Traditional HTTP Download is the basic technology for transmission of content over internet.
It uses HTTP protocol to download the content to the receiver device and then it play out
locally. The most commonly used traditional HTTP download methods are the following:
1.2.1.1 HTTP Download
HTTP Download is widely used technology for data transfer. When using http download the
content is downloaded and stored on the receiver’s device and playback does not begin until the
media is downloaded completely which may cause delays for large media.
1.2.1.2 HTTP Progressive Download
HTTP progressive download is a widely used media streaming technology and it uses HTTP/TCP
protocol. In progressive download, contents are downloaded partially and progressively stored
on user’s device [14] which improves HTTP Download method by reducing the delay before
playback begins. First, the metadata which tells the player how to play the media is down-
loaded. Then, playback begins after the metadata and sufficient data has been buffered on the
receiver’s device and the rest of the content is continued to be downloaded and save while the
player plays what is already downloaded data. However, there is no bandwidth adaption since
it does not consider the variation of network condition between client and server. In addition,
it cannot be used to stream a live media as it needs offline preparation and can be inefficient in
controlling the bandwidth from the ISP point of view [9].
Chapter 1. Introduction 3
1.2.1.3 HTTP Pseudo Streaming
HTTP Pseudo Streaming is very similar to HTTP Progressive Download, except that the player
can seek forward or backward even the content is not yet downloaded. The player uses byte
offset or number of seconds from the start of the video to find the desired part of video. The
player can buffer the content without saving it in the receiver’s device and it is not mandatory
to download the video from start to finish which means the player can stop the stream and
jump to different point.
1.2.2 True Streaming Technologies
True streaming technologies are the most popular streaming protocols for Flash and windows
media streaming. These technologies create a connection to a dedicated media server and the
content is sent to the end-user device as a series of small packets. True Streaming technologies
use a stateful protocol, which means from the first time a client connects to the streaming server
until the time it disconnects from the streaming server, the server keeps track of the client’s
state. Commands like PLAY, PAUSE and STOP can be issued by the end-user for playback
control and multi-bitrate delivery is supported. However, it is not common to switch between
bitrates once streaming has started. The two most common methods are Adobe Flash Media
Server and Microsoft Windows Media Server.
1.2.2.1 Adobe Flash Media Server
Adobe Flash Media Server (FMS) uses proprietary data from Adobe Systems (formerly Macro-
media) and is a hugely popular streaming platform. FMS is commonly installed on a media or
origin server running linux operating system; however, FMS can be supported on the Windows
Server operating system. FMS supports stored Video on Demand(VOD) and live media delivery
[14].
1.2.2.2 Microsoft Windows Media Services
Microsoft Windows Media Services (WMS) supports both VOD and live media delivery. WMS
is normally installed on a media or origin server running the Windows Server 2003 or 2008
operating system; however, there are proprietary variants which run on non-Windows servers.
WMS has the ability to enforce authentication and impose connection limits. The preferred
protocol for WMS is Real Time Streaming Protocol(RTSP) [14].
4 Chapter 1. Introduction
1.2.3 HTTP Adaptive Bitrate Streaming Technologies
HTTP adaptive bit-rate streaming is currently the most sophisticated method for streaming
media delivery [13]. The content is encoded at multiple bit rates allowing selection of differ-
ent encoded versions during streaming based on the available network bandwidth and client
resources. For each encoded version, the content is divided into a series of smaller segments
or chunks each with 2-10 seconds in length which are reassembled and played back as a single
continuous stream. This makes very easy for the receiver player to jump forward or backward
in the video. Eventually, a manifest file is created to act as a table of contents for the segments.
The quality of the segments can be adapted during streaming based on network quality condi-
tions which means player is able to switch seamlessly between the different fragments at any time
during the playback, so the player can select the desired quality level and adjust automatically
based on real-time network conditions. Therefore, during streaming only the relevant segments
are requested by the player and sent for reassembly and playback on the receiver device. An
additional benefit of HTTP adaptive bit rate streaming is the ability to utilize CDNs to cache
video content closer to end-viewers.
In addition, Apple HTTP live streaming, adobe HTTP dynamic streaming, microsoft IIS smooth
streaming and MPEG dynamic adaptive streaming over HTTP are the most widely used ex-
amples of HTTP adaptive bit rate streaming technologies.Each of them are described briefly as
follows.
1.2.3.1 Apple HTTP Live Streaming
Apple HTTP live streaming (HLS) is an HTTP-based media streaming technology imple-
mented by Apple in 2009 [15]. In HLS, video and/audio inputs are typically encoded using
H.264/Advanced audio coding(AAC) and then the stream segmenter breaks the stream into a
series of short segments that are saved as transport stream files(TS files)along with an index
file (.m3u8) which indicates the order of the TS files. These TS files are stored on a standard
HTTP web server for distribution along with the URL for the index file. The receiver player
begins by fetching the index file using its URL and then reading it to request the appropriate
segments, and displays content without any pauses or gaps as a continuous stream. HLS is
optimized for delivery to IOS devices and Safari browsers, but there are solutions on all other
platforms as well, with different quality.
1.2.3.2 Adobe HTTP Dynamic Streaming
HTTP Dynamic Streaming(HDS), is Adobe’s method for adaptive bitrate streaming and it sup-
ports live and on-demand delivery of MP4 media over regular HTTP connections. HDS allows
Chapter 1. Introduction 5
for adaptive streaming over HTTP to any device that’s compatible with Adobe Flash orAdobe
Integrated Runtime(AIR). It converts the content into a fragmented MP4 file format and deliv-
ers high-definition video/audio using any Flash compatible codec (H.264/AAC, VP6/MP3).On
demand content is converted using file packager utility as a post processing step; whereas, live
streams are converted real time using Adobe’s live Packer utility. An open source file specifica-
tion is used for fragmentation with .F4M for the manifest (index) file and .F4F for the media
segments which are stored in a single file. Moreover, adobe’s HTTP origin module is installed
on a standard HTTP web server to handle fragment requests and the streams are played using
Adobe Flash Player 10.1 or AIR.
1.2.3.3 Microsoft IIS Smooth Streaming
IIS(Internet Information Services) Smooth Streaming is Microsoft’s HTTP adaptive streaming

technology which uses Microsoft Silverlight as an application framework similar to Adobe Flash.
It supports multiple audio and video codecs. IIS is installed on an origin server running Windows
Server 2008 and Smooth Streaming is a plug-in software module to IIS. In IIS, media content
is stored as single file format with the segments stored as fragments MP4 within the file [14].
1.2.3.4 MPEG-Dynamic Adaptive Streaming over HTTP
MPEG Dynamic Adaptive Streaming over HTTP(DASH) is a standard which is defined by

MPEG to enable the interoperability between severs and clients of different vendors. It is a
generic solution based on HLS, HDS and Microsoft IIS Smooth Streaming. DASH is introduced
in order to standardize these proprietary solutions and have interoperability among themselves.
As a result, client applications can use all the streaming formats of different proprietary solutions
[10]. DASH, like HLS, HDS and Microsoft IIS Smooth Streaming, uses the concept of segments
and the equivalent of a playlist or manifest file, known as a Media Presentation Description
(MPD) file. Moreover, DASH can treats the video stream as a single file (does not create
segment files), so the MPD file points to offsets in the origin file rather than to segment files
[13].
Chapter 2
Background and Related Works
In this chapter the concept of content delivery networks (CDN) is explained along with it’s
components. Furthermore, the concept of proxy servers and how caching work with reverse
proxy servers are also introduced in a brief way to insure a better understanding of this work.
2.1 Content Delivery Networks(CDN)
High bandwidth requirement and rate variation of videos in compressed format introduces
some challenging issues to the end-to-end delivery over wide area network. As a consequence,
Content Delivery Networks has been evolved in order to overcome these challenges and improve
accessibility of the Internet. CDN is a large, geographically distributed network of specialized
servers that accelerate the delivery of web content and rich media to Internet-connected devices
[8].
The main concept at the basis of this technology is the delivery at edge points of the network, in
proximity to the request areas, to improve the user’s perceived Quality of Service (QoS) when
accessing Web content. This concept of CDN uses edge caching, which entails storing replicas
of static text, image, audio, and video including various forms of interactive media streaming
content in multiple servers around the ”edges” of the Internet, so that user requests can be
served by a nearby edge server rather than by a far-off origin server. The purpose of caching is to
reduce network traffic to a minimum; this is achieved by delivering content from caches as close
to the requesting user as possible but also by ensuring the delivery device has effectively cached
the content from previous requests [14]. CDNs typically host static content including images,
video, media clips, advertisements, and other embedded objects for dynamic Web content.
Typical customers of a CDN are media and Internet advertisement companies, data centers,
Internet Service Providers (ISPs), online music retailers, mobile operators, consumer electronics
manufacturers, and other carrier companies.
6
Chapter 2. Background and Related Works 7
Figure 2.1: CDN Architecture [8].
2.1.1 Components of CDN
Most CDN architectures are constructed from the following key components:
• Content delivery component: It contains origin server and a set of edge servers (cache
servers) to replicate the content and deployed as near as possible to the users. The origin
servers are the master sources for the content and can be deployed within the operator’s
network or more commonly within a content owner’s infrastructure. The primary purpose
of content delivery component is to deliver data to end users.
• Content distribution component: moves content from the origin server to cache servers
and ensures consistency. These can be deployed in a hierarchical model to allow tiered
caching and protection to any origin servers.
• Request-routing component: Direct user requests to cache servers and interact with the
distribution component to keep the content fresh.
• Accounting component: maintains logs of client accesses and records usage of the servers
assists in traffic reporting and usage-based billing.
2.2 Proxy servers
A proxy server is an intermediary server that intercepts requests from clients seeking resources
from different severs across the internet. Those resources can be images, files, web page, video,
audio,etc. Proxy server facilitates communication between clients and servers and can filter
requests based on its various rules. It can allow or reject communications by validating the
8 Chapter 2. Background and Related Works
requests against the available rules. There are different kinds of proxy servers, we will see here
three basic types of proxy servers although the focus of this work is on reverse proxy server.
2.2.1 Forward proxy servers
A forward proxy server intermediates traffic between client and the destination chosen by the
client. It enables a client to connect to a remote network to which it normally does not have
access. Moreover, it can also be used to cache data, reducing load on the networks between the
forward proxy and the remote web server. A forward proxy cache needs explicit configuration
of the browser to direct all requests to the proxy cache rather than the target web server.
2.2.2 Transparent proxy servers
Transparent proxy cache achieves the same goal as forward proxy cache, but operates trans-
parently to the browser. The browser does not need to be explicitly configured to access the
cache. Transparent caches are especially useful to ISPs, because they require no browser setup
modification. Moreover, they are the simplest way to use a cache internally on a network,
because they don’t require explicit coordination with other caches. However, many companies
like YouTube are currently trying to prevent the use of transparent proxies since they want to
have a full control of the communication between the service providers and clients.
2.2.3 Reverse proxy servers
Reverse proxy, also known as Web Server Accelerator is an intermediary server which stores
responses from the origin server( a server that contains the content) in its cache and serve
subsequent requests for the same content from this cache. It proxies on behalf of servers and
appears to the end users as the origin server. The origin servers will never be accessed from
outside since every request for the origin server is passed through the reverse proxies. When a
client requests some content, the DNS will route the request to the reverse proxy server instead
of the origin server. The reverse proxy checks the content in its cache, if not it connects to the
origin server and fetches the requested content to its cache and serves the users. Requested
content can be fetched from one or more origin servers but for the user it looks like a content of
one server. Reverse Proxy servers check validity of the stored data using the additional HTTP
headers received from the origin server. In addition, origin server controls weather a given
content should be cached by the proxy server or not using HTTP headers.
Reverse proxy server checks weather the requested data is in cache and still valid when it receives
a request on behalf of a server. So, if the content is not in the cache it forwards the request to
the origin server. Moreover, if the data is in cache but not valid, it deletes the content from the
cache and forwards the request to the origin server. On the other hand, if the data is in cache
and it is still valid the reverse proxy forwards the requested data to the client from its cache.
It also checks weather the response is cacheable or not before storing the content to its cache,
when it receives a response from the origin servers.
Reverse proxy reduces load on the origin server rather than reducing upstream network band-
width on the client side as forward and transparent proxy servers. Reverse proxy handles all
traffics before it can reach the origin server by sitting between the client and the origin server.
Reverse proxy servers are used to reduce bandwidth usage and improve performance by stor-
ing static contents like images, videos, audios, etc on its cache and then serving users without
going to the origin server. This can also help to offload a very busy server and to reduce the
response time and enhance customer’s browsing experience. Moreover, proxy servers protect
origin servers and act as additional defence against security attacks because they intercept
requests to the origin servers.
2.2.3.1 How caching works on reverse proxy
Clients always use HTTP when talking to a caching proxy(reverse proxy) even if the application
is an FTP transfer.
• Is it cacheable
A response is called cacheable if it can be used to answer a future request. A cache decides
whether a particular response is cacheable or not by checking some parts of the request and
response. In particular it check the following: the response status code, request method,
response cache-control directives, a response validator and request authentication. More-
over, in some of the caches, valuable responses(frequently requested) are cacheable than
those requested once. The most important HTTP header tags which are used by reverse
proxy to check validity of the cached content and to check whether the response from the
origin is cacheable are:
– Last-Modified:-Tells the proxy when the file was last modified.
– Expires:- Tells the proxy when to drop the file from the cache.
– Cache-Control: Tells the proxy if the file should be cached.
– Pragma: Also tells the proxy if the file should be cached.
• Definition of Cache Hit and Miss

When a cache receives a request, it checks to see if the response has been cached. If it is
found in the cache we call it cache hit, otherwise we call it cache miss. When the object
is found, the cache has to decide if the stored is fresh or stale. A cached response is called
fresh if the expiration time has not been reached; otherwise it is stale. Moreover, fresh
10 Chapter 2. Background and Related Works
response will be send to the client immediately; on the other hand stale responses require
validation from the origin server.
Hit ratio is used in order to measure the effectiveness of cache. It refers to the percentage
of requests that are satisfied as cache hits and usually it includes the validated and non
validated hits.
• Cache replacement policies

Cache replacement refers to the process of removing the old responses when the cache is
full and there is need of space for the new ones. Usually cache assigns some kind of value
to the object cached and the least valuable are removed first. Although, the definition
of “valuable“ differs from cache to cache. Typically, an object’s value is related to the
probability that it will be requested again, thus maximizing the hit ratio. The most known
cache replacement algorithms according to caching researchers are listed below.
1 Least recently used (LRU) This algorithm is the most popular replacement algorithm
that provides high performance in almost all situations. It removes the objects that
have not used for long time. This algorithm can be implemented by a simple list and
every time an object is accessed it will be on the top of the list. The least recently
accessed object will automatically be moved to the bottom of the list.
2 First in First out (FIFO) FIFO replacement algorithm is even simpler to implement
than LRU. In this algorithm objects are removed in the order they are added to the
cache.
3 Least Frequently used (LFU) LFU is similar to LRU. However, instead of selecting
objects only in terms of time since access and it also considers the number of access
as a significant parameter. LFU replaces objects with small access count and keeps
objects with high access account.
4 Size A size based algorithm uses the object size as the primary removal criteria. This
means that the largest object is removed first from the cache. However, the algorithm
needs an algorithm that measures how old an object stayed in the cache in order to
remove old objects first. Otherwise, the cache will only have smaller objects.
5 GreedyDual-Size(GDS) GDS assigns a value for every object based on the cost of the
cache miss and size of the object. Since GDS doesn’t specify what ”cost ” means,
it offers a lot of flexibility to optimize what you want. For example cost can be
defined as the latency which is the time it takes to receive the response. It can be
also defend as the number of packets transmitted over the network or the number of
hops between the origin server and the cache.
6 Greedy-Dual Size Frequency Greedy–Dual Size Frequency policy is proposed to max-
imize hit and byte hit rates for WWW proxies. This Proposed caching strategy
incorporates the main characteristics of a file such as file size, file access frequency
and recentness of the last access. This algorithm is an improvement of Greedy-Dual-
Size algorithm current champion among the replacement strategies proposed for Web
proxy caches. In general, GDF-like replacement policies emphasizing frequency have
better byte hit ratio but result in worse file hit ratio.
Chapter 3
The Orbit streaming server
This chapter starts with an introduction to Edgeware solutions and then the Orbit server
features and functionalities are explained which are mainly based on Edgeware’s marketing
material. Finally, we describe the proposed solution on this thesis work.
3.1 Edgeware solutions
Edgeware is a leading provider of video streaming solutions to network and service operators.
It has a special platform for providing this solutions called Edgeware Video Consolidation
Platform(VCP). VCP is a highly accelerated and consolidated platform which significantly
reduces the high infrastructure costs required for the delivery of TV and video services [11].
VCP is a highly scalable platform to deliver a high quality video services to any screen, across
any network topology. It supports all major adaptive streaming frameworks such as Microsoft,
Apple and Adobe. It also provides CDN (described in 2.1) video delivery management functions.
VCP reduces capital expenditure of a company by at least 50 % [11]. As shown in 3.1 below, VCP
contains a highly accelerated video origin, called the VCP Origin, and a widely deployed operator
Content Delivery Network (CDN) solution, called the VCP Edge. The VCP Origin solution
Figure 3.1: Components of Edgeware Video Consolidation Platform [11]
12
Chapter 3. The Orbit streaming server 13
reduces complexity, performance requirements and cost of the origin servers by offloading all
recording, ingest, re-packaging and play out capacity [11]. Therefore, load balancers or complex
file systems are not required. The VCP Edge is based on Edgeware’s widely deployed Distributed
Video Delivery Network (D-VDN) solution, incorporated into the new Video Consolidation
Platform [11]. VCP Edge is the main part of Edgeware video consolidation platform and it
addresses the network infrastructure costs of networks service providers and operators which
are growing due to the rapid increase of TV and video demand over the Internet. Moreover,
VCP Edge is a highly optimized CDN caching and distribution solution for a wide range of
service applications. It is designed to deliver next generation video services with the highest
Quality of Experience (QoE) and scalability to any screen [11]. In addition, VCP Edge is
easily integrated with any content management, middleware, conditional access and resource
management systems. The VCP Edge has two components called Orbit hardware platform
and convoy management software which are fully integrated. The convoy management software
allows an operator to set-up and manage a complete video delivery network across any network
topology. It ensures efficient and effective configuration, content management, license control,
session management, monitoring, and an open integration framework in close integration with
the optimized Orbit Delivery Servers.
3.2 Edgeware Orbit server
The Orbit servers are fully integrated with Convoy Management Software, providing highly
scalable asset propagation, session management and fault tolerance. The Orbit servers offer
advanced capabilities for operators and content providers to offer a full range of Cloud TV and
video services, irrespective of network topology and core bandwidth. These servers use a com-
bination of a dedicated hardware streaming engine and a purpose designed flash-based storage
system, coupled with a Linux based control plane to give up to 80 Gbps or 128 000 streams
from a single unit.
The main functionalities of the Orbit server are Ingest, repackaging, encryption, caching and
streaming. Repackaging and encryption are done just in time. The Orbit platform have func-
tionalities like session handling, logging and backend selector.
• Backend selector: the origin servers that contains the content in Edgeware are organized as
a group of servers which contains a set of nodes which are a model of physical computers.
Each node contains a set of IP addresses that models network interfaces. The server groups
model data center. The main functionality of the backend selector is load balancing and
fail-over. The load can be spread randomly over servers in a group, or be based on the
content requested to optimize cache utilization.
14 Chapter 3. The Orbit streaming server
• Session handling: is a module which is used to limit access to the content, for setting
maximum number of TCP connections per session(client) and to group related requests
i.e during a fragmented streaming and when user sends HTTP requests with a few seconds
apart.
• Session Logger: This logging is enabled when the session handling module is enabled and
it logs:
– TIMESTAMP: Time, relative to epoch, when gathering of data for the sample
started.
– DURATION: Duration from first request to the last transfer.
– SENDTIME: The time spend streaming from the video server.
– IP: IP address of the client (remote host) that initiated the session.
– SESSIONID: The identifier of a session.
– CONTENT: URI of the initial request.
– BYTES: Number of bytes transferred from the video server.
– REFERRER: The Referer HTTP request header, if provided by the client.
– USERAGENT: The User-Agent HTTP request header, if provided by the client.
3.3 Proposed solution
The main aim of this thesis is to replace this Orbit sever hardware with an implemented software
version and then see the impact of performance in Edgeware solution. To do this we will select
three proxy servers using their general behaviour and then implement test cases for the use
cases that can help to evaluate those three proxy servers. Finally, after evaluating the three
proxy servers we will select one proxy server that will use as a cache server and then implement
some of the Orbit server functionality explained in 3.2 on top of this selected web server(reverse
proxy server).
Chapter 4
General comparisons of HTTP

reverse proxy servers
HTTP reverse proxy servers are proxy servers that intercept HTTP requests coming from the
clients as it is described in 2.2.3. In this section the most commonly used HTTP reverse proxy
servers i.e squid, Apache traffic server, Nginx, Varnish and aicache are compared based on
performance, flexibility and license in a general way. However, it is not easy to find up to
date research papers on the reverse proxy servers and every proxy server adds new features and
functionalities overtime. Therefore, comparing those reverse proxy servers is not an easy task.
Moreover, their performance depends on the implementation, architecture and the flexibility
room to optimize them. In this chapter the pros and cons of each reverse proxy server is
described one by one. In addition, we tried to find some benchmarks on the performance
measurement of those servers from existing companies which are currently using those proxy
servers if any. Finally, we have selected three reverse proxy servers for further evaluation and
testing.
4.1 Squid
Squid is originated from the Harvest project in 1990s and it is the oldest and well-known of the
popular HTTP reverse proxy servers [2]. It is open source software licensed under the GNU
GPL and it supports HTTP, HTTPS, FTP. Squid offers a rich access control, authorization
and logging environment. It runs on many platforms including Linux, FreeBSD, and Microsoft
Windows and it typically runs as a single-process, single-threaded, asynchronous event proces-
sor. Squid stores the content in RAM until the RAM is full and then on disk. Therefore, the
RAM size and disk speed are important factors for its performance. Thousands of web-sites
around the internet use Squid to considerably increase their content delivery [2].
15
16 Chapter 4. General comparisons of HTTP reverse proxy servers
The cache replacement algorithms which are used in squid are LRU described 2.2.3.1, Greedy-
Dual Size Frequency (GDSF) which keeps smaller objects in cache, Least Frequently Used
with Dynamic Aging (LFUDA) that keeps poplar objects in cache regardless of thier size thus
optimizes byte hit rate at the expense of hit rate since one large, popular object will prevent
many smaller, slightly less popular objects from being cached.
4.1.1 Pros and cons
Squid has the following advantages:
• Caching of static objects. These are served much faster, assuming that your cache size is
big enough to keep the most frequently requested objects in the cache.
• Buffering of dynamic content
• Nonlinear URL space/server setup. Squid can be used to do some tricks with the URL
space and/or domain-based virtual server support.
• Features: Squid is richer than any available reverse proxy servers in features
The disadvantages are:
• Buffering limit for log records: Squid could not keep more than 64KB in its buffer log.
• Speed: Squid is not very fast when compared with the other reverse proxy servers available
today. Only if you are using a lot of dynamic features then there is a reason to use Squid,
and then only if the application and the server are designed with caching in mind.
• Memory usage: Squid uses quite a bit of memory. It can grow three times bigger than
the limit provided in the configuration file.
• Stability: Compared to the other reverse proxy servers, Squid is not the most stable.
• Scalability: Squid is limited in scalability on modern multi-core systems since it runs as

a single process, single threaded, and asynchronous event processor.
4.2 Apache traffic server
Apache traffic server was originally developed by Inktomi, and later donated to the Apache
server foundation (ASF) by Yahoo. Apache traffic server is a fast, scalable and feature rich
proxy server [1]. It has feature rich plugin APIs to develop extensions. Since it is a multi-
threaded event driven server, it combines asynchronous event processing and multi-threading
technologies to deal with concurrency. Apache traffic server can draw benefits from each tech-
nologies but it also makes the code and the technology complex and sometimes difficult to
Chapter 4. General comparisons of HTTP reverse proxy servers 17
understand. Apache traffic server is free and open source software that has robust plugin APIs
to extend and modify its behaviours and functionalities. It scales very well in modern multi-core
systems because it is a multi-threaded event driven proxy server.
There are a small number of “worker threads” in Apache traffic server; each such worker thread
is running its own asynchronous event processor. In a typical setup, this means Traffic Server
will run with around 20-40 threads only. This is configurable, but increasing the number of
threads above the default (which is 3 threads per CPU core) will yield worse performance due
to the overhead caused by more threads [18]. In ATS, the cache eviction algorithms that the
RAM cache supports are, LRU, LFU and Clocked Least Frequently Used by Size ( CLFUS)
which balances recentness, frequency, and size to maximize hit rate. The default algorithm is
CLFUS, but the user can select and set it in the configuration of ATS. Besides, Apache traffic
server uses a FIFO algorithm to update it’s disk cache.
In Yahoo CDN apache traffic server deliver 350,000 requests per second and 30 Gbps(95 per-
centile ) on which there are around 100 servers distributed over all the world. Additionally in
their lab they got 105,000 request per second out of one cache for small content and 3,6 Gbps
out of one server for large content. In addition to this Comcast use ATS in their CDN.
4.2.1 Pros and cons
The advantages of using Apache:
• It is very scalable, which means it needs little configuration and it can work in many
modes
• It is easily adapts to your network
• It uses efficient subsystem storage
The disadvantages are:
• It has many configuration files.
• It is not stable compared to others.
• it needs restart for some cases
4.3 Nginx
Nginx is an HTTP web server that also can function as a HTTP reverse proxy server. It is free
and an open source software that has a lot of plug-ins and released under a BSD-like license.
Nginx uses event driven multiple processes to solve the concurrency problem which needs small
CPU [3]. In addition to HTTP, it can proxy several other TCP protocols, and also have a
flexible plugin interface for extensions and additions of its behaviour and functionalities. It is
also well documented and widely available compared to the other reverse proxy servers. Nginx
uses a persistent disk based cache and the OS page cache keeps object in RAM. Moreover,
Nginx uses LRU cache replacement policy to evict contents from it’s cache if the specified size
for the cache exceeds.
4.3.1 Pros and Cons
• It has high performance and is stable with simple configuration.
• Consumes lower CPU power and memory
Disadvantages:
• Nginx requires a recompile of the entire application for new plugin APIs.
• Has some latency in accepting new connections
• Storage time is unlimited
4.4 Varnish
Varnish is a free and open source software which is licensed under two-clause BSD license. It
was initiated in 2005 and the first version was released in 2006 [18]. It focuses mainly on per-
formance and flexibility. Varnish was designed originally as a reverse proxy with the principle
of solving real problems, optimize for modern hardware (64-bit, multi-core, etc) and modern
workloads, work with the kernel not against it, innovation not regurgitation [19]. It takes the
advantage of modern kernel features to simplify the code. Moreover, Varnish does not keep
track of whether your cache is on disk or in memory. Instead, Varnish will request a large
chunk of memory and leave it to the operating system to figure out where that memory really
is. The operating system can generally do a better job than a user-space program. In general
to get a simpler design and reduce the amount of work Varnish needs to do, but it sacrificed
portability. For example in 32-bit system the virtual memory address space is limited to 3GB
which limits the size of cache and number of concurrent users[4].
Varnish is developed and tested on GNU/Linux and FreeBSD and its development is governed
by Varnish Governance Board (VGB). Varnish moves a lot of complexities to the kernel by
using the advanced features of the operating system such as accept filters, epoll and kqueue.
All caching are done using virtual memory provided by the operating system and each active
Chapter 4. General comparisons of HTTP reverse proxy servers 19
connection uses up a thread. Besides, Varnish uses LRU cache replacement algorithm both in
RAM and disk to remove contents from the caches.
Varnish uses its own domain specific configuration language called varnish configuration lan-
guage which translate to C-code, compiled with a normal C compiler and then dynamically
linked directly into Varnish at run-time. Varnish configuration language is lightning fast and
gives freedom to system administrators by allowing developers to define their own policies rather
than constrained by vanish developers. Varnish also contains modules called vmods which makes
it easy to extend, add new functionalities to Varnish or integrate Varnish with other software,
such as database or other network services. Example Integration with GeoIP databases or de-
vice detection for mobile users.
Varnish has two processes called parent and child process. The parent process starts the child
process when the varnishadm daemon starts and recover it when it dies by any reason.
Varnish contains different subroutines such as vcl recv, vcl f etch, vcl pipe, vcl pass, vcl hit,
vcl miss, and vcl error. But most of the tasks of VCL can be performed by:
• vcl recv: receives the requests, parse them, makes decision of serving from the cache or a
backend etc. It is able to alter the headers as well.
• vcl f etch: This method is called when an object is retrieved from a back end. The
basic operations here are to change the header, change the backend if previous one was
unhealthy etc.
Varnish plus is the commercial version of Varnish which contains all features of Varnish plus
some. Varnish plus has a measured performance up to 20 Gbit per second per single server in
video and audio streaming and it can stream to as many as 6500 users from one single server
[20].
4.4.1 Pros and cons
The main advantages Varnish are:
• It is very flexible compared to other reverse proxy servers due to its own language VCL.
• Varnish gives you access to very detailed logs that are useful when debugging problems
without any cost.
• Developers can implement their own policies using VCL.
• It has modules called Vmods which are helpful to extend its functionalities.
Disadvantages of Varnish:
• Opens new thread for every connection use the advantage of the operating system to solve
this
• It is not portable for all systems because it is designed for modern hardware(64-bit, multi-
core, etc).
4.5 Aiscaler
Aiscaler is not a pure caching solution instead it is all-in-one application delivery controller
(ADC) solution which is normally installed as a reverse proxy on a dedicated machine. Some
features of aiscaler are caching, SSL offloading, DDoS protection, multiplexed session manage-
ment, mobile device detection and IP-based geocontent delivery. Aiscaler is easy to configure
and creates better user experience by increasing speed and availability of a web site by offload-
ing request processing from the web, reducing code complexity and reducing cost for servers,
reducing space, power and cooling. Aicache is a Linux application, custom written in C. it is a
right threaded application which means it use limited number of threads (processes)
4.5.1 Pros and cons
The advantages of Aicache which is a caching feature of Aiscaler are :
• High performance
• Has good configuration flexibility
• Support for real time alerting and responding
• Responses are cached in RAM no disk
• Low resource usage
Disadvantages of Aiscaler:
• Aiscaler performs well especially with a dynamic web sites. But in our case we need a
reverse proxy for static contents.
Pages load faster with over 250,000 RPS served directly from aiScaler. However, we are not
interested in aiscaler because it is all in one solution not pure caching solution.
4.6 Conclusion
Based on the advantages and disadvantages of the above HTTP reverse proxy servers, we
selected Apache traffic server, Nginx and Varnish for further evaluation and testing to see their
performance and behaviours.
Chapter 5
Test methodology
We need to define some test cases to evaluate the performance of the generic HTTP acceleration
servers(cache servers) which are defined in chapter 4 as well as the Orbit server demonstrated
in chapter 3. We can measure the performance of cache servers using cache hit, cache miss and
live test cases. But in the case of cache miss, since the cache server should fetch the content
from the origin server (the server that contains the content) the performance will depend both
on the cache server and the origin server from which the content is fetched. Therefore, due
to such limitations only live , 100% cache hit and 90% cache hit test cases are implemented
and evaluated as part of this thesis. Moreover, to compare the cache servers we need some
parameters (characteristics) as a performance measure. In this thesis the characteristics that
we have used to evaluate those cache servers and the Orbit server are response time, CPU usage
and network traffic (bandwidth). We will describe them more in 6.
The video assets that we have used in our test case are stored in an origin server so that the
proxy servers and the Orbit server will fetch those video assets. All assets are chunked into
different length of small fragments. The fragments are grouped based on their content bitrate.
In other words, an asset contains different length of many fragments with a different sizes.
Therefore, we should state number of assets, number of fragments and their length and the
quality(content bitrate) of the fragments as an input in all test cases.
This chapter starts with the definition of test cases that we have used for our evaluation.
Following the definition, the implementation of those test cases is explained and then, the
configuration of the generic proxy servers which are selected in chapter 4 will be demonstrated.
Finally setup of the test environment is summarized in short.
21
22 Chapter 5. Test methodology
5.1 Definition of test cases
As it is mentioned in the above we have used three test cases namely, live, 100% cache hit and
90% cache hit test cases to evaluate the performance of the three reverse proxy servers selected
in chapter 4 and Orbit server. But, for Apache traffic server, the 90% cache hit test case is not
implemented due to the bad result collected from the live and 100% cache hit test cases. In the
case of 100% cache hit and 90% cache hit test cases, the video transmission was not a real video
on demand(VoD) since all clients were synchronized to spread over the assets and request the
fragments of the assets sequentially i.e all clients were requesting the same portion of the assets
at the same time. However, in a real VoD different clients can request different portion of the
assets at any time. The size of each fragment that we have used in our test cases was 132 KB.
5.1.1 Live test case
In this test case all clients were requesting a single asset with 5 second fragments at 300 kbps.
The asset was not in cache before the test so that all servers fetch and serve the first request for
a fragment from the origin server and then serve the next requests for the same fragment from
their cache. Proxy servers put data in memory to increase performance by decreasing the access
of hard disk. Every client was requesting the fragments sequentially and continuously as they
are represented in figure 5.1. Fragment length is the time that a client waits for that fragment
before requesting the next fragment which means if the client didn’t receive the fragment within
5 seconds its timeout will reach and the request will be a late request. In this test case, we have
tested the maximum number of clients(streams) in each server. In addition, the response time,
CPU usage and egress bandwidth are tested having the same number of clients for all servers.
Figure 5.1: Live fragments

Chapter 5. Test methodology 23
5.1.2 100% cache hit
In this test case all assets were saved in the cache of the proxy servers and the Orbit server
before the test and all clients were requesting those assets to simulate video on demand stream-
ing. In the proxy servers, the clients were spread over the assets and request the fragments in
the asset sequentially. Proxy servers put data in RAM cache as much as possible to decrease
the load of disk access and to keep contents in RAM for fast access. In the case of ATS the user
can specify the size of this RAM in its configuration. However in Varnish and nginx contents
will be stored in RAM as much as there is free space in the RAM. The assets are chunked into
5 second fragments with a 300 kbps content bitrate. Figure 5.2 represents the fragments in this
test case. The length of the fragments is a time that a clients waits to receive that fragment as
it is stated in 5.1.1. The focus of this test case was to see the number of assets that each proxy
server can serve without hitting any limitation which depends on their resource usage and then
having the maximum assets that can fit in the RAM cache of each proxy server, we have tested
and compared the response time, CPU usage and bandwidth of all servers.
Figure 5.2: 100% cached fragments.

5.1.3 90% cache hit
To see the ingest performance of the proxy servers, 90% test case is implemented. In this test
case, all clients spread over the assets and request the fragments sequentially the same as the
100% cache hit test case. The reverse proxy servers were serving 90% of the requests from
their cache and fetch and serve the rest from the origin. Figure 5.3 represent the 90% cache hit
fragments on which the fragments that are saved in the cache before the test are represented
by a green color and the other one represents the fragments that are not in the cache server.
Figure 5.3: 90% cached fragments.

5.2 Implementation of test cases
The test cases are implemented using Python and bash script. The implementations are based
on an in-house tool called rq which is used to generate TCP load. In those implementations rq
is configured with different parameters like the destination hosts and ports, number of clients,
number of video assets, number of fragments in each asset, quality and fragment length. In
the implementations, a reporter module is used to generate output of the test results in a pdf
format. Besides, VMstat and network statistics are collected from the proxy servers machine to
produce CPU usage and network traffic streaming respectively. For each test case, the number
of assets and fragments are varying as can be clearly seen in the table 6.2. However, for the
100% and 90% cache hit test cases, the video on demand property were not simulated exactly
as it is stated in section 5.1.
5.3 Configuration of proxy servers
In this work, each proxy server was configured as a cache server. The configuration is different
for each proxy server and the default configuration could not give us a reliable result for the
implemented test cases discussed above. Therefore, a lot of optimizations were made for each
proxy server to get a result which is comparable with Orbit server.
In the case of ATS we have configured it as reverse proxy server that can be used as a cache
server. It is only configured for the live and 100% cache hit test cases. The main configuration
files are called records, remap, storage and cache. Records configuration of ATS is used to
configure the server to act as a reverse proxy server and set how much RAM should be used to
store the most accessed assets.It also sets the IP address and port on which it should be accessed
and the IP address to access the origin server and some more tunings. In remap configuration
file we have defined both map and reverse-map rules. A map rule translates the URL in the
client requests into the URL of the origin server where the content is located. It constructs
a complete request URL from the the client URL and its headers and then looks for match
with its list of targeted URLs in the remap rules. Besides, a reverse-map translates the URL of
redirect responses from the origin server into the address of ATS, so that clients are redirected
to ATS instead of accessing an origin server directly. Therefore, clients cannot access the origin
server without knowledge of Apache traffic server. Furthermore, storage configuration is used
to set how much hard disk will be used by Apache traffic server. Likewise, some caching rules
are set in the cache configuration file.
Varnish uses its own configuration language called varnish configuration language(VCL) to con-
figure it. This language was used to tell Varnish which origin server it has to use and all required
caching rules. Unlike Apache traffic server, VCL is a very flexible configuration language. More-
over, another configuration file was used to tweak Varnish, to set the IP address and port on
which Varnish should listen.
In the case of Nginx, it has its own way of configuration and caching rules are defined to configure
Nginx as a reverse proxy server. In addition, we have defined the address of the origin server,
the storage to be used, the address and port on which Nginx listens and many optimizations in
the configuration file.
5.4 Test Environment
The test environment that we have used in our test case contains the following components:
client machine, Orbit server, origin server and switch. Only the Orbit machine is replaced by
the proxy server’s machine when we are testing proxy servers.
5.4.1 Test setup
The proxy servers were installed and configured one by one on the same machine so that the
test environment will be the same for all test cases. All the components of the test environment
are demonstrated in short as follows.
• Client machine with Ubuntu 12.04

Application called rq is installed in this machine. Rq is an in-house purpose built perfor-
mance streaming application which is used to generate TCP load. It can do progressive
streaming and adaptive bitrate streaming. Rq can be configured with number of clients,
number of assets, number of fragments, fragment length, ramp-up, timeout, duration, etc
and it creates tcp sockets and send GET requests based on the implementation of the test
cases. In this machine, two network interfaces each with 10 Gbps were used to communi-
cate with the proxy servers machine. In addition, rq can emulate thousands of concurrent
HTTP clients.
• Proxy server with Ubuntu 12.04

This is a machine on which we installed the reverse proxy servers to be tested. Those
are used as a cache server, when a request comes from a client the proxy server checks
if the requested data is in cache. If data is in cache it serve the client from the cache,
otherwise it fetches the data from the origin server and serve the clients simultaneously.
In this machine, we have 11 GB RAM and we specified 50 GB disk space to do the test
and two 10 Gbps interfaces were used, one is shared by the client machine and the origin
server and the other one with the client machine only. All proxy servers uses RAM cache
to serve objects as quick as possible and reduces load on disks in addition to the specified
cache storage. Therefore, memory and CPU are the basic constraint in the proxy servers.
This machine is replaced with Orbit server during the Orbit tests runs.
• Origin server with Ubuntu 12.04

In order to support HTTP requests, lighttpd web server is installed and configured in this
machine. Video segments of different bit rates are stored on this server and this is the
server from which reverse proxy servers get the content when there is a cache miss.
• Switch
The above components are connected as shown in the figures 5.4 and 5.5 through switch
both in the Orbit and proxy servers test environment.
Figure 5.4: Orbit test. Figure 5.5: Proxy servers test.

5.4.2 Parameters
The parameters that we have used in all test case are, number of clients, number of assets, num-
ber of fragments in each asset, ramp up in milliseconds, timeout in seconds, quality, fragment
length, duration in minutes, content type which is video. The value of each of the parameters
varies according to the specific test cases as it can be seen in table 5.1 below. We have used
10000 number of fragments in the live test case to make sure the proxy servers download the
first request from the origin server and serve the next requests from its cache to simulate the
live streaming. However, in the 100% cache hit and 90% cache hit test case there are 180 frag-
ments(the duration of the test(900 seconds) divided by the fragment length(5 seconds)) in each
asset. The number of assets are different for all proxy servers in the 100% cache hit and 90%
cache hit test cases due to the drawback of disk access and memory limitation that we have in
the proxy servers machine. In other words, the number of assets that a proxy server can serve
depends on its memory and CPU usage.
Parameters Live test case 100% cache hit 90% cache hit
Clients 25000 25000 25000
Fragments 10000 180 180
ramp up 2ms 4ms 4ms
Quality 300 kbps 300 kbps 300 kbps
fragment length in seconds 5s 5s 5s
durationin minutes 15m 15m 15m
Table 5.1: Parameters of test cases.
As part of the test setup, different test cases are implemented in the client machine and the
proxy server machine was configured and optimized for each proxy server as it is described in
sections 5.2 and 5.3.
Chapter 6
Performance comparison of Orbit

and proxy servers
The results of the above test cases for the generic proxy servers and Orbit server are presented
in this chapter to compare response time, CPU usage and bandwidth of all servers. The com-
parison is accompanied with the analysis of each result. The Orbit, client and the proxy servers
machines have two interfaces with 10 Gbps each as they are stated in the figures 5.4 and 5.5.
As it can be seen in the figures below, in the response time there is a heat-map graph with
a blue color in it’s right side which represents the number of request in percentage. 0-20% of
the requests are represented with a light blue color and the weight of the blue color increases
proportionally with the number of requests in percentage. The y-axis represents response time
in milliseconds and x-axis is the duration of the test in percentage.
In the CPU usage figures, the red graph represents the time on which the CPU is idle, the blue
and green graph are the time of CPU that are spent waiting for the system and user respec-
tively. The graph which is represented by a cyan color is the CPU time waiting for the I/O
operation(disk access).
In the figures of network streaming ports, for the proxy server tests red and blue colors are used
to represent the egress bandwidth of both interfaces(since we have two interfaces) and green
and cyan colors are used for the ingest bandwidth. Besides, in the Orbit test blue and red colors
are used for both the egress and ingest bandwidths.
29
30 Chapter 6. Performance comparison of Orbit and proxy servers
6.1 Live test case
In this test case, 25000 clients were requesting a single asset which contains 10000 fragments
each with 5 second length at 300 kbps, and the ramp up between each clients was 2 milliseconds
to simulate the live test case. Orbit, Varnish and Nginx creates only one connection with the
origin server for each cache miss even many clients request the same fragment at the same time.
However, ATS might send more than one request per fragment to the origin server if many
client requests come at the same time for that fragment, but it tries to decrease the number of
connections to the origin server for the same fragment.
Proxy servers put data in RAM cache to increase performance by decreasing the access of disk
cache. Since the size of the fragments was 132 KB and we have a single asset in this test case
the overall size of the asset was 10000 times 132 KB which is 1.32 GB. Therefore, the size of the
RAM cache was enough to save all the fragments and proxy servers was accessing the fragments
from RAM cache in this test case.
In live test case, we encountered a limitation related with the number of concurrent requests
that every proxy server can support. As a result, the maximum number of concurrent requests
that Orbit, ATS and Varnish can support are 32000. When there are more than 32000 clients,
Varnish hits memory limitation and it restarts in the mid of the test. Moreover, Apache traffic
server will be very slow when the number of concurrent clients were more than 32000 and then
there will be many late requests. On the other hand, Nginx can support up to 45000 concur-
rent requests without any late requests which implies Nginx can handle many more concurrent
requests than Varnish and Apache traffic server with the same resources. The reason for the
different number of concurrent request in the proxy servers, is due to they way they used to
handle incoming requests. Nginx uses asynchronous event-driven connection handling algorithm
on which it does not create new thread for each request. However, Varnish is a multi-threaded
program that uses one thread per each connection and ATS uses a hybrid event-driven engine
with a multi-threaded processing model to handle incoming requests.
In addition to the number concurrent requests, we used response time, CPU usage and network
traffic as a main criteria to compare the proxy servers and the Orbit server. A test that runs
with 25000 clients is used for all proxy servers and the Orbit server in this section to evaluate
the results for the above criteria. The result of live test case for all proxy servers and Orbit
server are demonstrated as follows.
• Response time
Response time is the amount of time taken between a client request and receipt of the
response. Having enough memory, Varnish is the best reverse proxy server in terms of
Chapter 6. Performance comparison of Orbit and proxy servers 31
response time. It has comparable response time with the Orbit server as it can be seen in
the figures 6.3 and 6.4. However, in Nginx 6.2 and ATS 6.1 the response time of 0-20% of
the total requests reaches up to 1.1 and 2.4 seconds respectively. As a result, Varnish is
very fast compared to Nginx and Apache traffic server in live test case.
Figure 6.1: ATS response time. Figure 6.2: Nginx response time.
Figure 6.3: Varnish response time. Figure 6.4: orbit response time
• CPU usage
As it can be seen from the figures below, in Apache traffic server 6.5 and Varnish 6.7 the
CPU is 75% idle but in Nginx CPU is idle around 80%. Therefore, Nginx 6.6 uses less
CPU compared to Varnish and Apache traffic server in live test case. However, 95% of
CPU is idle in the Orbit server 6.8 since it doesn’t use CPU and memory to perform its
activities. It uses FPGA instead of cpu and RAM and flash memory as a storage.
Figure 6.6: Nginx CPU usage.

Figure 6.5: ATS CPU usage.
Figure 6.7: Varnish CPU usage. Figure 6.8: Orbit CPU usage
• Network traffic
As it can be seen below, the ingest bandwidth for each interface is around zero since all
the servers send a single request per fragment to the fetch from the origin server and the
egress bandwidth is 3.5 Gbps in all interfaces. The egress bandwidth starts from zero and
then gradually grows until all clients joined and it stays constant. The egress bandwidth
is around 7 Gbps for every server since each server has two interfaces as it is stated in the
figures 5.4 and 5.5 above.
Figure 6.9: ATS network traffic. Figure 6.10: Nginx network traffic.
Figure 6.11: Varnish network traffic. Figure 6.12: Orbit network traffic
Results of all servers for the live test case are summarized in the table below.
Servers Response time CPU usage Egress bandwidth

Orbit < 1 ms 10% 2*3.5 Gbps
Varnish < 1 ms 20% 2*3.5 Gbps
Nginx <= 10 ms 20% 2*3.5 Gbps
ATS <= 20 ms 30% 2*3.5 Gbps
Table 6.1: live test case results of all servers.
6.2 100% cache hit
In this test case all assets were saved in the cache before the test and the number of assets that
each proxy server can support differs due to their difference of memory and CPU usage. In this
test case, 25000 clients make requests for 5 second fragments with 300 kbps content bitrate.
There were 180 fragments in each asset and the size of every fragment was 132 KB. Moreover,
the rump up between the clients was 4 milliseconds and the test runs for 15 minutes in the
proxy servers and 10 minutes in the Orbit server.
As it is stated above the RAM size in the proxy servers machine is 11 GB and the proxy servers
put contents in the RAM cache to serve objects as quickly as possible and reduces load on disk
cache. However, since the RAM size in the proxy servers machine is not large enough to store all
the contents, the proxy servers put contents in the RAM cache as much as possible and access
the rest from the disk cache. As a result, the proxy servers perform very low when the RAM
cache is full and they start to access more assets from disk cache, even though enough disk
cache storage was specified in all proxy servers. In other words, when they start to access more
requests from disk cache CPU spends a lot of its time to wait the disk I/O operation(I/O wait).
Therefore, one of the criteria that we use to compare the proxy servers with each other was how
many assets can each proxy server support without late requests and hitting any limitation;
which depends on the memory usage of the proxy servers. The Orbit server uses flash memory
as a storage system hence, there is no limitation on the number of assets up to the size of the
storage system.
ATS can only support 32 assets without late requests in the given environment which are less
than 0.76 GB in size(32 times 180 fragments in each asset times 132 KB which is the size one
fragment). When the number of assets are increased, ATS will access the disk cache many times
and the overall time of the CPU is took by the disk I/O wait operation. The main reason that
ATS supports only few assets in our test cases was, ATS puts data in RAM cache only extremely
popular objects which means it only put a requested object in RAM cache if ATS thinks it is
accessed many times. In other words, when there are more assets the number of requests for a
fragment will decrease since the clients spread over the assets and request fragments sequentially
and then it may not be put in the RAM cache since ATS thinks it is not popular. However,
Varnish and Nginx puts an object in RAM cache if it is accessed at least once. Vanish has
two storage options namely malloc(memory) and file(disk) storage. But, we use the disk option
in our test to have the same criteria with the other proxy servers. In the disk storage option,
Varnish uses the OS page cache to put data in memory and keep contents in RAM for fast
access. The maximum number of assets that can be served by Varnish without hitting memory
limitation were 330 which are 7.8 GB. This is due to Varnish needs some memory to perform its
activities and it has an overhead of about 1 kB per object regardless of the size i.e if the number
of objects stored in Varnish increases the overhead might be significant and Varnish’s memory
usage will increase accordingly. In our case, Varnish was using around 3.2 GB of the RAM for
its activity and for the overhead needed for each fragment. Besides, if the number of assets are
increased, Varnish uses more RAM to save the assets and hit memory limitation for its activity
and the overhead needed so that it restarts automatically in the mid of the test. Moreover,
Varnish file storage is not a persistent storage which means it doesn’t retain the objects in the
cache if Varnish stops or restarts. In the other hand, Nginx can support 500 assets which are
11.88 GB in size which implies Nginx uses a very small memory for its activity and it was also
serving some fragments from the disk cache. Moreover, when the number of assets are increased
in Nginx, the limitation that we hit was CPU time(the CPU time waiting for the I/O operation
was increased accordingly). Therefore, the limitation for number of assets in Nginx was fully
due to the bottleneck of the disk access operation.
We have also compared the response time, CPU usage and bandwidth having limitation of the
number of assets for each proxy server stated in the above paragraph. To compare the proxy
servers and the Orbit in terms those characteristics, we have used 350 assets which are 8.3 GB
for Nginx to have the same environment with the others. In other words, all proxy servers were
serving all the assets from the RAM cache and there wasn’t any impact of disk I/O operation.
Having all assets in a RAM cache the response time, CPU usage and egress bandwidth of the
proxy servers and the Orbit are demonstrated below.
• Response time
Varnish is the best reverse proxy server in terms of response time. However, the response
time of 0-20% of the total requests in Nginx 6.14 and ATS 6.13 reaches 2 and 2.5 seconds
respectively. Therefore, Varnish is very fast 6.15 compared to Nginx 6.14 and Apache
traffic server 6.13 as for the live test case having enough RAM cache in the proxy servers.
As it can be seen in the figure 6.16 the Orbit server has very short response time and
there is no any impact on the performance even the number of assets are increased as it
is described in above paragraphs.
Figure 6.13: ATS response time Figure 6.14: Nginx response time
Figure 6.15: Varnish response time Figure 6.16: Orbit response time
• Network traffic
As it is shown the figures below, there is no any ingest as all the contents were saved
in the cache before the test and the egress bandwidth is zero when it starts and then
gradually grows to 3.5 Gbps in each interface. Therefore, the total bandwidth is 7 Gbps
in all servers.
Figure 6.17: ATS network traffic Figure 6.18: Nginx network traffic
Figure 6.19: Varnish network traffic Figure 6.20: Orbit network traffic
• CPU usage
The figures below represents the CPU usage of all servers, in Apache traffic server 6.21
and Varnish 6.23 70% of the CPU is idle but in Nginx 6.22 CPU is idle 75%. Therefore,
Nginx uses less CPU compared to both Apache traffic server and Varnish as live test case.
The Orbit server is not used CPU as it is stated in the live test case above and as it can
be seen in the figure 6.24 below.
Figure 6.21: ATS CPU usage. Figure 6.22: Nginx CPU usage.
Figure 6.23: Varnish CPU usage. Figure 6.24: Orbit CPU usage
The overall results for the 100% cache hit test case are summarized in the table below.
Servers Response time CPU usage Bandwidth

Orbit < 1 ms 15% 2*3.5 Gbps
Varnish <= 5 ms 30% 2*3.5 Gbps
Nginx <= 15 ms 25% 2*3.5 Gbps
ATS <= 27 ms 30% 2*3.5 Gbps
Table 6.2: 100% cached results of all servers.
6.3 90% cached
In this test case we only tested Nginx and Varnish due to the bad result collected from the
above test cases in ATS. Furthermore, Varnish uses virtual memory as a cache storage and
Nginx was serving many assets from disk, hence the main focus of this test case is to see the
ingest bandwidth of the proxy servers. Furthermore, this test case is not implemented for Orbit
and we did not test the Orbit server for this test case.
6.3.1 Nginx
In Nginx 32000 clients were requesting a 5 second fragments with 300kbps content bitrate. The
test runs for 15 minutes and the ramp up between the clients was 4 milliseconds. Nginx was
serving 90% of the assets from its cache and 10% from the origin server. The main focus was
the number of assets Nginx can support and the number of clients that can be served without
hitting any limitation and any late requests. Nginx can serve 550 assets each with 180 number
of 5 second fragments and it can have up to 60000 concurrent requests but there were some
late requests because the CPU was fully utilized and many actions were waiting CPU to be
performed. Therefore, we have used the test that run with 32000 clients in this case.
6.3.2 Varnish
As it is stated above Varnish can use virtual memory as a cache storage in addition to disk(file)
storage. Using this storage option, we got the following results for the 90% cache hit in Varnish.
This test was run using 32000 clients requesting 3000 assets each with 180 fragments of 5 second
length. The ramp up between the clients was 2 milliseconds and data rate 300 Kbps. Varnish
was serving 90% of the fragments from its cache and 10% from the origin server. The cache
was 5GB virtual memory and it was deleting the Least Recently Used (LRU) fragments when
the cache is full.
• Response time
As it is shown in the figures 6.25 and 6.26 , the response time of Nginx reaches up to 2.7
seconds for some requests but in Varnish only few request have response time of 9 second
in the first few seconds. Therefore, Varnish have good response time compared to Nginx
using virtual memory as a storage in the 90% cache hit test case.
Figure 6.25: Nginx response time. Figure 6.26: Varnish response time.
• CPU usage
Figures 6.27 and 6.28 represents the CPU usage of nginx and Varnish respectively. CPU
time spent waiting the disk(I/O wait) reaches 50% in nginx, since Nginx was accessing the
disk for many requests and CPU time waiting for the system is around 25%. Therefore,
the CPU is idle only 25% in the worst cases in Nginx. However, in varnish 55% of the
CPU is idle since it was using virtual memory and it was not accessing any disk.
Figure 6.27: Nginx CPU usage. Figure 6.28: Varnish CPU usage
• Network load
As it is shown in the figures below, the ingest bandwidth is around 0.96 Gbps and the
egress bandwidth is 4.5 Gbps for each interface for both Nginx 6.29 and Varnish 6.30.
Therefore, the total ingest bandwidth is 0.96 Gbps since we have only one interface con-
nected with origin and total egress bandwidth is 9 Gbps (we have two interfaces connected
with client)in both servers. Both ingest and egress bandwidth starts with zero and grad-
ually increases and then it stays constant after all clients joined.
Figure 6.29: Nginx network traffic. Figure 6.30: Varnish network traffic
6.4 Summary
We have selected Varnish based on the results of the above comparisons and the flexibility of
extending its functionality through Vmods.
6.5 Implementation of logger
In this section, we will describe the implementation of one of the features(modules) of the Orbit
called logger(session logger) on top of Varnish. There are also other functionalities on the Orbit
server like session management and backeend selector demonstrated in 3.2 but those function-
alities are not expected to have any major impact on the performance of Varnish and they are
already implemented on same or different way in Varnish also.
The logger module calculates the duration from the start of the first request to the end of the
last request and find the sum of bytes sent and send time in each minute of the test for all
sessions(clients). Varnish can give us the bytes sent and send time(time to finish the request)
informations for a single request from its logging but we need those values per every minute
for each session. A session can generate hundreds of requests in each minute and we have to
catch and add the bytes sent and send time for all requests in that interval. Duration is a
period between the start of the first request and the end(sum of start time and send time) of
the last request in that interval. In other words, duration is not exactly one minute, but rather
as close as possible to one minute, but only including those segments that have been completely
delivered during this sample interval. On average the sample duration will be one minute.
Logger is implemented as a Varnish module(Vmod) using C++ to see if there is any impact
on the performance of Varnish. Since Varnish puts all the logs in a shared memory, we have
written some configurations to log session ID, URL, start time of the request, time taken to
finish that request and the total bytes sent to the client in that request into a file instead of
shared memory. The current implementation takes this log file as an input and read every line
to extract the required columns for the implementation.
To implement the logger functionality, a session ID is added to the URL of each request in
the test case implementation of the client machine and rewrite rules are written in the VCL
configuration of Varnish to remove this session ID and send the correct URL to the origin server
since the URL to the origin server does not include the session ID. This session ID is assigned
for each client uniquely in the test case implementations and it is used to identify the clients in
our logger implementation. Logger logs for every session:
• Session ID: This an ID which is used to identify the session (the client) uniquely.
• IP: IP address of the client (remote host) that initiated the session.
• Bytes sent: is the total number of bytes that are sent to that session during the interval.
• Send time: is the total time spent actually streaming to the client, i.e. the sum of the
time spent fulfilling each HTTP request in the interval.
• Duration: is time from the beginning of the first request to the end of the last request in
that interval.
From those parameters we can approximate the download bitrate that the client got by dividing
the bytes sent to the send time and the content encoding bitrate by the division of the bytes sent
to the duration during the interval as they are shown in figure 6.31 below. This implementation
Figure 6.31
was a first, simplified, implementation to get initial results. To implement like it is implemented
in the Orbit server, the session handling feature should be implemented first and then the logger
takes its input from the session handling feature rather than from file.
6.6 Orbit and modified Varnish performance comparison
In this section we will describe the results of Varnish before and after the implementation of log-
ging in Varnish and then the results of the Orbit server with the same parameter. In the Orbit
server the logger is related with the session handling functionality. This means a session object
that contains session ID, bytes sent, send time and duration informations is created using the
session management module and then the logger module uses those values to generate the log
file described in 3.2. However, due to time constraint we implement logger without implement-
ing session handling functionality of the Orbit. Therefore, the result of the implementation is
not fully optimized and we can only show that there is an impact in the performance of Varnish
when the logger functionality is enabled. This is due to Varnish uses CPU and memory unlike
the Orbit server which uses FPGA instead of CPU and memory. The parameters that we have
used in this case are stated in the table below.
Parameters Live test case 100% cache hit

Clients 9800 9800
Fragments 10000 100
ramp up 0.5 ms 4 ms
Quality 2 MB 2 MB
fragment length in seconds 5s 2s
durationin minutes 10m 10m
number of assets 1 100
Table 6.3: Parameters of test cases.
6.6.1 Varnish results
The implemented logger plugin uses a lot of resource since it is reading and processing file to
get the expected result. This implementation is not fully optimized since it takes a log file as
an input. As a result, the results in figures below show only additional processing will have an
impact on the streaming performance of Varnish. The limitation here was the CPU, the logger
was taking a lot of CPU to read and process the file and then there was an effect in response
time of Varnish since most of the CPU was used by the logger module. All assets were served
from RAM cache and if we increase the number of assets more than 100 Varnish hits a memory
limitation stated in chapter 6.
• Response time
As it can be seen in the figures below there is an impact in the response time of varnish
after the logger implementation both in live and 100% cache hit test cases.
Figure 6.32: live without logger Figure 6.33: live with logger
Figure 6.34: 100% cached without logger Figure 6.35: 100% cached with logger
• CPU usage
The CPU usage of Varnish before and after of the logger implementation is different.
Varnish with the logger functionality uses more CPU both in the live and 100% cache hit
test cases as it can be seen in the figures below.
• Network traffic
There is no any difference in the bandwidth of Varnish with and without the logger
functionality.
6.6.2 Orbit results
Orbit uses flash memory storage therefore, there is no limitation in the number of assets since
it doesn’t use any RAM and CPU. The test results of both live and 100% cached test cases are
described in the figures below.
• Response time
The Orbit response time is not affected by the number of assets like Varnish.
Figure 6.44: orbit live Figure 6.45: orbit 100% cached
• CPU usage
Since Orbit does not used CPU and RAM to perform its activity, more that 90% of CPU
is idle in both live and 100% cache hit test cases.
• Network traffic

Chapter 7
Conclusion
Nowadays, multimedia transmission is growing rapidly due to high demand of users. As a result,
different multimedia streaming technologies are developed to support the high quality demand.
Moreover, a high bandwidth requirement and rate variation of videos in compressed format
introduces some challenging issues to the end-to-end delivery over wide area network. As a
consequence, Content Delivery Networks(CDN) has been evolved in order to overcome these
challenges and improve accessibility of the Internet. CDN contains cache server as a compo-
nent which is intermediary server which stores responses from the origin server( a server that
contains the content) in its cache and serve subsequent requests for the same content from its
cache that improves response time. In Edgeware VCP solution, this cache server is a purpose
designed hardware implementation called Orbit server.
The main focus of this thesis was to virtualize the Orbit server by implementing backend selec-
tor, session handling and logger features of the Orbit server on top of one the selected HTTP
acceleration servers(reverse proxy servers) based on performance comparison. However, the de-
fault configuration of the proxy servers couldn’t give us a valuable result that can be compared
with the Orbit server even before implementing the above functionalities of the Orbit server
on top of them. Therefore, we have optimized the proxy servers until they give us a valuable
result and see the limitations they have. Moreover, we have implemented only the logger feature
of the Orbit since the other functionalities are not expected to have any major impact on the
performance of Varnish.
In this work, the performance of Orbit server is compared with a software based reverse proxy
servers which are configured as video streaming cache servers. We have implemented live, 100%
cache hit and 90% cache hit test cases to evaluate and compare the performance of the proxy
servers with each other as well as with Orbit. Moreover, we have configured three reverse proxy
servers i.e ATS, Nginx and Varnish and optimize them until they give us a valuable result that
can be compared with Orbit performance test results. In order to do the performance com-
49
50 Chapter 7. Conclusion
parison response time, resource usage, network traffic and flexibility of extension were used as
fundamental criteria.
Nginx is best server in terms of resource usage i.e it uses small memory and CPU compared
to Varnish and Apache traffic server. However, Varnish is better in the case of response time
having enough memory. Varnish have malloc(virtual memory) and disk(file) storage options
but Varnish’s disk storage is not persistent i.e if Varnish stops or restarts, the information that
tells Varnish weather the data is in cache(cache key) will be lost and the data in cache will be
a random data.
In the cases of 100% cache hit, there was a limitation in the number of assets that every proxy
server can serve. This is due to the bad performance of disk access i.e CPU spends a lot of its
time waiting the I/O operation to finish. In other words, the number of assets that the proxy
servers can serve depends on the CPU and memory usage of the proxy servers. Nginx supports
more assets than Varnish and ATS since it uses low memory and CPU than varnish and ATS.
The RAM size in the proxy servers machine was 11 GB and the proxy servers put contents in
the RAM cache to serve requests as quickly as possible and reduces load on disks. However,
since the RAM size in the proxy servers machine is not large enough to store all contents, the
proxy servers put contents in the RAM cache as much as possible and access the rest from the
disk cache. As a result, the proxy servers perform very low when the RAM cache is full and
they start to access more assets from disk cache, even though enough disk cache storage was
specified in all proxy servers.
A raw disk is recommended in the documentation of ATS as cache storage, but we have used
a normal disk cache to use the same resources with the others. Therefore, ATS supports only
32 number of assets due to the way how it puts contents in RAM cache and it was worst in all
criteria both in live and 100% cache hit test cases.
Varnish needs more memory to perform its activities and it has an overhead of about 1 KB per
object regardless of the size i.e if the number of assets stored in Varnish increases the overhead
might be significant and Varnish’s memory usage will increase accordingly. Moreover, Varnish
will restart when it becomes out of memory in the mid of the test and since Varnish’s disk cache
is not persistent all the contents in the cache will be random data. As a result, Varnish serves
less assets than Nginx but it has the best response time in all test cases than Nginx and ATS.
Besides, Varnish is flexible than Nginx and ATS since it has its own configuration language and
it is easier to extend it through Varnish modules called Vmods. Therefore, we have selected
Varnish based on the results of the test cases and its flexibility for extension.
Chapter 7. Conclusion 51
Finally, the logger functionality of the Orbit server is implemented using C++ as a plugin(vmod)
of Varnish to see if there is any effect in its performance. However, since we have implemented
the logger functionality without implementing the session management feature like in the Orbit
server our implementation is not optimized and complete. Therefore, either implementing the
logger functionality in the same way as the it is implemented in the Orbit or using a shared
memory like Varnish do to log the information for every request may reduce the impact of the
logger in Varnish performance.
The main limitation on the software based reverse proxy server was the number of assets that
they can serve without hitting any limitation as it is described above. This is due to the load
of disk and memory limitation that we have in the proxy servers machine. However, since the
Orbit server uses flash memory as a storage there is no any limitation in the number of assets
it can serve and its performance is not affected by the number of assets. Therefore, since we
can not have unlimited memory the Orbit server is better than the proxy servers for a large
contents like video. For small contents like HTML and text the proxy servers perform very well
and can be used easily.
Varnish plus is a commercial version of Varnish that solves the storage problem with disk by
introducing a new storage option called massive storage engine(MSE). Varnish plus can store
almost unlimited contents(100+ terabytes)[30] in this MSE which is also persistence storage
unlike the disk storage of Varnish. Hence, in the future it will be important to compare perfor-
mance of Varnish plus and Orbit since Varnish has comparable response time with the Orbit
having enough memory for its activity and to save the assets in the RAM cache.
List of Figures
2.1 CDN Architecture [8]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1 Components of Edgeware Video Consolidation Platform [11] . . . . . . . . . . . . 12
5.1 Live fragments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.2 100% cached fragments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.3 90% cached fragments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.4 Orbit test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.5 Proxy servers test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6.1 ATS response time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.2 Nginx response time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.3 Varnish response time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.4 orbit response time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.5 ATS CPU usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.6 Nginx CPU usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.7 Varnish CPU usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.8 Orbit CPU usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.9 ATS network traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.10 Nginx network traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.11 Varnish network traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.12 Orbit network traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.13 ATS response time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.14 Nginx response time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.15 Varnish response time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
52
List of Figures 53
6.16 Orbit response time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.17 ATS network traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.18 Nginx network traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.19 Varnish network traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.20 Orbit network traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.21 ATS CPU usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.22 Nginx CPU usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.23 Varnish CPU usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.24 Orbit CPU usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.25 Nginx response time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
6.26 Varnish response time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
6.27 Nginx CPU usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
6.28 Varnish CPU usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
6.29 Nginx network traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.30 Varnish network traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.31 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.32 live without logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.33 live with logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.34 100% cached without logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.35 100% cached with logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.37 live with logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
6.41 live with logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.44 orbit live . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.45 orbit 100% cached . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

54 List of Figures
6.46 orbit live . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.47 orbit 100% cached . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.48 orbit live . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.49 orbit 100% cached . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

List of Tables
5.1 Parameters of test cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.1 live test case results of all servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.2 100% cached results of all servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.3 Parameters of test cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
55
Bibliography
[1] Apache traffic server, http://trafficserver.apache.org/ last accessed June 15, 2015.
[2] Squid, http://www.squid-cache.org/ last accessed June 15, 2015.
[3] NGiNX, http://nginx.org/ last accessed June 20, 2015.
[4] Varnish, https://www.varnish-cache.org/ last accessed June 20, 2015.
[5] Aicache, https://aiscaler.com/ last accessed June 25, 2015.
[6] Ali Begen, Tankut Akgul, and Mark Baugher. Watching video over the web: Part 1: Stream-
ing protocols.IEEE Internet Computing 15(2):54–63, March 2011.
[7] Cisco. Cisco visual networking index: Global mobile data traffic forecast update, 2012-2017,
2013.
[8] A.Pathan and R.Buyya A Taxonomy and Survey of Content Delivery Networks
[9] R. Dubin, O. Hadar, R. Ohayon, and N. Amram. Progressive download video rate traffic
shaping using tcp window and deep packet inspection May 2012.
[10] I. Sodagar. The MPEG-DASH Standard for Multimedia Streaming Over the Internet. In
IEEE Multimedia, pages 62 – 67, 2011.
[11] Edgeware, http://www.edgeware.tv/products/ last accessed on April 20, 2016
[12] L.Larson-Kelley, Adobe Media Server 5 White Pape August, 2012.
[13] NGINX.Inc, Serving Media with NGINX Plus April 11, 2016.
[14] Digital Media Delivery Platform Introduction to Content Delivery Networks SEPTEM-
BER, 2011.
[15] HLS protocol https://tools.ietf.org/html/draft-pantos-http-live-streaming-16 last accessed

May 20, 2015.
[16] HTTP Live Streaming Overview. https://developer.apple.com/library/ios/documentation/NetworkingInte

last accessed on May 20, 2015.
57
58 Bibliography
[17] EdgeCast.http://blog.edgecast.com/post/55198896476/hds-hls-hss-adaptive-http-
streaming last accessed May 20, 2015.
[18] L.Hedstrom, Apache Traffic Server Development Team Apache Traffic Server HTTP Proxy
Server on the Edge.
[19] White paper Varnish cache whitepaper November 2013.
[20] White paper HTTP Streaming with Varnish Cache.
[21] Logren Dély, Tobias. Caching HTTP: A comparative study of caching reverse proxies Var-
nish and Nginx 2014.
[22] Bakhtiyari, Shahab Performance Evaluation of the Apache Traffic Server and Varnish Re-
verse Proxies 2012.
[23] Nedelcu, Clément. Nginx HTTP Server: Adopt Nginx for Your Web Applications to Make
the Most of Your Infrastructure and Serve Pages Faster Than Ever. Packt Publishing Ltd
2010.
[24] Gannes, Liz (10 June 2009). ”The Next Big Thing in Video: Adaptive Bitrate Streaming”.
Archived from the original on 19 June 2010. Retrieved 1 June 2010.
[25] C. Müller, S. Lederer and C. Timmerer, “An Evaluation of Dynamic Adaptive Streaming
over HTTP in Vehicular Environments”, In Proceedings of the ACM Multimedia Systems
Conference 2012 and the 4th ACM Workshop on Mobile Video, Chapel Hill, North Carolina,
February 24, 2012.
[26] C. Mueller, S. Lederer, C. Timmerer DASH at ITEC, VLC Plugin, DASHEncoder and
Dataset.
[27] Duane Wessels. web caching. O’Reilly, June 2001.
[28] Charu Aggarwal, Joel Wolf and Philip Yu Caching on the World Wide Web. IBM T.J.
Watson Research Center, Yorktown Heights, New York.
[29] How Caching Works. by Guy Provost,last accessed April 15.
[30] https://www.varnish-software.com/plus/massive-storage-engine. last accessed April 13,

2016.

Virtualization of Video Streaming Functions

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Virtualization of Video Streaming Functions

Загружено:

Авторское право:

Доступные форматы

Saarland University

Faculty of Natural Sciences and Technology I

Virtualization of Video Streaming Functions

April 25, 2016

Statement in Lieu of an Oath

Saarbrücken, on April 25, 2016

Birhan Tadele Teklehaimanot

Saarbrücken, on April 25, 2016

Birhan Tadele Teklehaimanot

1.1 Multimedia Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Streaming Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 Traditional HTTP Download Technologies . . . . . . . . . . . . . . . . . . 2

1.2.2 True Streaming Technologies . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.3 HTTP Adaptive Bitrate Streaming Technologies . . . . . . . . . . . . . . 4

2 Background and Related Works 6

2.1 Content Delivery Networks(CDN) . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.1 Components of CDN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2 Proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2.1 Forward proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2.2 Transparent proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2.3 Reverse proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 The Orbit streaming server 12

3.1 Edgeware solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.2 Edgeware Orbit server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.3 Proposed solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 General comparisons of HTTP reverse proxy servers 15

4.1.1 Pros and cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.2 Apache traffic server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.2.1 Pros and cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.3.1 Pros and Cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.4.1 Pros and cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.5.1 Pros and cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5.1 Definition of test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5.1.1 Live test case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5.1.2 100% cache hit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

5.1.3 90% cache hit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5.2 Implementation of test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

5.3 Configuration of proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

5.4 Test Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5.4.1 Test setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

6 Performance comparison of Orbit and proxy servers 29

6.1 Live test case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

6.2 100% cache hit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

6.3 90% cached . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

6.5 Implementation of logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

6.6 Orbit and modified Varnish performance comparison . . . . . . . . . . . . . . . . 43

6.6.1 Varnish results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

6.6.2 Orbit results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

CDN Content Delivery Network

HLS HTTP Live Streaming

HDS HTTP Dynamic Streaming

SSL Secure Socket Layer

ATS Apache Traffic Server

QoS Quality of Service

ASF Apache Software Foundation

FMS Flash Media Server

VoD Video on demand

RTMP Real Time Message Protocol

WMS Windows Media Services

TCP Transport Control Protocol

IIS Internet Information Services

ATS Apacche Traffic Server

CAPEX Capital Expenditure

OPEX Operational Expenditure