Analysis of IPTables and Shorewall PDF

COMPARATIVE ANALYSIS OF
IPTABLES AND SHOREWALL
Muhammad Zeeshan Ahmad
This thesis is presented as a part of Degree of

Bachelor of Science in Electrical Engineering
Supervisor:
Raja M. Khurram Shahzad
School of Engineering
Blekinge Institute of Technology
371 79 Karlskrona, Sweden
Internet: www.bth.se/ing
Phone: +46 455 38 50 00
Fax: +46 455 38 50 57
Start by doing what's necessary, then what's possible; and suddenly you are doing the
impossible.
Saint Francis
ii
Abstract
The use of internet has increased over the past years. Many users may not have good intentions.
Some people use the internet to gain access to the unauthorized information. Although absolute
security of information is not possible for any network connected to the Internet however,
firewalls make an important contribution to the network security. A firewall is a barrier placed
between the network and the outside world to prevent the unwanted and potentially damaging
intrusion of the network.
This thesis compares the performance of Linux packet filtering firewalls, i.e. iptables and
shorewall. The firewall performance testing helps in selecting the right firewall as needed. In
addition, it highlights the strength and weakness of each firewall. Both firewalls were tested by
using the identical parameters.
During the experiments, recommended benchmarking methodology for firewall performance

testing is taken into account as described in RFC 3511. The comparison process includes
experiments which are performed by using different tools. To validate the effectiveness of
firewalls, several performance metrics such as throughput, latency, connection establishment and
teardown rate, HTTP transfer rate and system resource consumption are used.
The experimental results indicate that the performance of Iptables firewall decreases as compared
to shorewall in all the aspects taken into account. All the selected metrics show that large
numbers of filtering rules have a negative impact on the performance of both firewalls. However,
UDP throughput is not affected by the number of filtering rules. The experimental results also
indicate that traffic sent with different packet sizes do not affect the performance of firewalls.
Keywords:
Linux Iptables, Firewall performance, Comparison of Linux firewalls
iii
iv
Acknowledgements
In the name of Allah, the most Gracious, the most Compassionate
Foremost, I would like to express my sincere gratitude to supervisor Raja Muhammad Khurram
Shahzad for his continuous help, support, motivation and guidance during the project.
I am thankful to school of engineering at Blekinge Institute of Technology for allowing to use the
Network laboratory for the thesis.
I am grateful to my parents for their endless love and unconditional support during the studies
and life. Finally, I offer regards and blessings to all of those who supported me in the project in
any way.
Karlskrona 2012
Muhammad Zeeshan Ahmad
zeeshnets@msn.com
Table of Contents
1
INTRODUCTION .................................................................................................................. 1
1.1
Aim and Scope .................................................................................................................................................... 2
1.2
Evaluation Method ............................................................................................................................................. 2
1.2.1 Performance Metrics ........................................................................................................................................ 2

1.2.2 Hypotheses ....................................................................................................................................................... 2
1.3
Outline of the Project ......................................................................................................................................... 3
BACKGROUND ..................................................................................................................... 4
2.1
Working of a Firewall ........................................................................................................................................ 5
2.1.1 Packet Inspection .............................................................................................................................................. 5

2.2
Types of Firewalls ............................................................................................................................................... 7
2.2.1 Packet Filtering Firewall .................................................................................................................................. 7

2.2.2 Circuit Level Gateways .................................................................................................................................... 7
2.2.3 Application Level Gateways ............................................................................................................................ 8
2.2.4 Stateful Multilayer Inspection Firewall ............................................................................................................ 8
2.3
Linux Iptables ..................................................................................................................................................... 8
2.4
Firewalls Selection .............................................................................................................................................. 8
2.5
Limitations of Firewalls ..................................................................................................................................... 9
2.6
Related Work .................................................................................................................................................... 10
METHODOLOGY ............................................................................................................... 11
3.1
Requirements .................................................................................................................................................... 11
3.2
Experimental steps ........................................................................................................................................... 11
3.3
Traffic Classification ........................................................................................................................................ 11
3.3.1 Real-time (Online) Evaluation ........................................................................................................................ 11

3.3.2 Off-line Evaluation ......................................................................................................................................... 11
3.4
Traffic Duration ................................................................................................................................................ 12
vi
3.5
Evaluation Procedure ....................................................................................................................................... 12
3.6
Evaluation Tools ............................................................................................................................................... 13
3.6.1 Netperf ............................................................................................................................................................ 13

3.6.2 Iperf ................................................................................................................................................................ 14
3.6.3 THTTPd.......................................................................................................................................................... 14
3.6.4 HTTPing ......................................................................................................................................................... 14
3.6.5 Nmap .............................................................................................................................................................. 15
3.6.6 Webmin .......................................................................................................................................................... 15
3.6.7 Wireshark ....................................................................................................................................................... 15
3.7
Evaluation Metrics ........................................................................................................................................... 15
3.7.1 Throughput ..................................................................................................................................................... 17

3.7.2 Latency ........................................................................................................................................................... 18
3.7.3 Connection Establishment rate ....................................................................................................................... 19
3.7.4 Connection Teardown Rate ............................................................................................................................ 20
3.7.5 HTTP Transfer Rate ....................................................................................................................................... 20
3.7.6 System Resource Consumption ...................................................................................................................... 21
EXPERIMENT ..................................................................................................................... 22
4.1
Specifications of computers ............................................................................................................................. 22
4.2
Firewalls Installation ........................................................................................................................................ 22
4.2.1 Iptables ........................................................................................................................................................... 22

4.2.2 Shorewall ........................................................................................................................................................ 23
4.3
Configuration of Firewalls ............................................................................................................................... 23
4.3.1 Iptables ........................................................................................................................................................... 23

4.3.2 Shorewall ........................................................................................................................................................ 24
4.4
Implementation of Evaluation Metrics ........................................................................................................... 25
4.4.1 Throughput ..................................................................................................................................................... 25

4.4.2 Latency ........................................................................................................................................................... 26
4.4.3 Connection Establishment Rate ...................................................................................................................... 27
4.4.4 Connection Teardown Rate ............................................................................................................................ 28
4.4.5 HTTP Transfer Rate ....................................................................................................................................... 28
4.4.6 System Resource Consumption ...................................................................................................................... 29
RESULTS .............................................................................................................................. 31
5.1
Throughput ....................................................................................................................................................... 31
vii
5.2
Latency .............................................................................................................................................................. 34
5.3
Connection Establishment and Teardown Rate ............................................................................................. 37
5.4
HTTP Transfer Rate ........................................................................................................................................ 38
5.5
System Resource Consumption ....................................................................................................................... 39
5.6
Analysis .............................................................................................................................................................. 40
CONCLUSION ..................................................................................................................... 41
6.1
Future Work ..................................................................................................................................................... 42
BIBLIOGRAPHY ................................................................................................................ 43
viii
List of figures
Figure 1.1.1: A Firewall General Model ......................................................................................... 1
Figure 1.3.1: Basic Firewalls function ........................................................................................... 4
Figure 2.1.1: The OSI and the TCP/IP models ................................................................................ 5
Figure 2.1.2: An IPv4 TCP header ................................................................................................... 6
Figure 2.1.3: An IPv4 UDP header .................................................................................................. 6
Figure 4.2.1: Iptables Installation ................................................................................................... 23
Figure 4.2.2: Shorewall Installation ............................................................................................... 23
Figure 4.3.1: Iptables Rules Configuration .................................................................................... 23
Figure 4.3.2: Shorewall Zones ....................................................................................................... 24
Figure 4.3.3: Shorewall Interfaces ................................................................................................. 24
Figure 4.3.4: Shorewall Policies..................................................................................................... 24
Figure 4.3.5: Shorewall Rules Configuration ................................................................................. 25
Figure 4.4.1: TCP Throughput Calculation .................................................................................... 25
Figure 4.4.2: UDP Throughput Calculation ................................................................................... 26
Figure 4.4.3: TCP Latency Calculation .......................................................................................... 26
Figure 4.4.4: UDP Latency Calculation ......................................................................................... 27
Figure 4.4.5: Connection Establishment rate Calculation .............................................................. 27
Figure 4.4.6: Connection Teardown rate Calculation .................................................................... 28
Figure 4.4.7: Iperf Traffic Generation ............................................................................................ 28
Figure 4.4.8: Nmap View ............................................................................................................... 29
Figure 4.4.9: HTTPing status ......................................................................................................... 29
Figure 4.4.10: CPU and Memory usage Calculation...................................................................... 30
Figure 5.1.1: Iptables TCP Throughput.......................................................................................... 31
Figure 5.1.2: Iptables UDP Throughput ......................................................................................... 32
Figure 5.1.3: Shorewall TCP Throughput ...................................................................................... 33
Figure 5.1.4: Shorewall UDP Throughput ..................................................................................... 34
Figure 5.2.1: Iptables TCP Latency (Request/Response)............................................................... 34
Figure 5.2.2: Iptables UDP Latency (Request/Response) .............................................................. 35
Figure 5.2.3: Shorewall TCP Latency (Request/Response) ........................................................... 36
ix
Figure 5.2.4: Shorewall UDP Latency (Request/Response) .......................................................... 36

Figure 5.3.1: Iptables (Connect/Request/Response) and (Connect/Close) .................................... 37
Figure 5.3.2: Shorewall (Connect/Request/Response) and (Connect/Close) ................................. 37
Figure 5.4.1: Iptables HTTP transfer rate....................................................................................... 38
Figure 5.4.2: Shorewall HTTP transfer rate ................................................................................... 39
Figure 5.5.1: Iptables CPU/Memory usage .................................................................................... 39
Figure 5.5.2: Shorewall CPU/Memory usage ................................................................................ 40
List of Tables
Table 3.6.1: Evaluation tools and their usage in the project .......................................................... 13
Table 3.7.1: Performance Metrics and their importance [26] ........................................................ 16
Table 3.7.2: Difference between bandwidth and throughput ......................................................... 18
Table 4.1.1: Computer Specifications ............................................................................................ 22
xi
List of Acronyms
CPU
Central Processing Unit
GUI
Graphical User Interface
HTTP
Hypertext Transfer Protocol
I/O
Input/output
ICMP
Internet Control Messaging Protocol
IP
Internet Protocol
LAN
Local Area Network
MTU
Maximum Transmission Unit
NIC
Network Interface Card
OS
Operating System
OSI
Open Systems Interconnection
PING
Packet Inter Net Groper
PPS
Packets per Second
RFC
Request for Comments
RTT
Round Trip Time
SSL
Secure Sockets Layer
TCP
Transmission Control Protocol
UDP
User Datagram Protocol
URL
Uniform Resource Locator
xii
1 Introduction
The growth in computer systems and their interconnections via networks has increased the
dependency of both organizations and individuals on the information stored and communication
using these systems. This has led to the need to protect data and resources from unauthorized
disclosure, to guarantee the authenticity of data and to protect systems from attacks. The attacks
on networks are increasing and are becoming more and more complex. The ability to access and
transfer information in a few seconds allows the government, companies, educational institutions,
and individuals to accelerate the decision process. However, the information can be very valuable
and there is a need for better and faster security systems to protect information and networks. A
secure system is defined as1.
The only truly secure system is one that is powered off, cast in a block of concrete and sealed in
a lead-lined room with armed guards - and even then I have my doubts.
Eugene H. Spafford
Absolute security is not possible for any network connected to the Internet. However, firewalls
make a critical contribution in the network security. A firewall is a part of a computer system or
network that is designed to block an unauthorized access while permitting the authorized
communication. It works as a barrier placed between the network and the outside world to
regulate the flow of traffic between the computer networks.
It acts as the first line of defense against intruders [1] hence; it is located at the entry point of the
network. Figure 1.1.1 presents the general model of a firewall.
Figure 1.1.1: A Firewall General Model 2
1
2
http://spaf.cerias.purdue.edu/quotes.html
http://www.soporte21.com/que_es_un_firewall.php
1.1 Aim and Scope

The aim of this project is to evaluate the performance of Iptables and Shorewall firewalls. In
order to evaluate the packet filtering firewalls, the different performance requirements are met for
evaluation.
Since, this project is related to performance testing. Therefore, security issues of the firewalls are
not taken into account. No vulnerability tests or the security holes in firewalls are discovered.
Moreover, this project is limited to IPv4 only. IPv6 is out of scope of this project.
1.2 Evaluation Method

The firewalls performance is tested by recommended benchmarking methodology as described
in RFC 3511 of Internet Engineering Task Force (IETF) [2]. This methodology defines a number
of tests that are used to describe the performance characteristics of firewalls. In addition to the
tests, it also describes the specific formats for reporting the results.
1.2.1 Performance Metrics
To validate the effectiveness of both firewalls, several essential performance metrics are selected
from the mentioned methodology in the test plan, and the tests are designed to meet these
objectives. These metrics will evaluate the performance of firewalls by using different methods
and tools.
Throughput
Latency
Connection Establishment Rate
Connection Teardown Rate
HTTP Transfer Rate
System Resource Consumption
The benchmarking methodology does not give any information about the testing tools. Therefore,
several tools will be studied for this purpose.
1.2.2 Hypotheses
H0: There is no difference in both firewalls.
H1: Iptables is more efficient in terms of performance.
H2: Shorewall provides reliable performance.
2
The experimental results collected from all the mentioned performance metrics will help in
comparing hypothesis.
1.3 Outline of the Project

The rest of thesis is divided into five main chapters. These are as follows:
Chapter 2- Background provides an overview of related work in the area and comparison to this
project. It covers the Linux packet filtering mechanism and functionality of a firewall. In addition
to this it also discusses the selection and limitations of firewalls.
Chapter 3- Methodology discuss how and for what purpose the tests are performed, introduces
the design involved in such evaluation and methodologies of both firewalls. It gives information
about the tools used in evaluating firewalls. In addition to this it also tells the requirements of
devices, description and importance of selected metrics.
Chapter 4- Experiment covers the implementation and configuration of both firewalls. It also
deals with the sidelined aspects of this project, such as the time duration of the test. The
implementation of selected metrics, choice of a Linux system and experimental setup is also
included.
Chapter 5- Results discuss the outcome of each firewall. A discussion about analysis then
clarifies the outcome.
Chapter 6- Conclusion explains which of the hypothesis is selected and why. Discuss the future
work that could be done to further improve the outcome of firewalls.
2 Background
The firewall provides an additional layer of defense, insulating the internal systems from external
networks. This follows the classic military doctrine of defense in depth, [3] which is just as
applicable to IT security. A firewall examines all the traffic routed between two or more networks
according to certain criteria, if criteria are met traffic is routed between the networks, otherwise it
is stopped 3. There are two access denial methodologies used by firewalls i.e. allow traffic or
restrict. A firewall may allow all traffic through unless it meets specified criteria, or it may deny
all the traffic unless it meets certain criteria. The type of criteria used to determine whether the
traffic is allowed through depends on configuration of firewall done by the user. It can also use
complex rule bases to analyze the application data to determine if the traffic should be allowed.
Figure 1.3.1 presents the functionality of a firewall.
Figure 1.3.1: Basic Firewalls function 3
A Firewall prevents unauthorized access to a device or a network. Its function is to carefully

inspect data entering and exiting the device based on user configurations and ignore access that
comes from a suspicious network. It can be used to log all attempts to enter the private network
and trigger alarms when an unauthorized entry is attempted.
http://www.vicomsoft.com/learning-center/firewalls/
2.1 Working of a Firewall

To understand the working of firewalls, the first thing is having knowledge about how the
different layers of a network interact. The network architecture is designed around a seven layer
model [4]. Each layer has its own responsibilities and handles the data in a well-defined manner.
Figure 2.1.1 presents the network layer architecture.
Figure 2.1.1: The OSI and the TCP/IP models 4
Firewalls operate at different layers to use different criteria to restrict the traffic. In OSI model
this is the network layer, in the TCP / IP model it is the Internet protocol layer. This layer is
concerned with routing the packets to their destination. At this layer firewall determine whether a
packet is from a trusted source, but does not concern with what it contains or what are the other
packets associated with it. Some firewalls operate at the transport layer and know more about a
packet, which then results in grant or deny access depending on specified criteria.
2.1.1 Packet Inspection
A packet has an IP header followed by a TCP, UDP or ICMP headers [5]. TCP and UDP headers
are followed by application messages. Packet inspection focuses on the contents of IP, TCP and
UDP headers. An IPv4 TCP and UDP headers are shown below5.
4
5
http://www.vicomsoft.com/learning-center/firewalls
http://www.cipherdyne.com/LinuxFirewalls/ch03
Figure 2.1.2: An IPv4 TCP header
Figure 2.1.3: An IPv4 UDP header
When a packet enters a firewall it starts matching the packets information against its rules. The
packet filtering rules are based on:
Specific NIC
Host IP address
Network layers source and destination IP addresses
Transport layers TCP or UDP service ports
TCP connection flags
The network layers ICMP message types
Whether the packet is incoming or outgoing
If the packet matches the criteria of the first rule, then the firewall performs the action described
by the target. If the packet does not match the criteria, then the engine goes to the next rule in the
chain and so on [6]. If firewalls do not process the rules fast enough, the whole system will slow
down. The packet filter does not examine the data section of a packet. The order in which the
rules are defined is important as a firewall process the rules from top to bottom. The list of rules
defining what can come in and what can go out are called chains [7]. Each chain has a default
6
policy. If the packet doesnt match any of the rules, then the default policy is applied. There are
two basic approaches to a firewall.
1) Deny everything by default and explicitly allow selected packets through.
2) Accept everything by default and explicitly deny selected packets through.
Moreover, a firewall mechanism gives the option of either rejecting or denying a packet. If the
reject option is selected then, it returns an error to the sender but if deny option is chosen then, the
packet is immediately discarded without notifying the source.
2.2 Types of Firewalls

The software firewalls are divided into four main categories. These are listed below:
Packet filtering firewall
Circuit level gateways
Application level gateways
Stateful multilayer inspection firewall
2.2.1 Packet Filtering Firewall

A packet filtering firewall applies a set of rules and examines each packet to determine whether
to forward the packet or drop toward a particular destination [8]. The firewall is typically
configured to filter the packets going in both directions, inbound and outbound. Packet filters
permit or deny network traffic based on the following information:
Source IP address and Destination IP address.
Protocols, such as TCP, UDP.
Source ports and Destination ports.
Direction (inbound or outbound).
A physical interface in which the packet is traversing.
2.2.2 Circuit Level Gateways

Circuit level gateways work at the session layer of the OSI model, or the TCP layer of the TCP /
IP model [9]. This firewall monitors the TCP handshake between the packets to determine
whether a requested session is allowed. It gives the advantage of hiding information about the
private network it protect and do not filter individual packets.
2.2.3 Application Level Gateways

Application level gateways, also called proxies, are similar to the circuit level gateways except
that they are application specific [10]. In other words, incoming or outgoing packets cannot
access services for which there is no proxy. For example, if an application gateway is configured
to be a web proxy, it will not allow to use FTP, Telnet or any other traffic. This firewall is used to
log the user activity and logins. It offers more security, but have a significant impact on the
network performance. This is because of context switches, which slows down the network access
dramatically.
2.2.4 Stateful Multilayer Inspection Firewall
Stateful multilayer inspection firewall combines the aspects of all three types of firewalls
mentioned above. It filters packets at the network layer to determine whether the session packets
are allowed and evaluate the contents of packets at the application layer [11]. Stateful multilayer
firewall allows direct connection between client and host and offer more security, performance
and transparency to end users.
2.3 Linux Iptables

The Linux based firewall is controlled by the program called Iptables which handles packet
filtering. It is an administration program which is implemented within the operating system. It
works at the transport layer and protects the system by making routing decisions after filtering the
packets based on information in the IP packet header [12]. Iptables requires administrative
privileges to operate and must be executed by root otherwise, it will not function. Iptables is used
to set up, maintain and inspect packet filters in the Linux kernel. Each table contains a number of
user-defined chains. Each chain is a list of rules which are applied to the incoming packets. Each
rule specifies what to do with a packet that matches with rules. The rules can be set to accept,
reject or drop the packets from an external network. The rules are stored in kernel tables, as an
input, output or forward chain. Moreover, Linux packet filters are fast and easy to maintain
within an operating system.
2.4 Firewalls Selection

When considering a firewall there are many areas of concern such as installation, configuration,
ease of use, documentation, support and features that should factor in selection criteria. A number
8
of Linux firewalls were studied including Iptables, IPCop, Shorewall, Monowall and Firehol.
Firewall selection seems to be a better choice from the performance perspective [13]. After going
through the literature, the following Linux firewalls are found suitable for performance
comparison and evaluation due to their high support, capabilities and documentation.
Iptables
Shorewall
Software based firewalls have their own specific features. The mentioned firewalls fulfill the
performance metrics defined for evaluation. Moreover, they use a similar filtering mechanism to
secure a network. Both firewalls were exposed to identical tests to find out the difference between
each firewall.
2.5 Limitations of Firewalls

Firewalls have some limitations; some of them are described below:
Attacks against open ports, such as buffer overflow attacks against unblocked services.
Attacks on the firewall itself (e.g. trying to penetrate the firewall code by exploiting a
buffer overflow in the firewalls packet parsing code)6.
Firewalls cannot protect against attacks that pay pass the firewall. Internal systems may
have dial-out capability to connect to an ISP7.
Attacks from compromised machines that have a VPN or other tunnel through the firewall
applies to perimeter firewalls.
A Firewall cant find other vulnerabilities which might allow hackers to access the
internal network.
A firewall may not protect fully against internal threats, such as disgruntled employees
who know all the network architecture [14].
Firewalls dont guard against malware, viruses and worms from the Internet because the
firewalls do not scan or examine a packets payload [15].
Denial of service attacks against the network link or the firewall itself.
A firewall cant let the user know if it has been incorrectly configured. Moreover it cannot
encrypt confidential documents and messages sent within an organization.
6
7
http://www.linuxjournal.com/article/6701
http://www.linuxsecurity.com/resource_files/firewalls/nsc/500619.html
2.6 Related Work

To build the security on a network, all the aspects of the system need to be considered. It has
never been an easy task to secure a network that is connected to the Internet. As described earlier
that firewalls are the first line of defense mechanism against the attackers. Several researches
have been done on firewalls to provide security in a more efficient way. While dealing with
firewalls, one should also keep in mind the limitations of them, the kind of security that they
cannot provide. After going through the literature review about firewalls, it is found that a lot of
work is done on the firewalls security issues rather than performance. It is obvious that a firewall
provides security but, the firewalls performance is also important and cannot be neglected. A
well-organized network requires high performance from all the network devices including
firewalls. The network performance encloses the description of speed, capacity, system resources
and transaction rate carried across the network. Therefore, the firewall has to show the
performance at an acceptable level in the network.
The firewalls performance testing has been done by many people in a virtual cloud environment
using open source solutions. A private cloud was designed and implemented with open-nebula as
a cloud toolkit and Xen as a hypervisor [16]. Linux Iptables was chosen to secure the cloud, and
implemented in a strategic point of the virtual network infrastructure to work as a bridge-mode
firewall. The firewall was evaluated using metrics throughput, latency, goodput and DoS
handling. By implementing evaluation scenarios in the cloud infrastructure using different tools,
the virtual firewall is evaluated. Overall, the results showed that the firewall is sensitive to a
number of rules. The performance of virtual firewall decreased in different situations, traffic sent
with large packet sizes, large number of rules in the filtering table and in high throughput levels.
This project is followed by the work done on virtual firewalls but it is concerned with the
comparison of firewalls in real-time evaluation. It reflects the working of small scale firewalls in
a business. In this project all the basic metrics will be covered such as throughput, latency,
connection establishment and teardown rate, HTTP transfer rate and system resource
consumption. The design of firewalls consists of repeatable experiments in order to compare both
firewalls. This project will test the firewalls with both TCP and UDP traffics to find out the
differences in each on the firewall, which is not done in the related work as described above.
Finally it will conclude which of the firewall is good to implement after analyzing the results.
10
3 Methodology
This chapter focuses on the evaluation methodology and designs as described in RFC 3511. In
addition to this, different testing tools are described to evaluate the firewalls.
3.1 Requirements
In order to perform the project, Linux operating systems should be installed on computers in
addition with the Internet connectivity. The whole project requires a minimum of two computers.
The reason for choosing the Linux operating system is that selected firewalls are the Linux
distributions and it becomes quite easy to install and setup the entire platform without considering
cost and copyright issues. Moreover, using an open source platform for evaluation will also make
any future work or improvement based on this exposition.
3.2 Experimental steps

In order to perform any scientific experiment, there is a common process which is as follows:
Making hypothesis
Performing the experiment and collecting data
Analyzing the data
Interpretation of data and draw conclusions
3.3 Traffic Classification

There are two different approaches to a network device to evaluate it. These consist of canonical
packet traces for offline tests and traffic generation tools for online evaluation [17].
3.3.1 Real-time (Online) Evaluation
It involves live traffic generation. A tool like Iperf, Netperf is used to generate real-time traffic
that is sent to the network device to test it.
3.3.2 Off-line Evaluation
It is the playing back sets of data rather than live traffic generation. These sets of data may be
captured using sniffer programs like Wireshark and then played back with tool such as TCPreplay. This project is related to real-time traffic generation.
11
3.4 Traffic Duration

In order to capture the effects of evaluation, the test duration, i.e. time period for which traffic is
sent to the firewall, is set to 30 seconds as described in RFC 3511. There are a lot of
contradictions about specifying the measurement time. To come to a conclusion, several tests
were conducted using different metrics by giving different time durations, which does not prove
much difference in the results. Moreover, the computers having both firewalls were located at the
next hope address from the computer where the traffic is being sent. Further study about the time
duration in the literature review shows that duration of test matters when the experimental setup
is located miles away from each other. For example, if a web server is located in New Zealand
and the current location takes place in Germany then the time duration counts.
3.5 Evaluation Procedure

Many firewalls implement network address translation (NAT) which translates private internet
addresses to public internet addresses [18]. This involves additional processing and may impact
the performance. RFC 3511 recommends indicating that whether NAT was enabled or disabled
during the experiments. In this project, NAT was disabled as the experiments were done with
public IP addresses in a network laboratory.
The second thing to mention is the rule sets as recommended by RFC 3511. The tests are planned
by an equal number of filtering rules for each firewall. The rules are written by blocking a
network. In addition to this, testing was performed by increasing the rules to determine their
impact on the performance. The increment in rules will also stress the firewalls.
The third thing to mention is the packet size. All of the tests are performed by varying the packet
sizes as recommended in benchmarking methodology for network interconnect devices RFC 2544
[19]. The recommendation about packet sizes is too easy to understand and remember. It also
reflects the behavior of a small-scale firewall under changing load conditions and hence will help
in making performance comparisons easily.
The last thing to specify in the procedure is the transmission of traffic. A Firewall is tested by
network benchmarking tools which have client/server specifications8. Traffic flows from client to
server. In order to flow the traffic both the client and server must have the same software
versions. RFC 3511 recommends indicating either unidirectional or bi-directional traffic was sent
8
http://rfc-ref.org/RFC-TEXTS/3511/chapter4.html
12
during firewall testing. The test may consist of both types. The firewall handles the traffic from
multiple networks in real-time. However, the procedure is different when testing the firewall. The
related work found in the firewall performance testing was also related to unidirectional traffic.
This project also consists of unidirectional traffic.
3.6 Evaluation Tools

In order to choose the right tools for experiments, a deep search was conducted for readily
available tools. After studying, the following tools found useful for evaluation. Moreover, all the
tools are full filling the needs to accomplish the goal of this project. Table 3.6.1 shows a list of
tools and their main use in the project which is further described in their sections. The version of
tools is an important thing to mention in features perspective.
Table 3.6.1: Evaluation tools and their usage in the project
List of Tools
Netperf (version: 2.6.0)
Usage
Throughput, Latency, Connection establishment rate,
Connection teardown rate
Iperf (version: 2.0.5)
Bandwidth, HTTP transfer rate
THTTPd (version: 2.25b)
HTTP Web server
HTTPing (version: 1.3.1)
HTTP transfer rate
Nmap (version: 5.00)
Monitoring
Webmin (version: 1.590)
HTTP GUI configuration
Wireshark (version: 1.2.7)
Monitoring
In addition to the mentioned testing tools, firewalls will also be tested by several system statistics
tools in Linux like top, iostat, ps, vmstat, lsof, mpstat, sar etc.
3.6.1 Netperf
Netperf is a network benchmarking tool which measures unidirectional or bi-directional bulk data
transfer and request/response performance of networks using TCP and UDP traffic [20]. It
provides test functionalities for throughput, end to end latency and works on a client/server
model, where the client is the data sender and server is the receiver. Netperf is compatible with
Linux and Windows systems. The main drawback of this tool is that it can only test against one
server at a time. The most basic test is the TCP_STREAM test which measures the throughput.
13
Request/Response is the other feature which measures the transaction rate and the time needed
for a transaction to be complete. Furthermore, the speed of a TCP connection establishment and
closure can be measured with Netperf. It is used in the project because it provides more accurate
results and wide range of test functionalities.
3.6.2 Iperf
Iperf is an open source tool for measuring the network performance based on a client/server
architecture [21]. It can be used for measuring bandwidth, jitter and lost datagrams. It can run on
multiple operating systems like Linux and windows. It creates TCP and UDP data streams which
exchange data between client and server. It is limited to handle one client/server at a time. This
tool is written in C++. It also allows the user to set various parameters that can be used according
to the needs of the test.
It can be associated with a graphical user interface called Jperf which provides graphs and logs
for the tests. This tool in used in calculating bandwidth and HTTP transfer rate to create real-time
traffic.
3.6.3 THTTPd
THTTPd is an open source web server from ACME Laboratories. It is a simple, small, fast, and
secure HTTP server. It suffices for most uses of the web. Its about as fast as the best fullfeatured servers and it has one extremely useful feature (URL-traffic-based throttling) that no
other server currently has9. It is designed for simplicity, small execution and speed.
For calculating the HTTP transfer rate, there must be an HTTP web server running on the firewall
which must reply upon receiving requests from the client side. This tool is used due to its
availability and processing speed.
3.6.4 HTTPing
HTTPing is a substitute of "ping" but for http-requests [22]. It is an open source Linux tool which
probes a given URL with GET requests and displays relevant statistics like the amount of time to
connect, send a request and receive a reply from the web server including the number of bytes
transferred etc. It is used in order to send the ping GET requests to HTTP server running on the
firewall.
http://acme.com/software/thttpd
14
3.6.5 Nmap
Nmap "Network Mapper" is an open source tool for network exploration and security auditing
[23]. It is used to discover hosts and services on a computer network, thus creating a map of the
network. To accomplish its goal, Nmap sends specially crafted packets to the target host and then
analyzes the responses. It can determine the operating system of the target, names and versions of
the listening services, type of device and presence of a firewall. In this project it is used to
discover that an HTTP server is running on the computer where the firewall is located.
3.6.6 Webmin
Webmin is an open source web-based interface for system administration of Linux [24]. It is used
to configure operating system internals, such as users, disk quotas, services or configuration files,
as well as modify and control open source applications. It uses TCP port 10000 for
communicating and can be configured to use SSL. Webmin can be used for controlling many
machines through a single interface. It is built around modules, which have an interface to the
configuration files and the Webmin server. This makes it easy to add new functionality. Due to
the Webmin modular design, it is used in the project to configure files of both firewalls.
3.6.7 Wireshark
Wireshark is the world's foremost network packet analyzer used for network troubleshooting,
analysis, testing and debugging [25]. It provides deep inspection of protocols, live and offline
capture analysis, read/write many different captured file formats. It allows setting a network
interface in promiscuous-mode, which results in seeing all visible traffic on that interface. Packets
can be browsed through a GUI. While packets are captured, each packets source, destination
ports and addresses can be found from it. Moreover traffic can be stored for further offline
analysis. Wireshark in used in this project for monitoring and analyzing traffic generated by Iperf
and Netperf.
3.7 Evaluation Metrics

A number of metrics are selected from the methodology for firewall performance RFC 3511 to
test the firewalls. The selected metrics were further studied to find out the reason of highlighting
these metrics. After the literature review it is found that these metrics play an important role in
the performance as being basic and critical.
15
Several research papers also indicated the throughput, latency and connection establishment and
teardown rate as core metrics in the performance. The reason for selecting these metrics in the
project is explained below. Table 3.7.1 shows a brief description and importance of these metrics.
Table 3.7.1: Performance Metrics and their importance [26]
Metrics
Description
Throughput
Raw capability of the firewall to

pass bits per second or packets per
second from interface to interface;
similar to router and switch metrics
Time traffic is delayed in the
firewall, added to the total end-toend delay of the network
Latency
Can be measured in milliseconds
at an offered load near the
firewalls limit
Connection Establishment Rate
Speed at which firewalls can set

up connections and the full threeway handshake to establish a
TCP/IP session
Dozens of connections to be set
up across an enterprise firewall
Connection Teardown Rate
Speed at which firewalls can tear

down connections and free
resources to be used for other
traffic
HTTP Transfer Rate
Speed at which an HTTP

requested object traverses a
firewall
Importance
If basic throughput is not in
place, the firewall obviously
cannot handle the traffic
Basic throughput is a good
starting point to be sure that the
product is close to what is needed
Latency affects perceived
response time
Most firewalls have low latency
when lightly loaded, but latency
must be measured and reported
when the firewall is at its operating
load to understand whether
undesirable delay will occur
Every click on a web-based
application may cause
If this burst of connections
overwhelms the firewall, user
perception of application speed
will be adversely affected
Since every connection that is set
up must also be torn down, the
firewall has to keep up in both
directions
What is the impact of a firewall
on HTTP traffic
HTTP is the type of traffic a
firewall encounters the most
It is hard to measure the performance of a firewall without the mentioned evaluation metrics. This
project will cover all of the mentioned evaluation metrics to test the performance of both firewalls
in a Linux system. The evaluation scenarios are performed online as the selected tools generate
real-time traffic instead of playing back sets of data.
16
3.7.1 Throughput
Throughput is the amount of data transferred successfully over a link from one location to
another in a given period of time. It is usually expressed in bits per second (bits/s or bps).
During this metric test, both of the throughputs will be measured TCP and UDP. In order to do in
depth evaluation there are four scenarios involve in this metric test to perform on each firewall. In
the first scenario, packet sizes are increased to:
and
bytes,
according to RFC 2544. However the maximum size of a single data unit MTU is 1500 bytes
(Ethernet v2). It is the maximum size of an IP packet that can be transmitted in the network
without fragmentation and the largest allowed packet size by Ethernet at the network layer and
hence over the Internet10.
The time duration is set to 30 seconds for measuring the throughput as explained in the section
traffic duration. There are no firewall filtering rules involve in this scenario on each firewall. In
other words there will be no rules on firewalls. All kinds of traffic will be allowed which involves
sending the specified amount of data packets as stated above.
The whole task is done by the testing tool Netperf. It is a very powerful tool which measures the
throughput of a network by sending TCP and UDP stream from client to server. There are two
computers involved in this scenario, one act as a client and one will act as a server, including
Netperf installed on the both computers. The client sends the traffic while server listens to it. The
computer having Netperf server must be the one where the firewall is going to be tested whereas,
Netperf client can be any computer who will send the traffic to the server. Throughput will be
calculated and displayed by the Netperf automatically.
The second, third and fourth scenarios involve increasing the firewall filtering rules to 100, 500
and 1000 respectively; to see the impact on throughput. The involvement of filtering rules and
varying packet sizes will stress the firewall by means of assessing its behavior under the changing
load conditions to show the worst case scenario. Since, firewalls cannot detect duplicate rules. It
is one of the limitations of firewalls found during the literature review and verified during the
experiments. The incoming packets are forced to match to all of the rules before being allowed by
the firewall. In actuality, there are 100 rules written for each firewall which are repeated over and
over to stress it. The tests time duration is same for the rest of the scenarios.
10
http://www.linfo.org/mtu.html
17
The UDP throughput test is also performed in a similar way to TCP. Results of all the scenarios
will be collected to draw any conclusions about firewalls TCP and UDP throughput. The result of
this metric test will tell the ability of a firewall to support transactions that include the transfer of
large volumes of data, as well as supporting a large number of simultaneous transactions. It will
also give the overall picture of network load and hence firewall throughput performance in the
network.
Throughput is related to bandwidth. In general, both are usually measured in bits per second. The
purpose of stating the difference is to clarify both phenomena because, many people misjudge
both terms.
Table 3.7.2: Difference between bandwidth and throughput
Bandwidth
Throughput
Bandwidth relates to how fast a device can send

data over a single cable.
Throughput is how much data is actually transferred

between two hosts.
3.7.2 Latency
Latency is a measure of time delay experienced in a system. The processing delay of a firewall
is the time when a packet enters the firewall and ends when this same packet leaves it. It is
measured in seconds.
There are several methods to calculate the latency. The most widely used method is by using
Netperf request/response. It is a synchronous, one transaction at a time test, where the transaction
is defined as the complete exchange of a request and a response. The results of this test return as
an average number of transactions which took place in a second. The TCP request/response
(TCP_RR) works as a standard TCP connection by establishing a connection, exchanging packets
and finally breaking the connection. On the other hand UDP request/response (UDP_RR)
assumes that packets reach the destination so, there is no mechanism to notify the source that the
packet has arrived. This metric test will also calculate both latencies TCP and UDP.
The way of calculating the latency is similar to throughput including involvement of two
computers, client server model and traffic generation from client to server. There are four
scenarios involved again in measuring the latency. The first scenario is conducted without any
18
filtering rules, while others involve changing the filtering rules to 100, 500 and 1000 respectively.
In this experiment different request/response sizes in bytes will be sent. There is an option to
control the size of a request and a response. Latency test is performed by changing the standard
values of request/response to
bytes. Netperf will display the transaction rate per second which is known as
latency. Time duration of the test is set to 30 seconds as mentioned in the traffic duration. The
UDP latency test will be calculated using the same way as TCP is calculated.
The outcome of this metric test will display the total delay between the end points, or latency as
well as the small-scale variation of this latency. Given that if one knew the latency, available
bandwidth and transaction rates as a profile of network performance between two network end
points then, it is possible to make a reasonable prediction relating to the performance of a firewall
in a network. The results of all the scenarios of each firewall will be compared in order to come
to any conclusion.
3.7.3 Connection Establishment rate
It is the length of time needed for data exchange between two hosts to agree to set up connection
using three-way handshake to establish a TCP/IP Session. The measurement unit is seconds.
The above definition is only valid for connection oriented protocols like TCP, for connectionless
protocols like UDP it is meaningless. The connection establishment rate will also be performed
with the help of Netperf. This time Netperf TCP connect/request/response (TCP_CRR) feature
will be used for calculating the connection establishment rate. The TCP_CRR is the combination
of both TCP_RR and TCP_CC. It measures the performance of establishing a connection,
exchanging a single request/response transaction, and tearing-down that connection.
Request/response values are important by means of the speed of transmission.
The test scenarios will be repeated in this experiment by changing the filtering rules to 0, 100,
500 and 1000 respectively. The test will be performed by changing the standard values of
request/response to 1/1 and 32/1024 which is a typical request/response size.
This test will be conducted by changing the time duration to 60 seconds, as there is no time
duration recommended in RFC 3511. The conclusion made in metric test time is achieved by
doing different timing tests. There are two computers involved in performing this test, one act as
19
a client and one will act as a server. The computer having Netperf server must be the one where
the firewalls will conduct the TCP_CRR test. The results of this test will show the rate at which
each firewall setup a TCP connection by using three-way handshake. The outcome of both
firewalls will be judged in order to come to any conclusion.
3.7.4 Connection Teardown Rate
Connection teardown rate is the speed at which firewalls can tear down the connection and free
the resources to be used for other traffic. It is measured in seconds.
It applies to only connection-oriented protocol like TCP as, UDP is an unreliable and
connectionless protocol. The connection teardown rate is tested by Netperf TCP connect/close
(TCP_CC) feature. Connection teardown rate measures how fast the pair of systems can open and
close connections between one another in a synchronous (one at a time) manner.
Again, test scenarios will be repeated in the same way connection establishment rate is calculated
by changing the filtering rules from 0 to 1000. Standard values of request/response will also be
same in this situation to 1/1 and 32/1024 which represents a typical request/response size.
All the procedure will be same in calculating connection teardown rate as establishment rate is
calculated including time duration and setup. The results of this test will show the rate at which
each firewall teardown the connection and free the resources. The outcome of both firewalls will
be judged in order to come to any conclusion.
3.7.5 HTTP Transfer Rate
HTTP transfer rate is the speed at which HTTP requested objects traverse a firewall. It is
measured in seconds.
It probes a given URL to GET requests and displays relevant statistics like the time take to
complete the request and transferred speed etc. This task involves the use of tool HTTPing which
is like "ping" but for http-requests. HTTP requests will be sent at a constant rate through the
firewall. It is the type of traffic a firewall encounters the most and hence considered in the
project.
In order to calculate the HTTP transfer rate there must an HTTP web server running on the
firewall which should reply upon receiving the requests from the client. For this purpose THTTPd
is used. It will run on the computer having a firewall on port 80 because, HTTP uses port 80. To
20
make sure that the HTTP web server is configured correctly and running on the firewall, Nmap
will be used, which will display the number of open ports.
HTTPing will be installed on one computer and it will send the GET requests to an HTTP web
server running with firewall. At the same time Iperf will generate the traffic with constant packet
size of 512 bytes to stress the firewall. The HTTP transfer rate will be measured while increasing
the number of filtering rules to 0, 500 and 1000 on the firewall to see the impact on the HTTP
transfer rate. The time duration is kept to 60 seconds, as there is no recommendation in RFC 3511
for this type of test. Different options of HTTPing will be used which will display different
statistics of HTTP transfer rate like round-trip time and transfer speed. The results of each
firewall will be criticized to see which firewall takes less time to give response to HTTP requests.
3.7.6 System Resource Consumption
Since the aim of this project is to evaluate the performance of firewalls. All the core evaluation
metrics are taken into account. Far beyond evaluation, it would be interesting to see the individual
and collective CPU statistics, overall I/O activities of the system and memory usage. For this
purpose Linux (sar) utility is used. This utility can do two things.
1. Monitor the real time CPU and memory usage over a specified time.
2. Collect performance data in the background on an on-going basis and do analysis on the
historical data to identify bottlenecks.
The specifications of computer including processor type, RAM capacity and memory also play a
vital role in system resource consumption. Moreover the version of system software like Linux
kernel should also be kept in mind while doing the process.
System resource consumption will be conducted on both firewalls by changing the filtering rules
to 0, 500 and 1000 with a constant packet size of 1024 bytes from Iperf. The time duration is set
to 30 seconds for each firewall. Results of both firewalls will be recorded and compared with
each other to conclude the fact that which firewall consumes more system resources.
21
4 Experiment
All the tools and evaluation methods have been discussed in detail to measure the performance of
firewalls for making comparisons. This chapter will cover the implementation of all the designed
scenarios including configurations to get the results and to come to a conclusion.
In order to do the configuration of firewalls, setup guides and tutorials were consulted. The
firewalls are configured according to the documentation available for them. Manuals of all the
tools were studied to consider the option suitable for performance evaluation.
4.1 Specifications of computers

A Linux system can display all the system hardware specifications and software versions by the
(lshw) command. The computers used during the experiments have the similar specifications
which are described in table 4.1.1.
Table 4.1.1: Computer Specifications
Model
CPU
RAM
HDD
NIC
Operating System
Iptables version
Computer Specifications
Dell Optiplex 745
Intel Core 2 Duo 6600 @ 2.40GHz
2.00 GB
80GB 7200RPM
Gigabit Ethernet LAN 10/100/1000
KUbuntu 10.04.4 LTS
1.4.4
4.2 Firewalls Installation

The installation of firewalls is an easy step. As both firewalls are an open source software
therefore, can be downloaded at any time from the Internet.
4.2.1 Iptables
If a Linux system is installed on a computer, it must have an Iptables firewall. Although it is not
configured by default and all types of traffic at anywhere is allowed. The user has to write the
configuration of the firewall on its own. If, firewall is not present then it can be installed
manually. Moreover on Ubuntu it can be installed by the command listed below. To ensure that
22
firewall is present on the system, the following commands are used; where -L displays the rules
of the firewall and -V shows the current version.
sudo apt-get install iptables
iptables L
iptables V
Figure 4.2.1: Iptables Installation
4.2.2 Shorewall
Shorewall is not installed on Linux systems by default. The user has to download and install it
manually from the shorewall.net website. On Ubuntu it can be installed by the command listed
below and can be verified by the status and version commands.
sudo apt-get install shorewall
shorewall status
shorewall version
Figure 4.2.2: Shorewall Installation
4.3 Configuration of Firewalls

The key to Internet Security begins with a firewall, but when configured wrong, even the best
firewall can leave the system exposed and vulnerable [27]. A well configured firewall permits,
deny or reject the network traffic in and out on the basis of rules. The configuration of both
firewalls will be the same however; the method to configure each firewall differs from each other.
4.3.1 Iptables
The configuration of Iptables firewall is rather easy and simple as compared to Shorewall. The
configuration file name is iptables.rules which is located in root/etc/iptables.rules. For
convenience Webmin is used, which is a web-based system configuration tool for Linux. It
allows the user to add, remove or modify a script. The rules are saved in the configuration file
mentioned above. Below is the script for allowing all types of traffic.
#!/bin/sh
iptables -F INPUT
iptables -F OUTPUT
iptables -F FORWARD
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
Figure 4.3.1: Iptables Rules Configuration
In the above script command -F flushes the table of all contents and -P sets the policy of input,
output and forward chains to accept. An Iptables firewall script can be found in the Appendix B.
23
4.3.2 Shorewall
The configuration of shorewall is not simple as Iptables because, Shorewall has different
configuration settings including zones, interface, policies and rules11. All the configuration files
are located in the directory root/etc/shorewall. Firewall zone will be set at first. The zone is what
the firewall will refer to when determining where requests are coming from and what to do with
them depending on where they are going. Zones can be set by modifying the file zones which is
located in root/etc/shorewall/zones. Figure 4.3.2 shows the Shorewall zone settings.
#ZONE
TYPE
OPTIONS
IN OPTIONS
OUT OPTIONS
#
fw
firewall
net
ipv4
loc
ipv4
#LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE
Figure 4.3.2: Shorewall Zones
Next step is to add the computers physical interface with those zones so, public interface will
reflect the net zone and lo interface corresponds to the loc zone, the system itself. This is done via
root/etc/shorewall/interfaces. Linux command (ifconfig) was used to make sure that eth1 is the
computers physical interface. The interface is set according to the computer system as shown in
figure 4.3.3.
#ZONE
INTERFACE
BROADCAST
OPTIONS
neteth1
detect
loc
lodetect
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE
Figure 4.3.3: Shorewall Interfaces
The Shorewall policy file will define what the default rules of the firewall are, i.e. what the
firewall will do with a request if there's no specific rule covering that request. In order to set the
policies, the policy file should be adjusted which is located in root/etc/shorewall/policy.
According to the test scenarios policy file is set as shown below:
#SOURCE
DEST
POLICY
LOG LEVEL
LIMIT:BURST CONNLIMIT:MASK
#
Fw
net
ACCEPT
Fw
loc
ACCEPT
Net
fw
ACCEPT
Loc
fw
ACCEPT
net
all
DROP
info
# THE FOLLOWING POLICY MUST BE LAST
all
all
REJECT
info
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE
Figure 4.3.4: Shorewall Policies
11
http://articles.slicehost.com/2009/7/22/rhel-shorewall-configuration
24
The last step in the configuration of Shorewall is to set up the rules. The rules tell the firewall
what to do with a specific request based on source and destination address. The rule file is also
located in /etc/shorewall/rules. Below is an example of rules which allow all the traffic.
#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE
#
PORT
PORT(S)
DEST
USER/ MARK
LIMIT
GROUP
ACCEPT net
fw
all
ACCEPT fw
net all
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE
Figure 4.3.5: Shorewall Rules Configuration
As described earlier about the web-based system configuration software Webmin, it is much
easier to set up all the configuration files with it. After configuration of all the files, firewall is
ready to use. The above mentioned rules are allowing everything through which will be used as
zero firewall rules. The Shorewall rules configuration script is located in Appendix C.
4.4 Implementation of Evaluation Metrics

After configuration of both firewalls it is time to test their performance by evaluation design and
methods that are discussed in detail in the previous chapter.
4.4.1 Throughput
Throughput test was conducted by Netperf. It sends a TCP or UDP stream to the specified IP
address where the firewall is installed. Figure 4.4.1 shows the commands used to launch the
Netperf on server and client hosts. To calculate the UDP throughput, the whole method is similar
to TCP except mentioning the UDP_STREAM. There must be a Netserver running on the
firewall in order to receive the traffic from the client.
Server (IP: 194.47.155.156)
root@labcomputer:# netserver
Starting netserver with host 'IN(6)ADDR_ANY' port '12865' and family AF_UNSPEC
Client
root@labcomputer:# netperf -l 30 -t TCP_STREAM -H 194.47.155.156 -- -s 16384 -S 16K -m
64
MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to 194.47.155.156 () port 0
AF_INET
Recv
Send
Send
Socket Socket Message Elapsed
Size
Size
Size
Time
Throughput
bytes bytes
bytes
secs.
10^6bits/sec
32768
32768
64
30.00
605.06
Figure 4.4.1: TCP Throughput Calculation
In Netperf client -l option is used to specify the test time duration which is kept to 30 seconds; -t
states that it will be TCP traffic, -H gives an option to specify the server IP address, -s waits
25
seconds between test setup and test start, which is displayed in sending socket size bytes; -S set
SO_KEEPALIVE on the data connection, which can be seen in received socket size bytes; -m is
used to change the packet size in bytes. In the above test packet size is set to 64 bytes.
Server (IP: 194.47.155.157)
Client
root@labcomputer:# netperf -l 30 -t UDP_STREAM -H 194.47.155.157 -- -s 16384 -S 16K -m
512
MIGRATED UDP STREAM TEST from 0.0.0.0 () port 0 AF_INET to 194.47.155.157 () port 0
AF_INET
Socket Message Elapsed
Messages
Size
Size
Time
Okay Errors
Throughput
Bytes
bytes
secs
#
#
10^6bits/sec
32768
32768
512
30.00
30.00
3955459
2287221
540.05
312.28
Figure 4.4.2: UDP Throughput Calculation
In figure 4.4.2 the upper statistics are from the sending (Netperf) side. The lower figures are from
the receiving (Netserver) side. In this figure UDP throughput packet size is set to 512 bytes.
Graphical representation of the results is discussed in the next chapter.
4.4.2 Latency
Latency test was also performed with the help of Netperf. The request/response feature of Netperf
is already explained in detail for calculating the latency. The commands in testing the latency of
both firewalls are explained below. The UDP latency calculation is same as TCP except, to state
that it is a UDP_RR so that Netperf will generate UDP traffic.
Server (IP: 194.47.155.156)
Client
root@labcomputer:# netperf -H 194.47.155.156 -l 30 -t TCP_RR -- -r64,1024
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 () port 0 AF_INET to 194.47.155.156 ()
port 0 AF_INET : first burst 0
Local /Remote
Socket Size
Request Resp.
Elapsed Trans.
Send
Recv
Size
Size
Time
Rate
bytes Bytes bytes
bytes
secs.
per sec
16384
16384
87380
87380
64
1024
30.00
6650.46
Figure 4.4.3: TCP Latency Calculation
Netperf -H command is used to specify the IP address of the server; -l gives information about the
test time duration; -t TCP_RR shows that Netperf is using a TCP request/response feature; -r
command is used to change the request/response size.
26
Server (IP: 194.47.155.156)

Client
root@labcomputer:# netperf -T 1 -H 194.47.155.156 -l 30 -t UDP_RR -- -r32,1024
MIGRATED UDP REQUEST/RESPONSE TEST from 0.0.0.0 () port 0 AF_INET to 194.47.155.156 ()
port 0 AF_INET : first burst 0 : cpu bind
Local /Remote
Socket Size
Request Resp.
Elapsed Trans.
Send
Recv
Size
Size
Time
Rate
bytes Bytes bytes
bytes
secs.
per sec
112640 112640 32
112640 112640
1024
30.00
7284.23
Figure 4.4.4: UDP Latency Calculation
In UDP latency calculation -T option is used to make sure that Netperf and Netserver ran on a
given CPU and did not move around during the test. In the above figure request/response size is
kept to 32/1024 byes. The results of both firewalls TCP and UDP latency is discussed in the next
chapter in a graphical format with latency transaction rate per second evaluated along the y-axis
and number of filtering rules on the x-axis.
4.4.3 Connection Establishment Rate
The method to calculate the connection establishment rate via Netperf TCP_CRR feature has
already explained. The command shown below states that server must be running on one of the
firewall from Netperf netserver command. On the client side -H is used to specify the IP address
of the server; -l shows the duration of the test which is kept to 60 seconds in this case; -t states
that it is a TCP connect/request/response feature; -r is used to change the size of request and
response which is kept to 32,1024.
Server (IP: 194.47.155.157)
Client
root@labcomputer:# netperf -H 194.47.155.157 -l 60 -t TCP_CRR -- -r32,1024
MIGRATED TCP Connect/Request/Response TEST from 0.0.0.0 () port 0
194.47.155.157 () port 0 AF_INET
Local /Remote
Socket Size
Request Resp.
Elapsed Trans.
Send
Recv
Size
Size
Time
Rate
bytes Bytes bytes
bytes
secs.
per sec
16384
16384
87380
87380
32
1024
60.00
AF_INET
to
3180.55
Figure 4.4.5: Connection Establishment rate Calculation
27
4.4.4 Connection Teardown Rate

For calculating connection teardown rate TCP_CC feature of Netperf is used. By applying the
same commands as the connection establishment rate was calculated, connection teardown rate is
calculated except mentioning to Netperf that it is a TCP connect/close TCP_CC test. The
implementation is described below in figure 4.4.6
Server (IP: 194.47.155.156)
Client
root@labcomputer:# netperf -H 194.47.155.156 -l 60 -t TCP_CC -- -r32,1024
TCP Connect/Close TEST from 0.0.0.0 () port 0 AF_INET to 194.47.155.156 () port 0
AF_INET
Local /Remote
Socket Size
Request Resp.
Elapsed Trans.
Send
Recv
Size
Size
Time
Rate
bytes Bytes bytes
bytes
secs.
per sec
16384
16384
87380
87380
32
1024
60.00
4273.89
Figure 4.4.6: Connection Teardown rate Calculation
4.4.5 HTTP Transfer Rate

Tools should be changed while calculating the HTTP transfer rate as described in the in the
design and methods. Iperf, Nmap and HTTPing are used to find the HTTP transfer rate. Iperf -s
command is used to run it as a server where the firewall is configured. Option -c specifies that it
is client side which is going to be connected to the server including the IP address of the server; -l
change the size of the packet, this command is transferring 512 bytes of packet to the server; -t is
used to mention the time duration of the test.
Server (IP: 194.47.155.157)
root@labcomputer:# iperf -s
-----------------------------------------------------------Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
-----------------------------------------------------------Client
root@labcomputer:# iperf -c 194.47.155.157 -l 512 -t 65
-----------------------------------------------------------Client connecting to 194.47.155.157, TCP port 5001
TCP window size: 16.0 KByte (default)
-----------------------------------------------------------[ 3] local 194.47.155.160 port 49724 connected with 194.47.155.157 port 5001
[ ID] Interval
Transfer
Bandwidth
[ 3] 0.0-65.0 sec
2.14 GBytes
282 Mbits/sec
Figure 4.4.7: Iperf Traffic Generation
To make sure that HTTP server running on the firewall, it is verified by Nmap. The commands
are shown in the figure 4.4.8.
28
Client (Nmap)
root@labcomputer:# nmap 194.47.155.157
Starting Nmap 5.00 ( http://nmap.org ) at 2012-09-27 09:04 CEST
Interesting ports on compc-2412-08.comlab.bth.se (194.47.155.157):
Not shown: 997 closed ports
PORT
STATE SERVICE
22/tcpopen ssh
80/tcpopen http
10000/tcpopen snet-sensor-mgmt
MAC Address: 00:19:B9:20:C6:C2 (Dell)
Nmap done: 1 IP address (1 host up) scanned in 0.25 seconds
Figure 4.4.8: Nmap View
After the verification of HTTP web server on port 80, HTTPing is used to send the ping to the
firewall; -G option is used to send the GET request; -b will show the transfer speed in KB/s; -g is
used to specify the URL, it can be an IP address; -c counts the number of requests, in other words
it allows how many times to connect.
Client (HTTPing)
root@labcomputer:# httping -G -b -g http://194.47.155.157 -c 60
PING 194.47.155.157:80 (http://194.47.155.157):
connected to 194.47.155.157:80, seq=0 time=1.29 ms 206554KB/s
Continued...
--- http://194.47.155.157 ping statistics --60 connects, 60 ok, 0.00% failed
round-trip min/avg/max = 1.1/1.5/4.1 ms
Transfer speed: min/avg/max = 2219/224093/321235 KB
Figure 4.4.9: HTTPing status
The outcome of this metric test is discussed in the next chapter in a graph where the y-axis will
represent the minimum transfer rate and the number of filtering rules will be along the x-axis.
Experimental results of all the metrics can be found numerically in Appendix A. The
experimental results are reported according to the format recommended by benchmarking
methodology for firewall performance in RFC 3511.
4.4.6 System Resource Consumption
In order to see which firewall consume more system resources, Linux (sar) utility is used to
display the average CPU and memory utilization during the traffic. Both firewalls are stressed by
a large number of rules to see their behavior and making a comparison between them.
The computer specifications are same where both firewalls are installed, including the computer
brands. The Figure 4.4.10 displays the method to calculate the CPU and memory utilization in a
29
given amount of time. The command shows the usage at every single second and displays the
average usage after the mentioned time ends.
In the figure option -u displays all CPU utilization statistics during the time which is kept to 30
seconds; -r shows memory utilization statistics. The readings of both firewalls are saved on both
computers in a text file. The results are criticized later on to see which of the firewall consumes
more resources.
Linux [sar] command
root@labcomputer:# sar u 1 30 >> /home/student/Documents/sar_cpu_shorewall_1000
root@labcomputer:# sar r 1 30 >> /home/student/Documents/sar_mem_shorewall_1000
Figure 4.4.10: CPU and Memory usage Calculation
30
5 Results
This chapter provides the comparative results of each firewall. All the metrics result is explained
in their sections.
5.1 Throughput
In any network, the maximum throughput is equal to the data rate supported by NIC, which
defines the theoretical baseline as 1000 Mbps. But in practice, these theoretical numbers are not
achievable because of various factors in the network conditions. The results of both firewalls
show that smaller packet sizes have lower throughput than larger ones. Moreover increasing
filtering rules does not affect the smaller packet size at all. Figure 5.1.1 shows the TCP
throughput of Iptables firewall.
800
Throughput (Mbits/s)
700
600
500
0 Rules
400
100 Rules
300
500 Rules
200
1000 Rules
100
0
32
64
128
256
512
1024
1500
Packet Size (bytes)

Figure 5.1.1: Iptables TCP Throughput
UDP is a protocol that defines how to form messages that are sent over an IP. A device that sends
UDP packets assumes that they reach the destination. There is no mechanism to tell the sender
that the packet has arrived. It is usually used for streaming media applications where an
occasional packet loss does not matter.
31
The results of Iptables firewall show that UDP traffic is not affected by the number of rules on
the firewall. Smaller packet sizes have lower throughput levels and larger gives higher
throughput. However, at 1 kilobytes of packet size it gave maximum throughput which can be
seen from the figure below. At 1.5 kilobytes of packet this level has decreased. A
UDP_STREAM test has no end-to-end flow control. The biggest of these implications is the data
which is sent might not be received by the receiver. On some platforms it may be possible for the
sending throughput to be reported as a value greater than the maximum rate of the link. It is
common when the CPU is faster than network and there is no flow control.
800
700
600
500
0 Rules
400
100 Rules
300
500 Rules
200
1000 Rules
100
0
32
64
128
256
512
1024
1500
Packet Size (bytes)

Figure 5.1.2: Iptables UDP Throughput
The increasing number of filtering rules on firewalls is important to determine their impact on
performance. The firewall is forced to check each packet against its rules; this also reflects the
working of firewall during peak hours when there is a lot of load on firewalls to handle. It can be
seen clearly from the results of TCP and UDP throughput that TCP transfers more data as
compared to UDP. In general, UDP is faster than TCP, and the reason is because its non-existent
acknowledge packet (ACK) that permits a continuous packet stream. TCP acknowledges a set of
packets, calculated by using the TCP window size and round-trip time (RTT).
In this experiment, TCP and UDP window sizes were the same which can be clearly seen from
the implementation. Moreover, the TCP and UDP window size was also same during the transfer
32
of different packet sizes. The reason behind more TCP throughput is that TCP buffers the data
and fills a network segment thus, making more efficient use of the available bandwidth. UDP on
the other hand puts the packet on the network immediately thus congesting the network with lots
of small packets.
800
700
600
500
0 Rules
400
100 Rules
500 Rules
300
1000 Rules
200
100
0
32
64
128
256
512
1024
1500
Packet Size (bytes)
Figure 5.1.3: Shorewall TCP Throughput
Figure 5.1.3 shows the Shorewall TCP throughput. The shorewall TCP throughput performance
does not degrade in contrast with Iptables even though it was a similar test for both firewalls. The
big difference in both firewalls throughput results is surprising. At 32 bytes of packet size both
firewalls give almost equal results. But, the situation changes when the packet size was 64 bytes
and there 1000 filtering rules.
After comparing both firewall results, it can be seen that Iptables degrade their performance when
the number of filtering rules were increased, whereas Shorewall performance does not affect so
much by number of filtering rules. The Shorewall UDP throughput result is similar to Iptables
UDP throughput which clearly indicates that UDP does not affected by the number of rules or
firewall type.
33
800
700
600
500
0 Rules
400
100 Rules
300
500 Rules
200
1000 Rules
100
0
32
64
128
256
512
1024
1500
Packet Size (bytes)

Figure 5.1.4: Shorewall UDP Throughput
5.2 Latency
Latency of firewalls was measured from the Netperf request/response feature. During the test
request response values were changed while incrementing the filtering rules. The bigger the
request/response size, the lower the transaction rate becomes in firewalls.
12000
10000
Trans. Rate per second
Req=1, Res=1
Req=16, Res=32
8000
Req=32, Res=64
Req=64, Res=128
6000
Req=64, Res=256
4000
Req=64, Res=512
Req=32, Res=1024
2000
Req=64, Res=1024
Req=512, Res=1024
0
0
100
500
1000
Number of filtering rules

Figure 5.2.1: Iptables TCP Latency (Request/Response)
34
The results show that if the number of filtering rules are incremented on firewalls; there is a slight
decrease on the transaction rate, which proves that the performance degraded as the rules are
increased.
Since the sender of UDP packets does not require any knowledge that the destination has received
the packets, UDP is relatively immune to latency. The only effect that latency has on a UDP
stream is an increased delay of the entire stream. The filtering rules on firewall also affected the
performance in UDP latency whereas, in UDP throughput filtering rules does not affect the
performance of firewall at all. It is important to note that latency and throughput are completely
independent with UDP traffic. In other words, if latency goes up or down, UDP throughput
remains the same. This concept has more meaning on the effects of latency on TCP traffic.
12000
10000
Req=1, Res=1
Req=16, Res=32
8000
Req=32, Res=64
Req=64, Res=128
6000
Req=64, Res=256
4000
Req=64, Res=512
Req=32, Res=1024
2000
Req=64, Res=1024
Req=512, Res=1024
0
0
100
500
1000
Figure 5.2.2: Iptables UDP Latency (Request/Response)
Shorewall TCP latency shows that it does not delay the traffic in contrast with Iptables which can
be concluded after looking at both results. The transaction rate of Shorewall is more than Iptables
which indicate that it is more efficient than Iptables. Figure 5.2.3 shows the Shorewall TCP
latency. The smaller request/response sizes are not affected by the number of filtering rules on the
Shorewall which can be seen from the figure. Whereas, this was not the case in Iptables. The
conclusion is made after analyzing the outcome.
35
12000
10000
Req=1, Res=1
8000
Req=16, Res=32
Req=32, Res=64
Req=64, Res=128
6000
Req=64, Res=256
Req=64, Res=512
4000
Req=32, Res=1024
Req=64, Res=1024
2000
Req=512, Res=1024
0
0
100
500
1000

Figure 5.2.3: Shorewall TCP Latency (Request/Response)
There are no effects of latency on the sending device with UDP traffic. Shorewall performed
better in UDP traffic as well as in TCP. The conclusion is drawn after analyzing both results.
Shorewall handles the traffic well when there were a large number of filtering rules.
12000
10000
Req=1, Res=1
Req=16, Res=32
8000
Req=32, Res=64
Req=64, Res=128
6000
Req=64, Res=256
4000
Req=64, Res=512
Req=32, Res=1024
2000
Req=64, Res=1024
Req=512, Res=1024
0
0
100
500
1000

Figure 5.2.4: Shorewall UDP Latency (Request/Response)
36
5.3 Connection Establishment and Teardown Rate

TCP connection establishment and teardown rate are calculated by the Netperf.
4000
3500
3000
2500
2000
1500
1000
500
0
0
100
500
4400
4300
4200
4100
4000
3900
3800
3700
3600
3500
3400
1000

Req=1, Res= 1
100
500
1000
Req=32, Res= 1024
Req=1, Res= 1
Req=32, Res= 1024
Figure 5.3.1: Iptables (Connect/Request/Response) and (Connect/Close)
Request/response values are important by means of the speed of transmission, where the bigger
values produced lower transactions per second and the smaller values keep the transmission time
short. In this test, a typical request/response size of
and
is used to see the effect on
connection opening and closing.

3500
4300
3400
3300
3200
3100
3000
2900
2800
2700
4250
4200
4150
4100
4050
4000
100
500
1000

Req=1, Res= 1
Req=32, Res= 1024
100
500
1000

Req=1, Res= 1
Req=32, Res= 1024
Figure 5.3.2: Shorewall (Connect/Request/Response) and (Connect/Close)
37
The results show that when there is an increase in the number of filtering rules on both firewalls
transaction rate decrease, which conclude that filtering rules affect the firewall's performance as it
have to check every packet coming from outside against rules that are stored in its configuration
to make any decision. In addition to this, there is a remarkable change in the performance of both
firewalls when there are 1000 filtering rules in connection teardown rate. Transaction rate
decreases to almost half as compared to the time when there were no filtering rules which can be
seen clearly from the graphs of both firewalls.
The results highlight that Shorewall performs better than Iptables to establish and teardown a
TCP connection.
5.4 HTTP Transfer Rate

HTTPing is used to send 60 ping GET requests to both firewalls. It can be observed from the
graph results below that there is a sharp decrease in Iptables when the filtering rules were 500.
160
Min Transfer rate (Mbits/s)
140
120
100
80
HTTP Requests
60
40
20
0
0
500
1000

Figure 5.4.1: Iptables HTTP transfer rate
Filtering rules also affect the HTTP transfer rate of Shorewall but not as compared to Iptables
which suddenly gives the poor performance at 500 rules. Once again the results proved that
Shorewall performed better in this case, by not degrading the performance sharply. The results
can be seen from the figure 5.4.2.
38
60
Min Transfer rate (Mbits/s)
50
40
30
HTTP Requests
20
10
0
0
500
1000

Figure 5.4.2: Shorewall HTTP transfer rate
5.5 System Resource Consumption

Linux (sar) utility was used to record the memory and CPU consumption. The result of both
firewalls indicates the differences. There is a slight change in the results of Shorewall CPU usage
which can be seen from the graphs clearly. However, shorewall consumes less memory as
compared to Iptables.
100
90
80
Usage in (%)
70
60
50
CPU
40
Memory
30
20
10
0
0
500
1000

Figure 5.5.1: Iptables CPU/Memory usage
39
100
90
80
Usage in (%)
70
60
50
CPU
40
Memory
30
20
10
0
0
500
1000

Figure 5.5.2: Shorewall CPU/Memory usage
There is a little change in CPU usage when the rules were 1000 on both firewalls but at 500 rules
there is a minor change in the results. As it is mentioned in design and method section that system
specifications also matter in this case. Since the system specifications on both firewalls were
same. Therefore, the conclusion is that both firewalls shares almost equal amounts of system
resources, but Shorewall consumes a little more CPU than Iptables and provides higher
performance.
5.6 Analysis
The results of both firewalls showed that Shorewall gives higher performance than Iptables when
the numbers of filtering rules are increased. Both firewalls were exposed to identical tests, in
addition with system specifications, a number of rules, design and methods. However, Shorewall
configuration was different from Iptables. As far as the performance is concerned, Shorewall high
output results lie in its configuration and working.
Shorewall is flexible and divides the incoming packets into different categories. It allows the user
to set the interface which is on that machine, the policies which will be applied to the interfaces
and exception to the policy in the form of rules when a request is sent to the interfaces.
Shorewall can be used on a dedicated firewall system and on a standalone GNU/Linux system.
Shorewall does not use Netfilters IP chain compatibility mode and thus take the advantage of
Netfilters connection state tracking capabilities which then results in higher performance in the
network.
40
6 Conclusion
The results show differences between each firewall evaluated according to the metrics defined,
using different methods and tools. Even though the system specifications and configurations were
same for both firewalls however, the way to configure each firewall was different. The results
found during evaluation prove that the chosen evaluation methodology is correct and confirm that
the tools were used properly in the experiments. The results indicate that Shorewall performed
better than Iptables for almost all parameters.
Throughput, latency, connection establishment and teardown rate indicated that number of
filtering rules have a negative impact on the performance of Iptables firewall in contrast with
Shorewall. Since both firewalls cannot detect duplicate rules. Firewalls process the rules from top
to bottom. The order of rules in which they are written is also important. Both firewalls indicate
that small packet sizes do not affect the performance of a firewall. Shorewall performed well
under the stress, which rejects the hypotheses that there is no difference between each and
Iptables give better performance than Shorewall.
The UDP traffic is not affected by the number of filtering rules in throughput calculation.
Moreover, there are no effects of latency on the sending device with UDP traffic. The receiving
device may have to buffer the UDP packets longer with a high amount of jitter.
Some of the limitations of Linux firewalls are also found such as they dont provide redundancy.
This is usually not a huge issue since the both firewalls are low cost and are perfect for small
business. Large enterprises usually use a combination of high-end firewalls and servers to provide
layered security for servers. Therefore, in order to gain the maximum performance from the
firewall, all the network architectures should be considered.
Such performance measurements may help in predicting that how long it will take to send a file
from one network to another including the ratio of discarded packets with the total number of
packets sent or loss rate. If the latency, available bandwidth, HTTP transfer rate, connection
establishment and teardown rate as a profile of firewall performance is known, then it is possible
to improve the performance of a network.
41
6.1 Future Work

A possible future direction is the security evaluation of firewalls. Another future direction of
work is to build more efficient and powerful tools to generate the traffic and test the firewall.
During the experiments 2.85 GB of data was transferred from client to server in 30 seconds for
firewall testing to represent the traffic of a small organization. Other metrics may be of interest to
characterize the firewall performance, which are not used in this project. Some performance
measurements for further testing may be the rules processing speed, packet loss rate, DOS
handling, IP fragmentation handling and maximum connection rate. Considering the methods
followed in this project, this project can be extended for Ipv6. Another future direction is the
improvement in scenarios. This project was limited to 1000 filtering rules. Large rule sets may
not be needed in real-time scenarios for a small sized network. Increasing the filtering rules may
be considered as an option, to see the performance during peak hours and worst case scenarios.
One can further work on filtering rules and develop different methods for firewalls to deal with
large number of rules. There is also need to develop a mechanism for the detection of duplication
of rules.
42
7 Bibliography
[1] W. Stallings, "Intruders," in Network Security Essentials, Applications and standards,
Pearson, 2011, p. 319.
[2] D. N. S. T. T. M. B. Hickman, "Benchmarking Methodology for Firewall Performance,"
RFC3511, p. 34, 04 2003.
[3] W. Stallings, "Firewalls," in Network Security Essentials, Pearson International Edition,
2010, pp. 22-3.
[4] P. Simoneau, "The OSI Model: Understanding the Seven Layers of Computer Networks,"
Global Knowledge Training LLC, 2006.
[5] G. Yang, "Introduction to TCP/IP Network," Iowa State University, Iowa, 1997.
[6] P. P. Tihomir Katic, "Optimization of Firewall Rules," in Information technology Interfaces,
University of Zagreb, Zagreb, 2007.
[7] D. Huang, "A quick iptables Tutorial," Arizona State University, Tempe, 2009.
[8] P. H. Karen Scarfone, "Guidelines on Firewalls and Firewall Policy," Packet Filtering
Firewall, vol. 1.0, no. National Institute of Standards and Technology, p. 48, 2009.
[9] D. Krishnan, "Firewall Design Principles," Stephen Woodall, 2004.
[10] I. D. V.S. Bagad, "Applicatin level gateways," in Cryptography and Network Security,
Technical Publications, 2008, pp. 5-19.
[11] D. W. Chadwick, "Network Firewall Technologies," University of Salford, Salford.
[12] S. C. C. Rita J. Cressy, "Iptables," LeRoy D. Cressy, 2004.
[13] derkeiler, "Considerations In Developing Firewall Selection Criteria," Adeptech System,
2009.
[14] E. J. W. S. Sharon D. Nelson, "Disgruntled Employees in your Law firm," Sensei
Enterprises, 2005.
[15] C. Roeckl, "An overview of firewall technology and how juniper Networks implements it,"
Juniper Networks, Sunnyvale, California, 2004.
[16] C. Berthelot, "Evaluation of a virtual firewall in a cloud environment," School of computing,
43
Edinburgh, 2011.
[17] V. Y. P. B. Joel Sommers, "Toward Comprehensive Traffic Generation for Online IDS
Evaluation," University of Wisconsin-Madison.
[18] H. Haas, "Network Address Translation," NAT, vol. 1.0, no. Cisco Systems Inc., p. 43, 2005.
[19] J. M. S. Bradner, "Benchmarking methodology for network Interconnect devices," Packet
sizes, vol. 1.0, no. march 1999, pp. 1-28, 1999.
[20] S. J. Shah, "Netperf: Network Benchmarking," California State University San Luis Obispo,
California, 1997.
[21] L. C. T. D. Ajay Tirumala, "Measuring end-to-end bandwidth with Iperf," Stanford
University.
[22] C. Khnast, "HTTPing web check," Linux-magazine, 2006.
[23] M. Wolfgang, "Host Discovery with nmap," Exploring nmap's default behavior, vol. 1.0, p.
16, 2002.
[24] J. Cooper, "The Book of Webmin," How I Learned to Stop Worrying and Love UNIX, vol.
4.0, p. 315, 2003.
[25] R. S. E. W. Ulf Lamping, "Wireshark User's Guide," Ulf Lamping, 2012.
[26] J. Snyder, "Firewalls in the Data Center: Main Strategies and Metrics," Firewall
performance evaluation metrics, table 2, vol. 1, p. 6, 2003.
[27] T. Aduolf, "systems, Department of technology enhanced learning information," protecting
networks, vol. 1, no. University of North Carolina, p. 16, 2010.
44
Appendix A Experimental Results

TCP Throughput
30 sec
Iptables
Packet Size
(bytes)
0 FW
rules
100 FW
rules
Throughput
(Mbits/sec)
500 FW
rules
1000 FW
rules
32
64
128
256
512
1024
1500
326.14
632.70
664.68
702.92
698.77
704.58
705.04
325.47
609.09
658.56
685.80
684.02
688.66
691.01
324.15
528.91
570.98
558.37
555.45
566.63
568.48
323.48
424.85
418.22
417.33
423.73
432.33
431.04
30 sec
Shorewall
Packet Size
(bytes)
0 FW
rules
100 FW
rules
Throughput
(Mbits/sec)
500 FW
rules
1000 FW
rules
32
64
128
256
512
1024
1500
326.42
641.27
663.27
695.95
694.89
698.44
701.70
326.35
638.04
662.53
694.17
691.62
697.51
700.78
322.40
624.71
661.67
691.43
689.59
693.73
689.10
322.02
604.68
657.58
685.95
683.71
687.07
682.04
UDP Throughput
30 sec
Iptables
Packet Size
(bytes)
0 FW
rules
100 FW
rules
Throughput
(Mbits/sec)
500 FW
rules
1000 FW
rules
32
64
128
256
512
1024
1500
109.49
173.08
312.25
533.08
540.36
680.70
611.87
107.27
172.60
311.38
533.22
540.20
681.24
613.09
107.46
172.46
311.34
533.13
540.24
681.06
611.90
106.37
172.14
310.95
532.93
540.05
679.50
611.43
45
30 sec
Shorewall
Packet Size
(bytes)
0 FW
rules
100 FW
rules
Throughput
(Mbits/sec)
500 FW
rules
1000 FW
rules
32
64
128
256
512
1024
1500
107.80
172.99
311.13
533.16
540.12
680.88
613.14
107.42
172.99
312.34
533.55
540.01
681.24
610.93
106.22
172.83
311.08
533.21
540.37
680.58
610.47
104.18
171.81
312.62
532.95
539.89
681.85
610.83
TCP Latency(Request/Response)
Trans. Rate per sec
30 sec
Iptables
Firewall rules
Req=1, Res =1 (bytes)
100
500
1000
9708.29
9162.74
9062.12
8971.02
7575.06
7358.23
6698.80
6655.03
6055.11
9307.35
9009.69
8983.82
8845.16
7701.61
7312.30
6674.67
6610.01
6025.38
8646.57
8559.67
8522.42
8458.15
7487.20
7029.60
6432.11
6402.04
5633.75
8146.74
7994.10
7926.38
7779.76
7139.73
6714.92
6129.57
6075.07
5576.20
Trans. Rate per sec
30 sec
Shorewall
Firewall rules
100
500
1000
9813.36
9125.05
9055.34
8907.19
8525.38
7384.56
6739.04
6691.03
6123.40
9763.01
9109.72
9009.49
8872.62
8278.76
7380.00
6734.52
6686.77
6051.34
9660.33
9046.38
8999.44
8866.42
7831.76
7342.40
6695.80
6650.46
6019.22
9683.23
9018.02
8965.41
8837.09
7525.32
7313.96
6635.10
6633.20
6009.64
46
UDP Latency (Request/Response)
Trans. Rate per sec
30 sec
Iptables
Firewall rules
100
500
1000
10342.23
10327.03
10319.89
10250.27
10214.24
9637.95
7537.39
7431.31
6835.44
10334.66
10336.14
10330.82
10255.49
10128.24
9416.32
7439.50
7352.46
6750.33
10332.32
10325.03
10215.79
9628.55
9439.68
8090.96
7170.36
7089.10
6528.06
8809.41
8796.49
8704.88
8602.25
8320.52
7261.37
6547.00
6516.90
5832.14
Trans. Rate per sec
30 sec
Shorewall
Firewall rules
100
500
1000
10349.19
10330.29
10323.94
10234.12
9672.49
8938.79
7276.30
7197.13
6614.47
10332.65
10331.03
10327.16
10204.77
9668.24
8915.14
7265.50
7178.36
6607.53
10340.56
10325.86
10320.70
9975.33
9649.01
8657.42
7221.77
7137.80
6565.21
10332.86
10323.75
10312.90
10226.86
9665.87
9124.13
7284.23
7206.50
6629.10
Connection Establishment Rate(Connect/Request/Response)
60 sec
Iptables
Firewall rules
Trans.
Rate/ sec
Req=1, Res =1(bytes)

100
500
1000
3512.92
3180.55
3423.64
3144.69
3220.63
2944.90
2989.28
2763.14
47
60 sec
Shorewall
Firewall rules
Trans.
Rate/ sec

100
500
1000
3419.07
3126.78
3409.00
3121.85
3345.96
3073.44
3251.52
2977.00
Connection Teardown Rate (Connect/Close)
60 sec
Iptables
Firewall rules
Trans.
Rate/ sec

100
500
1000
4343.23
4343.34
4269.49
4270.63
3918.25
3922.44
3743.65
3762.96
100
500
1000
4274.43
4273.89
4255.20
4258.71
4180.26
4174.33
4101.76
4102.28
60 sec
Shorewall
Firewall rules
Trans.
Rate/ sec

HTTP Transfer Rate

Iptables
512
65 sec
Packet Size
(bytes)
0 FW rules
HTTP Transfer
rate (Req: 60)
500 FW rules
1000 FW rules
Round-trip time
(ms)
Min
Avg
Max
Transfer Speed
(KB)
Min
Avg
Max
1.0
1.5
3.3
16784
209490
325243
1.1
1.5
3.4
1790
207492
323699
1.1
1.5
4.1
2219
224093
321235
48
Shorewall
512
65 sec
Packet Size
(bytes)
0 FW rules
HTTP Transfer
rate (Req: 60)
500 FW rules
1000 FW rules
Round-trip time
(ms)
Min
Avg
Max
Transfer Speed
(KB)
Min
Avg
Max
1.2
1.8
3.7
7580
20211
28342
1.3
1.9
3.8
6271
18611
39522
1.2
2.1
3.7
4710
19989
114682
System Resource Consumption (CPU/Memory Usage)

Iptables
30 sec
Packet Size
(bytes)
0 FW rules
500 FW rules
1000 FW rules
1024
CPU Usage(%)
Memory Usage(%)
71.52
57.49
76.72
58.93
89.68
59.94
Shorewall
30 sec
Packet Size
(bytes)
0 FW rules
500 FW rules
1000 FW rules
1024
CPU Usage(%)
Memory Usage(%)
81.76
42.71
87.04
51.50
90.17
52.34
49
Appendix B Iptables Script

#!/bin/sh
iptables -F
iptables -F
iptables -F
iptables -P
iptables -P
iptables -P
iptables -A
iptables -A
iptables -A
iptables -A
iptables -A
iptables -A
iptables -A
iptables -A
iptables -A
iptables -A
INPUT
OUTPUT
FORWARD
INPUT ACCEPT
OUTPUT ACCEPT
FORWARD ACCEPT
INPUT -s 194.47.155.1 -j DROP
INPUT -d 192.168.14.1 -j DROP
INPUT -i eth1 -s 194.47.155.10
INPUT -i eth1 -s 194.47.155.11
INPUT -i eth1 -s 194.47.155.12
INPUT -i eth1 -s 194.47.155.13
INPUT -i eth1 -s 194.47.155.14
INPUT -i eth1 -s 194.47.155.15
INPUT -i eth1 -s 194.47.155.16
INPUT -i eth1 -s 194.47.155.17
-j
-j
-j
-j
-j
-j
-j
-j
DROP
REJECT
REJECT
REJECT
DROP
DROP
REJECT
DROP
10 Iptables rules used in configuration
Appendix C Shorewall Script

#ACTION
#
DROP
DROP
DROP
REJECT
REJECT
REJECT
DROP
DROP
REJECT
DROP
SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE

PORT
PORT(S)
DEST
net:194.47.155.1
all
all
all
fw:192.168.14.1all
net:194.47.155.10all
all
net:194.47.155.11all
all
net:194.47.155.12all
all
net:194.47.155.13all
all
net:194.47.155.14all
all
net:194.47.155.15all
all
net:194.47.155.16
all
all
net:194.47.155.17all
all
USER/ MARK
LIMIT
GROUP
10 Shorewall rules used in configuration
50

Analysis of IPTables and Shorewall PDF

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Analysis of IPTables and Shorewall PDF

Загружено:

Авторское право:

Доступные форматы

COMPARATIVE ANALYSIS OF

IPTABLES AND SHOREWALL

Muhammad Zeeshan Ahmad

This thesis is presented as a part of Degree of

During the experiments, recommended benchmarking methodology for firewall performance

Aim and Scope .................................................................................................................................................... 2

Evaluation Method ............................................................................................................................................. 2

1.2.1 Performance Metrics ........................................................................................................................................ 2

Outline of the Project ......................................................................................................................................... 3

Working of a Firewall ........................................................................................................................................ 5

2.1.1 Packet Inspection .............................................................................................................................................. 5

Types of Firewalls ............................................................................................................................................... 7

2.2.1 Packet Filtering Firewall .................................................................................................................................. 7

Linux Iptables ..................................................................................................................................................... 8

Firewalls Selection .............................................................................................................................................. 8

Limitations of Firewalls ..................................................................................................................................... 9

Related Work .................................................................................................................................................... 10

Experimental steps ........................................................................................................................................... 11

Traffic Classification ........................................................................................................................................ 11

3.3.1 Real-time (Online) Evaluation ........................................................................................................................ 11

Traffic Duration ................................................................................................................................................ 12

Evaluation Procedure ....................................................................................................................................... 12

Evaluation Tools ............................................................................................................................................... 13

3.6.1 Netperf ............................................................................................................................................................ 13

Evaluation Metrics ........................................................................................................................................... 15

3.7.1 Throughput ..................................................................................................................................................... 17

Specifications of computers ............................................................................................................................. 22

Firewalls Installation ........................................................................................................................................ 22

4.2.1 Iptables ........................................................................................................................................................... 22

Configuration of Firewalls ............................................................................................................................... 23

4.3.1 Iptables ........................................................................................................................................................... 23

Implementation of Evaluation Metrics ........................................................................................................... 25

4.4.1 Throughput ..................................................................................................................................................... 25

Connection Establishment and Teardown Rate ............................................................................................. 37

HTTP Transfer Rate ........................................................................................................................................ 38

System Resource Consumption ....................................................................................................................... 39

Future Work ..................................................................................................................................................... 42

Figure 5.2.4: Shorewall UDP Latency (Request/Response) .......................................................... 36

Central Processing Unit

Graphical User Interface

Hypertext Transfer Protocol

Internet Control Messaging Protocol

Local Area Network

Maximum Transmission Unit

Network Interface Card

Open Systems Interconnection

Packet Inter Net Groper

Packets per Second

Request for Comments

Round Trip Time

Secure Sockets Layer

Transmission Control Protocol

User Datagram Protocol

Uniform Resource Locator

Figure 1.1.1: A Firewall General Model 2

1.1 Aim and Scope

1.2 Evaluation Method

Connection Establishment Rate

Connection Teardown Rate

HTTP Transfer Rate

System Resource Consumption

1.3 Outline of the Project