INTERNATIONAL WORKSHOP ON WIRELESS AD-HOC NETWORKS (IWWAN) 2004

Testing and Analyzing TCP Performance in a Wireless-Wired Mobile Ad Hoc Test Bed
Andreas Hafslund1, Lars Landmark2, Paal Engelstad3 and Frank Y. Li2 Thales Communications AS, Norway UniK University Graduate Center, Norway 3 Telenor FoU, Norway
the network connections often have high bit error ratio and high probability of packet loss. When mobile ad hoc networks are going to interconnect with the Internet, TCP has to be adjusted or altered to overcome the problems it has when running over wireless links. The research community has been investigating TCP performance over wireless links for several years. There exist some suggestions to improve TCP or to introduce new connection-oriented transport protocols. However, most of these studies are based upon simulation work using the network simulator ns-2 with the wireless extensions developed at CMU [4]. The results presented in this paper, on the other hand, are obtained by testing TCP performance experimentally in a real-life testbed. The global connectivity problem for ad hoc networking is well known. Apart from the problem of using TCP, it also includes issues such as addressing schemes, gateway technologies, autoconfiguration, security and authentication. Another issue that is important for Internet connectivity from ad hoc networks is site multi-homing (i.e. that there is more than one MANET node operating as a gateway). Since MANETs have usually no infrastructure, it is difficult to eliminate the possibility of multi-homing. Coordination and election/negotiation mechanism to suppress multi-homing would face problems in partially connected MANETs and MANETs with network partitions and mergers. Instead, all solutions to the global connectivity problems should take the possibility of multi-homing into consideration. Multi-homing makes however the global connectivity problems considerably more complex and challenging. We have focused our study of the problem regarding TCP connectivity between a wireless, multi-homed ad hoc network and a wired, global Internet, since this issue not yet has been fully investigated, and especially not when the MANET is multi-homed. We have tested how TCP connections behave when the ad hoc network has two different gateways to the Internet, and when a sender has to switch between gateways during a TCP session. This situation could occur for instance if one of the gateways becomes unavailable due to node mobility. In this paper we will detail the TCP problems for wireless networking. We will present our TCP testbed scenario and
1

AbstractTCP is the most used connection-oriented transport protocol in the Internet today. However, TCP has a number of problems when the connections are running over wireless links. This paper addresses some of them for mobile ad hoc networking. We have made a real-life testbed for experimenting and analyzing TCP connections from a multi-homed ad hoc network into the global Internet. Through a set of experiments, we show how TCP behaves when switching gateways, and also the capture and unfairness problems of TCP. The testbed and our initial results are the first step for making an improvement to TCP for ad hoc networking. Index Terms ad hoc networking, global connectivity, TCP, wireless testbed.

I. INTRODUCTION

obile Ad-hoc Network (MANET) [1] is usually a network consisting of mobile nodes with wireless network interfaces. Each node may function as both an endhost and an intermediate router for other nodes in the network. MANETs have usually no fixed infrastructure, and services on the Internet might not be available in these networks. A likely scenario is that nodes on an ad-hoc network in some cases also want to connect to nodes on the Internet, using services available there. For a widespread and successful deployment of MANETs, the ability to provide easy access to the Internet is therefore a prerequisite. A possible approach is to let a node that is participating in a MANET operate as an Internet gateway and provide other nodes within the MANET with Internet access. Since the Transmission Control Protocol (TCP) [2] is the most commonly used connection-oriented transport protocol in the Internet today, using TCP for global Internet connectivity from a MANET is a particularly important problem. TCP was initially developed for wired networks. There are certain characteristics of wireless ad hoc networks, however, that cause problems for TCP. Unlike for a wired network, the network topology of a MANET can be very dynamic, due to mobility of the nodes and radio channel fading. Furthermore,

This work is supported by THALES Communications AS, Telenor FoU, UniK University Graduate Center, and Norges Forskningsrd (NFR).

INTERNATIONAL WORKSHOP ON WIRELESS AD-HOC NETWORKS (IWWAN) 2004 parameters, and we will provide experimental results for TCP connections through global connectivity. We will investigate the TCP unfairness problem for two or more TCP flows going out from a multi-homed network. The paper is divided into 6 sections. Section 2 gives a brief look into related work on the subject, whereas Section 3 gives an overview of TCP functionality, TCP problems related to wireless links, and the global connectivity problem for ad hoc networking and TCP. We present our testbed architecture in Section 4, and our initial results in Section 5. We conclude our work in Section 6, where we also present our plans for further study on this subject. II. RELATED WORK A lot of research regarding TCP over wireless links has been undertaken. One much cited work is [9], which studies TCP performance related to the 802.11 MAC layer. This simulation study shows that the relation between MAC and TCP causes severe unfairness conditions between TCP flows. Both the 802.11 MAC layer and TCP have back-off timers. These timers will affect each other to give a very low throughput for TCP when there are packet collisions due to the hidden terminal problems. This work also demonstrate conflicts between TCP data packets and TCP ACKs going in different direction in a wireless network when the window sizes are greater than one. Some related problems are the focus in [10]. This paper demonstrates instability problems of TCP connections over wireless links. The work is based upon simulations in ns-2, and shows that the TCP maximum window size has an effect on the problem. Like [9], it also shows that there are problems between the 802.11 MAC and TCP, and that this gives unfairness between TCP flows. Another paper that investigates TCP problems for mobile ad hoc networks is [12]. Unlike previous work, this paper tries to model a realistic scenario, by doing simulations with ns-2. This includes better models for mobility, channel error, and shared-channel contention. Their results show that network disconnections and reconnections due to mobility have the most significant impact on TCPs performance in mobile ad hoc networks. A recent paper [13] shows that TCPs throughput and loss can be minimized for a given network topology and node mobility, with an optimized TCP window size. TCP achieves best throughput via improved spatial channel reuse Several papers try to improve TCPs performance over wireless links. Most of these are related to wireless cells, where there are either a base satellite station or an 802.11 access point. Recently, there has also been focus on the TCP problem for wireless, multi-hop, ad hoc networks. The study [11] gives an overview of the TCP problem, and also a survey of possible solutions to the problem. III. OVERVIEW OF TCP

A. TCP functionality TCP [2] is an end-to-end transport protocol that provides reliable, connection-oriented services to the application layer. TCP detects packets that are lost in the intermediate network, and trigger retransmissions until the data is correctly and completely received. This requires that the receiver returns ACKs (acknowledgements) for each data packet transmitted by the sender. TCP continually measures the time ACKs take to return from the receiver. A missed ACK is interpreted by TCP as a lost packet, which in turn makes TCP to retransmit the lost packet. In order to make a decision for the time it should wait before retransmitting the lost packets, TCP maintains a running average of the Round Trip Time (RTT). If the unacknowledged packet is delayed more than four times the calculated average RTT, TCP assumes the packet is lost. In addition to retransmitting the lost packet, TCP interprets lost (or very delayed) packets as a result of network congestion. This assumptions stems from the fact that the protocol is designed for wired networks where other types of packet losses are rare. The resulting action is that TCP backs off and adjusts its transmission rate. TCP has three main components for rate adaptation: Slow Start Congestion Avoidance Fast Retransmit The Slow Start mechanism makes a quick test of the capacity of the network to find the optimum rate of transmission. The sender starts with a congestion window size set to 1. For each ACK received, the window size is increased exponentially. When a maximum slow start threshold (ssthresh) is reached, the sender goes into the congestion avoidance phase. In this phase the window is first reduced to a half, and thereafter the increase of the congestion window is linear. With the Fast Retransmit mechanism in use the sender goes at immediately into the congestion avoidance phase after having received three duplicate ACKs without waiting for the retransmit timer. This leads to better network utilization, and thus also to better TCP connection throughput. In addition to the use of packet drop for indication of network congestion, the new Explicit Congestion Notification (ECN) mechanism has been added to improve congestion control [3]. When there is congestion in the network, intermediate routers will mark the Congestion Experience (CE) code point in the IP header of the TCP traffic. This will notify the end hosts that their connections go through a congested network. The end hosts can adjust their transmission rate without waiting for either a retransmit timer or duplicate ACKs. Thus, ECN can prevent unnecessary packet drops. B. TCP problems due to wireless links As outlined in Section 2, many studies show that TCP

http://www.unik.no/personer/paalee

INTERNATIONAL WORKSHOP ON WIRELESS AD-HOC NETWORKS (IWWAN) 2004 performs poorly over wireless links. In this section we will give an overview of some of these problems. The main problem is due to non-congestion packet loss. TCP is designed for a wired network where packet loss normally is a result of network congestion. This is necessarily not true for mobile ad hoc networks. Wireless links can have high bit error rates, and high probability of packet loss even when there is no congestion. TCP, however, will assume that the network is congested, and the throughput will be degraded. Another problem is related to mobility. When there are disconnections and reconnections in the network, it can easily lead to TCP timeouts. The congestion window will be lowered, and TCP will take a long time to recover. Mobility can also cause arrival of out-of-order packets. The TCP receiver will generate ACK for the highest in-order packet. This will result in duplicate ACKs, and when three duplicate ACKs have been received, TCP goes into the fast retransmit phase. C. TCP interconnectivity for ad hoc networks We have assumed that the ad hoc network is multi-homed. This means that there are at least two gateways providing connectivity to the Internet. The mobile nodes in the ad hoc network have to be assigned one or two gateways to use for their TCP connection. Usually, the metrics of the routing protocol is based on number of hops, as it is in our testbed. Then, the outgoing traffic will be sent via the gateway that is located the fewest number of hops away from the source node. For incoming TCP traffic from the Internet, the destination node will normally check the first packet it receives from the source node, and send all return traffic to the source IP address in the packet. Thus, if the source IP address has an address prefix that belongs to one of the gateways, the return packet will end up at this gateway. If the gateway is equipped with a full networking prefix (which is a likely scenario with IPv6), a MANET node that wants to get return traffic over the gateway can get an IP address with this prefix assigned from the gateway. This solution was proposed in [14]. In other cases, the gateway has only one routable IP address, which is a likely case with IPv4. All nodes communicating over the gateway must share the same address. To make this possible, the gateway may implement a Mobile IPv4 Foreign Agent (FA) [15]. The source node discovers the gateway, registers with the FA (i.e. the gateway), and registers the globally routable IP care-of-address of the FA with its own Mobile IP Home Agent (HA). This solution was proposed in [6] and [7]. Alternatively, the gateway may implement Network Address Translation (NAT) [8]. Then the source node sends traffic out through one of the gateways. The source IP address (and source port number if NAT port translation is implemented) will be translated to the IP address of the NAT. Hence, return traffic will be routed back to the same gateway,

which translates the packet and forwards the packet onto the ad hoc network. This solution has been proposed in [16]. Normally outgoing traffic belonging to the same TCP session should be consistently routed through the same gateway as that of the incoming traffic. If a NAT-based gateway is being used, for example, the source node cannot start sending traffic out on another gateway. In that case the IP source address of outgoing packets would not be translated correctly. They would not be recognized by the destination, and the TCP connection would break. If, on the other and, one of the other gateway technologies mentioned above is being used, the packets should be still be sent out through the same gateway as that of the incoming traffic. The reason is that the source IP address belongs to the gateway of the incoming traffic, and not to that of the outgoing traffic. Thus, the outgoing traffic will be an easy target for ingress filtering [17], which nowadays is more and more common to implement on routers in access networks. In some special scenarios the return traffic can end up at either of the two gateways. This happens if the MANET has a given networking prefix, both gateways belong to the same interior routing domain. Then both gateways can advertise connectivity to the MANET prefix, and either of the gateways can receive the return packets. Another example is when the two gateways are on the same link. Since we are mostly interested in the TCP performance in this paper, the exact mechanisms for ensuring global connectivity are of minor importance. As a simplification to gain full control over the testing environment, we therefore assume a scenario where the destination node, and the two gateways are on the same link. IV.
TESTBED ARCHITECTURE

A. Testbed description Our testbed is based upon six laptop PCs running on Linux, kernel 2.4.20, where two laptops act as gateways to the Internet. The topology of our testbed is illustrated in Figure 1. One node is the common destination for two sources, and it is located in the wired domain. The other three nodes are entirely in the wireless domain. Each of the nodes has 802.11b WL110 Compaq Wireless PC Card for the wireless interface, except the destination node (D) in the wired domain. Each wireless interface has 11Mbps as the theoretical maximum throughput. This node is running only an Ethernet card (802.3), and it is connected through a hub to the two gateways (GW1 and GW2). In the wireless network, the gateways are connected to two intermediate nodes (IN1 and IN2, respectively). The sending node (S) roams between the nodes IN1 and IN2. The wireless ad hoc network is running the Optimized Link State Routing (OLSR) protocol [5], which is a proactive protocol. All nodes exchange topology information with the other nodes in the network regularly through routing messages. The wired network runs static routing between D and GW1 and GW2.

INTERNATIONAL WORKSHOP ON WIRELESS AD-HOC NETWORKS (IWWAN) 2004 We let GW2 has a low bandwidth link to D; the link has a maximum throughput of 600Kbps. This is to simulate the problem where a mobile node roams between gateways with different bandwidth constrains to the Internet. An example of such a roaming could be between a Wireless LAN hotspot and an UMTS base station. Each TCP flow will try to use the entire available bandwidth over the link, and sends out packets with length of 1470 bytes.

the rerouting between two gateways takes maximum 6 seconds. This is due to the rerouting of the OLSR protocol. This first scenario shows the rerouting of the TCP flow when roaming between GW1 and GW2. We have maximum throughput for the TCP flow, except the seconds the rerouting takes. The results is shown in Figure 2:

Figure 2: S roams between GW1 and GW2. Figure 1: Testbed scenario for wired-wireless connectivity. B. TCP connection scenarios As an initial study for the TCP problem related to multihoming and the capture effect described in [9], we have specified several different scenarios. Scenario I: Only one TCP connection is set up between S and D in this scenario. The mobile node S roams between IN1 and IN2, and thus S roams between GW1 and GW2. For this scenario, we let the link between GW2 and D have full bandwidth. The main goal for this test is to find the time for the rerouting of the TCP flow between GW1 and GW2. Scenario II: In this scenario we have two TCP flows, one from S and one from IN2. We set the bandwidth for the link between GW2 and D to 600Kbps. S is initially connected to GW1, and then roams to GW2. The TCP flow from IN2 starts 5 seconds before the TCP flow from S. Scenario III: This is the opposite scenario of Scenario II, S is initially connected to IN2, and then S roams over to IN1 and GW1. Again we have two TCP flows, one from IN2 and one from S. The flow from S starts 5 seconds after the flow from IN2. V. TEST RESULTS In this section we will present our results from the testing of the different scenarios. Scenario 1 is an initial test to verify the correctness and functionality of the OLSR protocol, and also to find the time TCP needed when being rerouted between two different gateways. The next two scenarios are for identifying the capture effect when there are more TCP flows in the wireless network. Scenario I: In our earlier study [16], we have shown that Figure 3: S roams from GW1 to GW2. Scenario III: This is the opposite scenario of Scenario II, S is initially connected to IN2, and then S roams over to IN1 and GW1 after 10 seconds. Here we can see that when S roams over to IN1, S gets full access to the available bandwidth immediately. Here we do not see any capture effects, as expected. However, when the flow from S starts, Scenario II: When the TCP flow from S starts, it has full utilization of the bandwidth, but when S roams over to GW2 after 5 seconds, the TCP flow has zero throughput for more than the 6 seconds the OLSR rerouting needs. After that, the TCP flows gradually gets access to the bandwidth between IN2 and GW2. We can see that the capture effect is such that IN2 takes nearly all resources for the link, and the flow from S will gradually get better throughput. This is shown in Figure 3.

INTERNATIONAL WORKSHOP ON WIRELESS AD-HOC NETWORKS (IWWAN) 2004 we do see the capture effect. Since the flow from IN2 starts 5 seconds before the flow from S, this flow takes almost all resources for the link between IN2 and GW2. When the flow from S starts, it takes 5 more seconds before this flow acquires a reasonably throughput. This is shown in Figure 4.

preparing the OLSR protocol, which is used in the ad hoc network. REFERENCES
[1] J. Macker, Mobile Ad hoc Networking (MANET): Routing Protocol Performance Issues and Evaluation Considerations. IETF RFC 2501, January 1999. [2] J. Postel, Transmission Control Protocol. IETF RFC 793, September 1981. [3] K. Ramakrishnan, S. Floyd, D. Black, The Addition of Explicit Congestion Notification (ECN) to IP. IETF RFC 3168, September 2001. [4] http://www.isi.edu/nsnam/ns/index.html. [5] T. Clausen, P. Jacquet, A Laoiti, P Minet, P. Muhlethaler, A Qayyum, L Viennot, Optimized Link State Routing Protocol. IETF RFC 3626 October 2003. [6] C. Perkins, E. Belding-Royer, Y. Sun, Internet Connectivity for Ad Hoc Mobile Networks, International Journal of Wireless Information Networks, April 2002. [7] E. Belding-Royer, Y. Sun, C. Perkins, Global Connectivity for IPv4 Mobile Ad Hoc Networks , IETF Internet Draft, work in progress, November 2001. [8] P. Srisuresh, M. Holdrege, "IP Network Address Translator (NAT) Terminology and Considerations", IETF RFC 2663, August 1999. [9] M. Gerla, K. Tang, R. Bagrodia, "TCP Performance in Wireless, Multihop Networks", IEEE WMCSA99, February 1999. [10] S. Xu, T. Saadawi, "Does the IEEE 802.11 MAC Protocol Work Well in Multihop Wireless Ad Hoc Networks", IEEE Communication Magazine, June 2001. [11] H. Elaarag, "Improving TCP Performance over Mobile Networks", ACM Computing surveys, Septemeber 2002. [12] Z. Fu, X. Meng, S. Lu, "How Bad TCP Can Perform In Mobile Ad Hoc Networks", IEEE ISCC, July 2002. [13] Z. Fu, P. Zerfos, H. Luo, S. Lu, L. Zhang, M. Gerla, "The Impact of Multihop Wireless Channel on TCP Throughput and Loss", IEEE INFOCOM 2003. [14] R. Wakikawa, J Malinen, C. Perkins, A. Nilsson, A. Tuominen Global Connectivity for IPv6 Mobile Ad Hoc Networks , IETF Internet Draft, work in progress, November 2001. [15] C. Perkins (editor), IP Mobility Support for IPv4. IETF RFC 3220, January 2002. [16] P. Engelstad, A. Tnnesen, A. Hafslund, G. Egeland Internet Connectivity for Multi-Homed Proactive Ad Hoc Networks To appear in Proceedings of the 2004 International Conference on Communication (ICC 2004), Paris, June 20-24, 2004. [17] P. Ferguson and S. Senie, Ingress Address Filtering: Defeating Denial of Service Attacks which employs IP Source Address Spoofing. IETF RFC 2827, May 2000.

Figure 4: S roams from GW2 to GW1. As was shown in [9] through simulations, we have shown with our wireless-wired testbed architecture. The TCP capture effect will degrade the TCP performance when more than one flow tries to use the same wireless link. This is a problem that should be investigated further. Also, we have shown that the multi-homing of an ad hoc network causes delay because of the rerouting, but this causes no problem for TCP. However, when one of the gateways has a lower bandwidth link to the Internet, TCP will have to adjust to this without timing off due to congestion. VI. CONCLUSION AND FURTHER STUDY The work presented in this paper is the initial part of our project in evaluating TCP performance, and further improving and implementing the protocol for real-life ad hoc networks. Our main interest is for multi-homed wireless, ad hoc networks. As an initial step, we have constructed two different scenarios for analysing TCP performance in a Linux testbed. Through a set of experiments, we have shown how rerouting in ad hoc networks affects TCPs performance, and how unfairness problem exists when multiple TCP connections coexists. For further study, we shall focus on the following three aspects: Further study of multi-homed ad hoc networks. Implementing and testing of an improved TCP for wireless networks. Investigating how a gateway can be used for controlling the TCP traffic and performance to and from the ad hoc network. ACKNOWLEDGMENT We would also like to thank Andreas Tnnesen for help in