On the convergence of path vector routing protocols

Jo o Lus Sobrinho a Instituto de Telecomunicacoes, Instituto Superior T cnico e Torre Norte, Av. Rovisco Pais 1 1049-001 Lisboa, PORTUGAL Email: joao.sobrinho@lx.it.pt
Abstract This work was motivated by recently reported experimental results on the delayed convergence of the Border Gateway Protocol (BGP), which is the standard for inter-domain routing in the Internet. BGP is a path vector protocol. We investigate the convergence properties of path vector protocols and, in particular, their alleged effectiveness in dealing with the count-to-innity problem that plagues conventional distance vector protocols. We assume synchronous operation of the protocols, and we study the cases when a destination comes up, the network topology changes while preserving connectedness, and a destination goes down. It is known that path vector protocols do not count to innity. We show that they may still count to the length of the longest possible path in the network, when a destination goes down. On the other hand, when a destination comes up or the topology of the network changes, convergence time depends on the diameter of the network. The length of the longest path in a network may differ substantially from its diameter. In order to further understand the convergence properties of path vector protocols, we simulated their behavior over random graphs. We veried that path vector protocols converge much more quickly when a destination comes up than when it goes down.

I. I NTRODUCTION As the Internet evolved from an experimental network to a world-wide public telecommunications infrastructure, its underlying protocols have become subject to intense scrutiny. The Internet is organized as a collection of interconnected Autonomous Systems (ASs). Routing between ASs, also called inter-domain routing, is the responsibility of the Border Gateway Protocol (BGP) [1], [2]. BGP is a path vector protocol, meaning that entire paths to any given destination are exchanged between neighbor ASs. BGP includes mechanisms for policybased routing, and defaults to shortest path routing if routing policies are not enforced. Recent experimental results on the routing behavior of BGP [3], [4], [5] show instability and routing convergence problems, and they lead researchers into seeking explanations for that behavior. Grifn and Wilfong [6] studied how policy-based routing can result in routing divergence and routing oscillations. Later, Labovitz et al. [5] have shown that delayed routing convergence may well arise even in the absence of routing policies. Our work was motivated by these results and studies. We investigate the convergence properties of path vector protocols, without policy-based routing, in general graphs. In path vector protocols, each network vertex collects the paths to a given destination vertex advertised by its neighbors. It selects the shortest among those paths that does not yield a routing loop. If there is more than one shortest path under these conditions, the tie is broken with a deterministic rule based on identiers assigned

by a vertex to its neighbor vertices. We assume synchronous operation of the protocols whereby path computation and advertising occurs at globally agreed instants of time. Synchronous operation may arise in practice [7], [5], but the main reason to consider it here is the simplicity with which convergence results can be gauged and compared based on the number of protocol iterations required to reach some stable state. Fixing any given vertex as destination, our main results are as follows. (i) When the destination vertex comes up, the diameter of the network provides a tight upper bound on the number of protocol iterations required for stability, much as in distance vector protocols. (ii) After topological changes in the network that preserve its connectedness, the diameter of the network again provides a tight upper bound on the number of protocol iterations required for stability, but now convergence may take longer than in distance vector protocols. (iii) When the destination vertex goes down, the number of protocol iterations required for stability is tightly upper bounded by the length of the longest possible path in the network; a distance vector protocol would count to innity in this case [8]. Similar results have recently been reported by Labovitz et al. [9], and we comment on them later in the paper. The length of the longest path in a network may differ substantially from its diameter. In order to further understand the convergence properties of path vector protocols, we have simulated their behavior over random graphs generated according to a model from [10]. We veried that, indeed, path vector protocols converge much more quickly when a vertex comes up than when a vertex goes down. The paper is organized as follows. In the next section, we present the network model and introduce the notation. In Section III, we describe the path vector protocol studied and allude to its afnity with BGP. The convergence properties for a destination coming up, topological changes, and a destination going down are derived in Sections IV, V, and VI, respectively. Section VII presents results of simulations over random graphs, and Section VIII summarizes the conclusions and points to future research. II. N ETWORK MODEL AND NOTATION A network is represented by a connected undirected graph, G = (V, E), on the set of vertices V = {0, , n 1}. Vertex j is said to be a neighbor of vertex i if ij E. The set of neighbors of vertex i is denoted by N i and the cardinality of this set

0-7803-6711-1/01/$10.00 (C) 2001 IEEE

292

is the degree of vertex i. Every vertex attributes an identier to each one of its neighbors. The identier that vertex i attributes to its neighbor j is denoted by u ij . The identiers attributed by a vertex are locally unique, i.e., u ij = uir implies j = r, for all i V and j, r Ni . A walk is a non-empty sequence of vertices v w vw1 v1 v0 such that vi vi1 E for w i 1. In walk v w vw1 v1 v0 , vi1 is called the successor of vi , w i 1. The length of walk , denoted by D(), is the number of edges it contains. A walk is a path if all of its vertices are distinct. Walks can be regarded as words over the alphabet V , and paths as a simple words over V ; simple words are words without repeated letters. Let W be the set of all non-empty simple words over V , and let W = W {}, where , W , denotes the absence of a path. Dene D() = , so that, informally, is a path of innite length. Further, dene the operation from V W to W as follows: i = i if W and i W otherwise,

PATH V ECTOR RULES

k1 dk = minjNi {D(i j )} i k1 choose s s. t. uis = min{uij |j Ni and D(i j ) = dk } i k k1 i = i s

Fig. 1. Path vector rules for vertex i = 0.

III. PATH VECTOR PROTOCOL In path vector protocols, each vertex learns estimated paths to reach destination vertex 0 from every one of its neighbors. The vertex builds its own estimated path from those, according to following rules: (i) the vertex is appended to each of the learned paths and the resulting shortest path is selected (walks that are not paths are excluded); (ii) in case of a tie, the path that goes through the neighbor having the lowest vertex identier is selected; (iii) if the vertex cannot construct a path to vertex 0, the latter is declared unreachable. When a vertex learns an estimated path from one of its neighbors it recomputes its own estimated path within nite time and, if there is a change, it advertises the new path to all of its neighbors. We assume synchronous operation of the protocol whereby estimated paths are computed and advertised at globally agreed instants of time, and the path advertisements are available at neighbor vertices at the next globally agreed instant of time. With this model, we can describe the evolution of the protocol in terms of iterations. k Let i denote the estimated path computed at vertex i at iteration k. Figure 1 presents the rules of operation of the path vector protocol for any vertex i = 0 using the notation introduced k in the previous section. For vertex i = 0, we have 0 = 0 for k 0. A number of observations are in order. (i) The operation excludes from consideration walks to vertex 0 that are not paths. (ii) The value d k is the distance to vertex 0 estimated i by vertex i at iteration k. The value d k serves only to establish i the set of neighbors that are eligible as successors of vertex i at iteration k. (iii) The estimated paths at vertex i depend on the vertex identiers, u ij , j Ni . We study the convergence properties of path vector protocols in the following situations: (i) destination vertex 0 comes up; (ii) the topology of the network changes, e.g., as a result of edges going down and coming up, but the network remains connected; (iii) destination vertex 0 goes down. In each of these 0 situations there is an initial set of path estimates, i , i = 0, and the convergence of the protocol is gauged by its time complexity, which is the number of protocol iterations required to reach a stable state. 0 Denition 1: Starting with the initial conditions i , i = 0, k and 0 = 0 for k 0, the stability index of the path vector k+1 k protocol, kP V , is the least integer k such that i = i , for all i V . Our network model and path vector protocol capture the essentials of inter-domain routing in the Internet, without policybased routing. Each vertex corresponds to an AS which is assigned a set of IP prexes. A destination corresponds to one of

where, for W , i denotes the concatenation of letter i with word , i.e., letter i followed by word . Clearly, either D(i ) = 1 + D(), if i is not a letter of , or D(i ) = , if i is a letter of . The distance from vertex i to vertex j, denoted by d , is ij the length of a shortest path from i to j. We let l ij denote the length of a longest path from vertex i to vertex j. The diameter of the network is dened as diam(G) = max i,jV (d ) ij and the length of the longest path in the network is given by long(G) = maxi,jV (lij ). They may differ by as much as the number of vertices in the network minus two, i.e., 0 long(G) diam(G) n 2; the rst and second inequalities become equalities in linear and complete graphs, respectively. Given a destination vertex t, the set N i of neighbors of vertex i can be partitioned into two sets, N it and Nit . The set Nit contains the neighbors of i that are along shortest paths towards t, and the set Nit contains all other neighbors of i. Formally, j Nit if j Ni and d = 1 + d , and j Nit if j Ni it jt and d < 1 + d . Clearly, Nit is not empty for i = t. The set it jt of stable paths for destination vertex t is composed of the paths it , i V , dened as follows:
it = ist t

with uis = minjNit {uij }, if i = t if i = t.

In words, the stable paths for vertex t are shortest paths to t where the successor of each vertex in each path is its neighbor of lowest vertex identier that is along a shortest path to t. Without loss of generality, from now on we take vertex 0 as the destination vertex, and we drop the subscript 0 when no ambi guity arises. For example, d = d , Ni = Ni0 and i = i0 . i i0 We also dene d = maxiV {di } and l = maxiV {li }; d is the longest distance to vertex 0 and l is the length of the longest possible path to vertex 0.

293

these IP prexes [2]. When we say that a destination vertex comes up this means that the AS announces a new IP prex. Similarly, a destination vertex goes down means either that the AS explicitly withdraws an IP prex or that the BGP sessions between the AS and its neighbor ASs have been broken. The vertex identiers used to break ties in the selection of a shortest path roughly correspond to BGP identiers [2]. As to the assumption of synchronization, the specication of BGP includes timers to limit the rate at which path advertisements can be sent. It has been noted in [5] that these timers may contribute to a synchronous operation. IV. D ESTINATION VERTEX COMES UP Suppose that destination vertex 0 comes up at iteration 0, and let G represent the resulting network at that time. The initial 0 conditions are i = , i = 0. Theorem 1: Suppose that destination vertex 0 comes up at iteration 0. The stability index of the path vector protocol becomes kP V = d diam(G), and it is independent of the k vertex identiers. Moreover, i = i for k kP V . Proof: We use induction to prove the following assertion, k which implies the statement of the theorem: i = i for d i k k, and i = for di > k. 0 0 The base is trivial, since 0 = 0, and i = for i = 0, that is, for di > 0. Let vertex i be such that d i k + 1 and examine the neighbors of i. For j N i , we have d k. The induction j k hypothesis tells us that j = j and since i cannot be a letter k of j we obtain D(i j ) = d . Suppose now that j N i . i k If d k, we have from the induction hypothesis j = j j k and, thus, D(i j ) 1 + dj > di . Similarly, if dj > k then k k j = and, thus, D(i j ) = > d . According to the i rules of the protocol in Figure 1, d k+1 = d , vertex i chooses i i as its successor the vertex s such that u is = minjNi {uij }, and k+1 k forms path i = i s = i s = i . To conclude the proof, consider the case d > k + 1. Then, i dj > k for j Ni . The induction hypothesis tells us that k+1 k = . j = , which implies i Theorem 1 states that the stability index equals the longest distance to vertex 0, d , and, hence, it is tightly upper bounded by the diameter of the network. This is the same as in a distance vector protocol. V. C HANGES IN THE TOPOLOGY OF THE NETWORK Suppose that starting from a set of stable paths for destination vertex 0 some edges and vertices fail while others come into operation, such that at iteration 0 the network G is con0 nected. The initial conditions can be any set of paths i such 0 that D(i ) > 0, i = 0. Theorem 2: Suppose that the topology of the network changes but its connectedness is preserved. The stability index of the path vector protocol is upper bounded as k P V d k diam(G). Moreover, i = i for k kP V . Proof: The proof is similar to that of Theorem 1 and is omitted.

5 4 3 2 0 5

5 5 4 3 1 0

4 3 2 0 4

4 4 3 1 0

320 3

3 310

10 1

2 0 2

10

2 0

0 0

0 0

Fig. 2. Illustrating that the stability index depends on the vertex identiers when edge 1 0 goes down. On the left, u3,2 < u3,1 , only vertex 1 has to update its estimated path; the stability index is one. On the right, u < u3,2 , all 3,1 vertices, except vertex 2, have to update their estimated paths; the stability index becomes four which is the diameter of the network.

Contrary to the case of destination vertex 0 coming up, the longest distance to vertex 0 is only an upper bound on the stability index. Furthermore, the stability index now depends on the vertex identiers and may be greater than in distance vector protocols. We illustrate these points with Figure 2. Starting from a set of stable paths for destination vertex 0, edge 1 0 goes down at iteration 0. If u 3,2 < u3,1 then the protocol converges at the rst iteration, since only vertex 1 has to change its estimated path from 1 0 to 1 3 2 0. On the other hand, if u3,1 < u3,2 all vertices, except vertex 2, have to update their estimated paths. The stability index equals the diameter of the network which is four. If a distance vector protocol were used instead, the protocol would converge as soon as the estimated distances and successor vertices had remained the same from one iteration to the next; if u 3,1 < u3,2 , it would converge at the second iteration. VI. D ESTINATION VERTEX GOES DOWN Suppose that starting from a set of stable paths for destination vertex 0, vertex 0 goes down at iteration 0. Now, G represents the network before destination vertex 0 went down. The initial conditions is the set of stable paths for destination vertex 0. That 0 is, i = i , i = 0. The protocol converges at the rst iteration k k for which i = or, equivalently, when D( i ) = for all i = 0. It is easy to see that the stability index depends on the vertex identiers, a fact illustrated in Figure 3. However, we show in Theorem 3 that, for any given network, the stability index when destination vertex 0 goes down is always upper bounded by the length of the longest possible path to vertex 0, l , and, moreover, that there are vertex identiers for which this bound is tight. The following two lemmas simplify the proof of Theorem 3. Lemma 1: Suppose destination vertex 0 goes down at iterak tion 0. We have D( i ) > k for every i = 0 and k 0. Proof: The proof is by induction, the base case being

294

0
3 1 0 3 4 2 0 4

2
4 2 0 4 4 2 3 1 0 4

3
3 4 2 3 1 0 4

4
3

3 1 0 3

10

2 0

1
0

2 3 1 0

1
0

2 3 1 0

1
0

1
0

0
3 2 0 3 4 2 0 4 1 3 2 0 1

0
3 2 0 3 4 2 0 4 1 3 2 0 1

0
3

0
4

0
4

1 0

2 0

v0 0 0

Fig. 3. Illustrating that the stability index depends on the vertex identiers when destination vertex 0 goes down. In the rst row, u < u3,2 ; the stability index 3,1 is four, and, thus, equals the length of a longest path to vertex 0. In the second row, u < u3,1 ; the stability index is three. 3,2

obvious. Consider vertex i at iteration k + 1. For every j k Ni , the induction hypothesis implies D( j ) > k. Therefore, k k D(i j ) 1 + D(j ) > k + 1, for every j N i . Whatever k+1 successor is chosen by vertex i, we obtain D( i ) > k + 1. Lemma 2: Let vp vp1 v1 0 be a path in the network from vertex vp to vertex 0. Then, there are vertex identiers, u ij , with i V and j Ni , such that, if destination vertex 0 goes down k at iteration 0, then vk+1 = vk+1 vk v1 0 for p > k 0. Proof: Suppose that the vertex identiers have been assigned such that u vk+1 ,vk < uvk+1 ,j for every j N vk+1 , j = vk+1 , and p > k > 0. 0 For k = 0, we have, trivially, v1 = v1 = v1 0. Assume that k vk+1 = vk+1 vk v1 0 and consider iteration k + 1. Vertex k k vk+2 is not a letter of vk+1 . Thus, D(vk+2 vk+1 ) = k + 2. Take any other neighbor j of vertex v k+2 . From Lemma 1, k k we know that D(j ) > k and, thus, D(vk+2 j ) > k + k k 1. Therefore, D(v k+2 vk+1 ) D(vk+2 j ) and, since uvk+2 ,vk+1 < uvk+2 ,j , vertex vk+2 chooses vertex vk+1 as its k+1 k successor at iteration k + 1, yielding vk+2 = vk+2 vk+1 = vk+2 vk+1 v1 0. Theorem 3: Suppose that destination vertex 0 goes down at iteration 0. The stability index of the path vector protocol satises the inequality kP V l long(G), whatever the vertex identiers. Moreover, there are vertex identiers for which kP V = l . l Proof: From Lemma 1, we know that D( i ) > l for all i = 0. However, there are no paths to vertex 0 with lengths l greater than l . Therefore, D( i ) = for every i = 0, which implies that kP V l . On the other hand, let v l vl 1 v1 0 be a path of length l from vertex v l to vertex 0. From Lemma 2, there are vertex

l 1 identiers for which vl = vl vl 1 v1 0. Hence, kP V > l 1 and, thus, kP V = l . Using a different approach, Labovitz et al. [9] have recently shown that the length of the longest path to a destination vertex provides an upper bound on the convergence time of a path vector protocol when that destination goes down. Our results further show that the convergence time depends on the vertex identiers and that, for any given network, there are vertex identiers for which the upper bound is tight. It remains an open question whether, for the same scenario, one can also nd a tight lower bound on the convergence time.

VII. C ONVERGENCE RESULTS OVER RANDOM GRAPHS We have seen that when a destination vertex comes up the stability index is tightly upper bounded by the diameter of the network, and that when a destination vertex goes down the stability index is tightly upper bounded by the length of the longest path in the network. These two parameters may differ by as much as the number of vertices in the network minus two. Moreover, as the network becomes more dense the difference between the diameter and the length of the longest path in the network increases. In this section, we assess, through simulations, the convergence behavior of path vector protocols over connected random graphs generated according to the exponential model of [10]. Briey, the model locates vertices randomly in the plane, and the probability of there being an edge between two vertices decreases with the Euclidean distance separating them. We present results for two representative graphs. Both have 100 vertices. In the sparser of the two graphs, the average vertex degree is 3.72; in the denser graph, the average vertex degree is 5.92. We have used different seeds to generate the random numbers that set the locations of the vertices in the plane, stipulate the edge set, and attribute the vertex identiers. In terms of

295

convergence, the cases of a destination vertex coming up and topological changes are similar, so we do not present results for the latter events. Figure 4 shows the stability index when both a destination vertex comes up and goes down, for 10 different destination vertices. When a destination vertex goes down, the stability index depends on the vertex identiers. For each of the 10 destination vertices we ran the path vector protocol 10 times with distinct, randomly chosen, vertex identiers. The error bars mark the minimum and maximum stability indices over the 10 sets of vertex identiers and the bullet gives the average stability index. The main conclusion to be drawn from these results is that the stability index when a destination vertex comes up differs signicantly from the stability index when a destination vertex goes down, even in relatively sparse networks. For example, in the sparser of the two networks presented, the stability index when a vertex comes up varies from 5 to 7 whereas the stability index when a vertex goes down varies from 61 to 76. This is consistent with results from random graph theory which show that the diameter of the network can be much less than the length of the longest path in the network [11]. Another interesting observation is that the vertex identiers do not signicantly affect the stability index when a destination vertex goes down. The highest variation we observe in Figure 4 is from 63 to 75, for the sparser network, with vertex 90 as the destination. In practice, these conclusions indicate that the intrinsic ability of path vector protocols to deal with the count-to-innity protocol does not guarantee fast convergence, and that alternatives for fast convergence have to be sought. VIII. C ONCLUSIONS AND FUTURE RESEARCH The growing importance of BGP warrants a better understanding of the convergence properties of path vector protocols. We investigated this issue from a fundamental standpoint. The convergence properties of path vector protocols under synchronous operation were related to invariants of the networks over which they run. The stability index when a destination vertex comes up is tightly upper bounded by the diameter of the network, whereas the stability index when a destination vertex goes down is tightly upper bounded by the length of the longest path in the network. Both indices depend on the specic destination vertex considered but the stability index when a destination vertex goes down depends, in addition, on the identiers that the vertices assign to their neighbors. As when a destination vertex comes up, changes in the topology of the network that preserve connectedness keep the stability index below or equal to the diameter of the network, with the exact value dependent on the vertex identiers. The theoretical results were substantiated by simulations of the behavior of path vector protocols over random graphs. We conrmed that it takes much longer to converge after a destination vertex goes down than after it comes up. These results suggest that, when a destination vertex goes down, we cannot rely on the mechanisms of path vector protocols for fast convergence. Spurred by this conclusion, we are currently exploring

Destination vertex comes up

10 9 8 7 100 vertices, 186 undirected edges 100 vertices, 296 undirected edges

Stability index

6 5 4 3 2 1 0 0 10 20 30 40 50 60 70 80 90

Vertex

Destination vertex goes down

100 95 90 85 100 vertices, 186 undirected edges 100 vertices, 296 undirected edges

Stability index

80 75 70 65 60 55 50 0 10 20 30 40 50 60 70 80 90

Vertex

Fig. 4. Stability indices when destination vertices come up and go down.

simple modications to path vector protocols that lead to improved convergence time. R EFERENCES
[1] B. Halabi, Internet Routing Architectures, Cisco Press, Indianapolis, IN, 1997, ISBN 1-56205-652-2. [2] J. W. Stewart III, BGP4: Inter-Domain Routing in the Internet, Addison Wesley, Reading, MA, 1999, ISBN 0201379511. [3] V. Paxson, End-to-end routing behavior in the Internet, IEEE/ACM Transactions on Networking, vol. 5, no. 5, pp. 601615, October 1997. [4] C. Labovitz, G. R. Malan, and F. Jahanian, Internet routing instability, IEEE/ACM Transactions on Networking, vol. 6, no. 5, pp. 515528, October 1998. [5] C. Labovitz, A. Ahuja, A. Bose, and F. Jahanian, Delayed Internet routing convergence, in Proc. ACM SIGCOMM2000, Stockholm, Sweden, August 2000, pp. 175187. [6] T. G. Grifn and G. Wilfong, An analysis of BGP convergence properties, in Proc. ACM SIGCOMM1999, Cambridge, MA, August 1999, pp. 277288. [7] S. Floyd and V. Jacobson, The synchronization of periodic routing messages, IEEE/ACM Transactions on Networking, vol. 2, no. 2, pp. 122 136, 1994. [8] R. Perlman, Interconnections: Bridges, Routers, Switches, and Internetworking Protocols, Addison Wesley, second edition, 2000, ISBN 0-20163448-1. [9] C. Labovitz, A. Ahuja, R. Wattenhofer, and S. Venkatachary, The impact of Internet policy and topology on delayed routing convergence, in Proc. IEEE INFOCOM2001, Anchorage, AK, April 2001. [10] E. W. Zegura, K. L. Calvert, and M. J. Donahoo, A quantitative comparison of graph-based models for Internet topology, IEEE/ACM Transactions on Networking, vol. 5, no. 9, pp. 770783, December 1997. [11] R. Diestel, Graph Theory, Springer-Verlag, New-York, second edition, 2000, ISBN 0-387-98976-5.

296