BGP Optimization

CP-1007 Optimizing BGP
Jeff Doyle
Principal Architect
FishNet Security
Agenda
Introduction
Improving Failure Detection and Convergence
Improving BGP Performance
Improving BGP Management
Conclusion
2014 Cisco and/or its affiliates. All rights reserved. 3

Introduction
Whats So Special About BGP?
Optimized for complex policy support
Menu of path attributes
Control incoming, internal, and outgoing routes separately
Menu of tools for setting policy
Route maps
Prefix filters
AS_PATH filters
Policy maps
Segmentation
Routes between (EBGP) or through (IBGP) Autonomous Systems
Path Vector protocol
Multiple address family support
Enables one protocol to support a multiservice core

BGP Overview
Path Vector
AS1 AS6 AS3

218.54.0.0/16
AS5 AS2 AS4

BGP Overview
Path Vector
AS1 218.54.0.0/16
[1]
AS6 AS3
218.54.0.0/16 > 218.54.0.0/16 [1]
218.54.0.0/16
[1]
AS5 AS2 AS4

> 218.54.0.0/16 [1]

BGP Overview
Path Vector
AS1 AS6 218.54.0.0/16 AS3

[6, 1]
218.54.0.0/16 > 218.54.0.0/16 [1] > 218.54.0.0/16 [6, 1]
AS5 218.54.0.0/16
[5, 1]
AS2 AS4
> 218.54.0.0/16 [1] > 218.54.0.0/16 [5, 1]

BGP Overview
Path Vector
AS1 AS6 AS3

218.54.0.0/16 > 218.54.0.0/16 [1] > 218.54.0.0/16 [6, 1]
AS5 AS2 218.54.0.0/16 AS4

[2, 5, 1]
> 218.54.0.0/16 [1] > 218.54.0.0/16 [5, 1] > 218.54.0.0/16 [2, 5, 1]

BGP Overview
Path Vector
AS1 AS6 AS3

> 218.54.0.0/16 [6, 1]
218.54.0.0/16 > 218.54.0.0/16 [1] 218.54.0.0/16 [4, 2, 5,1]
218.54.0.0/16
[4, 2, 5, 1]
AS5 AS2 AS4

> 218.54.0.0/16 [1] > 218.54.0.0/16 [5, 1] > 218.54.0.0/16 [2, 5, 1]

BGP Overview
Path Vector
AS1 AS6 AS3

> 218.54.0.0/16 [6, 1]
218.54.0.0/16 > 218.54.0.0/16 [1] 218.54.0.0/16 [4, 2, 5,1]
AS5 AS2 AS4

> 218.54.0.0/16 [1] > 218.54.0.0/16 [5, 1] > 218.54.0.0/16 [2, 5, 1]

BGP Overview
BGP Messages
Open
Update
Notification
Keepalive
Route Refresh

BGP Overview
BGP Messages
Open
Type 1
Requests BGP session open to a neighbor
Sent after TCP session is established
Update
Notification
Keepalive
Route Refresh

BGP Overview
BGP Messages
Open
Update
Type 2
Network Layer Reachability Information (NLRI)
Path attributes
Withdrawn routes
Notification
Keepalive
Route Refresh

BGP Overview
BGP Messages
Open
Update
Notification
Type 3
Notifies neighbor of detected error
BGP session always closed after sending Notification
Keepalive
Route Refresh

BGP Overview
BGP Messages
Open
Update
Notification
Keepalive
Type 4
Periodically sent after BGP session is open
Default 60 seconds
Route Refresh

BGP Overview
BGP Messages
Open
Update
Notification
Keepalive
Route Refresh
Type 5
RFC 2918
Asks neighbor to re-send Adj-RIB-Out
Allows non-disruptive application of new inbound policies

BGP Overview
BGP Routing Information Base (RIB)
Incoming Outgoing
Route Policies Route Policies
BGP
Incoming Outgoing
Adj-RIBs-In Decision Loc-RIB Adj-RIBs-Out
Updates Updates
Process
Locally Injected
Routes
Routing Table

BGP Overview Eldora#show processes cpu | include BGP
225 636 335 1898 0.08% 0.11% 0.12% 0 BGP Router
226 120 75 1600 0.00% 0.04% 0.02% 0 BGP I/O
The IOS BGP Process 227
228
92
0
10
1
9200 0.00% 0.07% 0.02% 0 BGP Scanner
0 0.00% 0.00% 0.00% 0 BGP Event
Eldora#
Incoming Outgoing
BGP BGP
Updates Updates
TCP BGP BGP BGP BGP BGP TCP
InQ OutQ
Socket I/O Router Table Router I/O Socket
BGP BGP
Router Scanner
Routing
Table
2014 Cisco and/or its affiliates. All rights reserved.
(RIB)
19
BGP Overview
Stuff We Dont Have Time to Cover
Neighbor states
Differences in EBGP and IBGP rules
BGP decision process
Policy configuration
Scaling tools
Route Reflectors
Confederations

BGP Challenges
Failure Detection
Default keepalive interval: 60 seconds
Default hold time: 180 seconds
Neighbor declared dead only after hold time expires
Aggressive keepalives and hold times can cause false flapping

BGP Challenges
Slow Convergence

BGP Challenges
Slow Convergence

BGP Challenges
Slow Convergence
X 1. Failure detected
2. Invalid routes deleted (BGP table, FIB, RIB)
3. Alternate routes (if any) installed

BGP Challenges
Slow Convergence
4. Update(s) generated and sent, withdrawing effected routes

BGP Challenges
Slow Convergence
5. Update(s) processed, withdrawn routes removed

6. Alternate routes (if any) evaluated, new best paths installed

BGP Challenges
Slow Convergence


and advertising new best paths

BGP Challenges
Slow Convergence


and advertising new best paths
8. Repeat.

BGP Challenges
Route Storage and Processing
Source: www.potaroo.net

route-views>show ip bgp summary
[Information removed]
BGP Challenges Neighbor
12.0.1.63
V
4.69.184.193 4
4
AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
3356 1594049 16329 8730261 0 0 1w3d
7018 2444534 11750 8730261 0 0 1w3d
497768
499387
Route Storage and Processing 66.59.190.221 4

66.110.0.86 4
6539 658504 18212 8730261 0 0 1w3d
6453 932399 11758 8730261 0 0 1w3d
503979
499327
66.110.0.124 4 6453 0 0 1 0 0 never Idle
66.185.128.48 4 1668 2019709 34221 8730261 0 0 1w3d 496950
69.31.111.244 4 4436 1528658 27161 8730261 0 0 1w1d 500482
89.149.178.10 4 3257 808553 2996 8730261 0 0 1w2d 499700
114.31.199.1 4 4826 1055694 18230 8730261 0 0 1w3d 508501
128.223.253.10 4 3582 1040285 34215 8730261 0 0 1w3d 501209
129.250.0.11 4 2914 1755917 32286 8730261 0 0 1w3d 499711
134.222.87.1 4 286 3930199 14884 8730261 0 0 1w2d 500985
144.228.241.130 4 1239 319848 5412 8730261 0 0 5d15h 498997
154.11.11.113 4 852 0 0 1 0 0 never Active
154.11.98.225 4 852 1368774 16301 8730261 0 0 1w3d 503369
157.130.10.233 4 701 814025 32262 8730261 0 0 1w3d 498385
162.250.137.254 4 4901 579351 32271 8730261 0 0 1w3d 500811
164.128.32.11 4 3303 603220 9836 8730261 0 0 1w3d 443763
192.203.116.253 4 22388 105312 9844 8730261 0 0 1w3d 14454
193.0.0.56 4 3333 2726844 32254 8730261 0 0 1w3d 506291
194.85.40.15 4 3267 1295694 16286 8730261 0 0 1w3d 508215
194.85.102.33 4 3277 1309804 9841 8730261 0 0 1w3d 511982
195.66.232.239 4 5459 865295 9838 8730261 0 0 1w3d 420027
196.7.106.245 4 2905 10531 3968 8730261 0 0 2d11h 3761
202.232.0.2 4 2497 1065618 9839 8730261 0 0 1w3d 501413
202.249.2.86 4 7500 410442 7578 8730261 0 0 4d18h 508870
Source: www.potaroo.net 203.62.252.186 4 1221 591747 9829 8730261 0 0 1w3d 501338
203.181.248.168 4 7660 1051042 2993 8730261 0 0 1w2d 506877
206.24.210.80 4 3561 944009 5409 8730261 0 0 5d15h 499857
207.46.32.34 4 8075 0 0 1 0 0 never Active
207.172.6.1 4 6079 1039070 16270 8730261 0 0 1w3d 499438
207.172.6.20 4 6079 1042677 16282 8730261 0 0 1w3d 499446
208.51.134.254 4 3549 1035057 14823 8730261 0 0 1w2d 500149
208.74.64.40 4 19214 1038983 16289 8730261 0 0 1w3d 500582
209.124.176.223 4 101 3626021 32214 8730261 0 0 1w3d 506792
216.218.252.164 4 6939 2588621 16274 8730261 0 0 1w3d 504974
217.75.96.60 4 16150 1080296 9828 8730261 0 0 1w3d 499310
route-views>
Source: University of Oregon Route Views Project

BGP Challenges
Configuration Management
BGP and policy configurations can be huge
Session and policy configurations are often repetitive
Or at least share common characteristics
Processing policy configurations can be CPU intensive

What Do We Mean By BGP Optimization?
Faster failure detection

Faster convergence
Better local performance
More efficient configuration management

Improving Failure Detection and
Convergence
Changing BGP Hold Time
Default keepalive interval: 60 seconds

Default hold time: 180 seconds
If hold time is reduced, reduce keepalive interval accordingly
Overly aggressive timers can cause mistaken flaps
BGP busy with other things
Keepalives, hold times processed in software
timers bgp keepalive holdtime [min-holdtime]

Fast External Failover
Tears down EBGP session as soon as link to neighbor fails

Enabled by default
Fast failover flapping links can cause BGP instabilities
Global configuration:
bgp fast-external-failover
Interface configuration (for more granularity):

ip bgp fast-external-failover [permit | deny]

Bidirectional Forwarding Detection (BFD)
Very fast failure detection

Processed in hardware
Independent of media types and encapsulations
BFD can be shared by multiple protocols
Interface configuration:
bfd interval milliseconds min_rx milliseconds multiplier interval-multiplier
BGP configuration:
neighbor ip-address fall-over bfd

Next-Hop Tracking (NHT)
Event-driven removal of BGP prefixes from RIB
Improves IBGP convergence time
IGP changes in next-hop addresses scanned quickly
Default trigger time: 5 seconds
Does not wait for BGP Scanner
Does not wait for hold timer
BGP routes with no next-hop are removed
Enabled by default
bgp nexthop {trigger delay seconds | route-map map-name}

Prefix-Independent Convergence (PIC)
Can reduce reconvergence to sub-second for individual prefixes
Assuming there is another path available
Similar concept to EIGRP feasible routes
Alternate path pre-installed in FIB
bgp additional-paths select best-external

TCP Transport
TCP port 179
Default TCP Maximum Segment Size (MSS) = 536 octets
Path MTU Discovery can negotiate larger MSS sizes
More efficient packing of Update messages
PMTU enabled by default in modern IOS
bgp transport path-mtu-discovery

Improving BGP Performance
Eldora#debug ip bgp events
BGP Scan Timer BGP events debugging is on

Eldora#
*Mar 9 07:33:20.943: BGP: Performing BGP general scanning
*Mar 9 07:33:20.947: BGP(0): scanning IPv4 Unicast routing tables
*Mar 9 07:33:20.947: BGP(IPv4 Unicast): Performing BGP Nexthop scanning for general scan
*Mar 9 07:33:20.951: BGP(0): Future scanner version: 74, current scanner version: 73
.
.
.
*Mar 9 07:33:20.959: BGP(NSAP Unicast): Performing BGP Nexthop scanning for general scan
*Mar 9 07:33:35.959: BGP: Import timer expired. Walking from 1 to 1
.
.
.

BGP Scan Timer
Scan time is time between scans

Scan of large tables can take tens of seconds
Reducing scan time:
Can increase BGP performance
But will increase CPU utilization
Scan is low priority process, should not interfere with other processes
bgp scan-time scanner-interval

BGP Advertisement Interval
Minimum route advertisement interval (MRAI) between Update messages

Allows time for multiple route changes to be collected into single Update
Defaults:
EBGP = 30 seconds
IBGP = 0 seconds (Update sent immediately upon change)
Reducing MRAI:
Decreases convergence time
Decreases BGP performance (efficiency)
neighbor ip-address advertisement-interval seconds

Controlling Table Size
Accept partial tables instead of full tables

Default route for prefixes not accepted
Advertised or locally configured Remember to account for AS_PATH
prepending
AS_PATH filters
Community filters Good guide for major ISPs
Community policies:
Example: http://onesc.net/communities/
Accept upstream peers customer routes
Accept upstream peers partner routes
Accept upstream peers locally generated routes Use distant next-hops for default
Default route to everything else routes to insure reachability
Loss of route information increases chances of sub-optimal routing

Outbound Route Filters
BGP Updates carrying thousands of prefixes can take time, bandwidth

Often, many or most prefixes are dropped on reception
Outbound Route Filtering (ORF):
Allows a router to tell a neighbor what incoming prefixes it will filter
Neighbor can then suppress transmission of prefixes that will be dropped on
reception
Caution: Configuring ORF causes a session reset
neighbor ip-address capability orf prefix-list [receive | send | both ]

BGP Route Dampening
Route
2250 Suppressed
Suppress Threshold
2000
1750
POINTS
1500
1250 Flap
1000
Reuse Threshold
750
500
Flap Suppression
250 Removed
Flap
0
5 10 15 20 25 30 35 40 45 50 55 60
MINUTES
Route Dampening Precautions
Current best practice recommends not using route damping
Effects of dampening can cascade through network
Un-intuitively, dampening can cause more instabilities than it solves
Default dampening parameters vary widely among vendors
Change defaults to match in multi-vendor networks
Dampening defaults are generally too aggressive
Change defaults to be more tolerant

Peer Groups
A Little History
Configuration scaling
Group peers sharing the same policies and session parameters
Changes can be performed just once for entire group
BGP performance (legacy)
Original motivation for peer groups
Without peer groups:
Router scans routing table separately for each peer
150,000 routes + 30 peers = 4,500,000 NLRI updates generated
With peer groups:
Each peer group represents an Update Group
Router scans routing table once for each peer group
150,000 routes + 30 peers = 150,000 NLRI updates generated
Dynamic peer groups eliminate this driver

Dynamic Update Groups
Why Peer Groups Are Obsolete
Introduced in IOS 12.0(24)S
Automatically groups neighbors sharing the same outbound policies
Eliminates need for peer groups to improve performance
Eliminates need for many small peer groups when outbound policies vary slightly
No configuration, built into IOS
Session and policy templates are a better way to scale configuration

Improving BGP Management
Session and Policy Templates
Improvement over Peer Groups

Dynamic Update Groups Performance
Policy and Session Templates Reduced configuration size
Peer Groups and Session / Policy Templates are mutually exclusive
Session templates for shared session parameters
Policy templates for shared policies
One session template and one policy template per neighbor
Inheritance enables nested templates

Route Refresh
Eliminates:
Need to reset BGP session
Need to store incoming prefixes
BGP Route Refresh message
Sent to neighbor, to request re-send of prefixes
Route Refresh capability negotiated during session establishment
Neighbors must understand and support Route Refresh messages
If neighbor supports Route Refresh capability:
clear ip bgp {* | as-number | ip-address | peer-group-name} in

Conclusion
Conclusions
BGP is effective but notoriously slow

This is not always a bad thing
Dont expect BGP to behave similarly to an IGP
Theyre built for different purposes
Tweaking timers can sometimes backfire
A good BGP implementation will have built-in optimizations
Controlling the information can be as effective as controlling the process
Know your routers limits
CPU
Memory

BGP Optimization

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

BGP Optimization

Загружено:

Авторское право:

Доступные форматы

CP-1007 Optimizing BGP

2014 Cisco and/or its affiliates. All rights reserved. 3

2014 Cisco and/or its affiliates. All rights reserved. 5

AS1 AS6 AS3

AS5 AS2 AS4

2014 Cisco and/or its affiliates. All rights reserved. 6

AS5 AS2 AS4

2014 Cisco and/or its affiliates. All rights reserved. 7

AS1 AS6 218.54.0.0/16 AS3

2014 Cisco and/or its affiliates. All rights reserved. 8

AS1 AS6 AS3

AS5 AS2 218.54.0.0/16 AS4

2014 Cisco and/or its affiliates. All rights reserved. 9

AS1 AS6 AS3

AS5 AS2 AS4

2014 Cisco and/or its affiliates. All rights reserved. 10

AS1 AS6 AS3

AS5 AS2 AS4

2014 Cisco and/or its affiliates. All rights reserved. 11

2014 Cisco and/or its affiliates. All rights reserved. 12

2014 Cisco and/or its affiliates. All rights reserved. 13

2014 Cisco and/or its affiliates. All rights reserved. 14

2014 Cisco and/or its affiliates. All rights reserved. 15

2014 Cisco and/or its affiliates. All rights reserved. 16

2014 Cisco and/or its affiliates. All rights reserved. 17

2014 Cisco and/or its affiliates. All rights reserved. 18

2014 Cisco and/or its affiliates. All rights reserved. 20

2014 Cisco and/or its affiliates. All rights reserved. 21

2014 Cisco and/or its affiliates. All rights reserved. 22

2014 Cisco and/or its affiliates. All rights reserved. 23

2014 Cisco and/or its affiliates. All rights reserved. 24

4. Update(s) generated and sent, withdrawing effected routes

2014 Cisco and/or its affiliates. All rights reserved. 25

4. Update(s) generated and sent, withdrawing effected routes

5. Update(s) processed, withdrawn routes removed

2014 Cisco and/or its affiliates. All rights reserved. 26

4. Update(s) generated and sent, withdrawing effected routes

5. Update(s) processed, withdrawn routes removed

7. Update(s) generated and sent, withdrawing effected routes

2014 Cisco and/or its affiliates. All rights reserved. 27

4. Update(s) generated and sent, withdrawing effected routes

5. Update(s) processed, withdrawn routes removed

7. Update(s) generated and sent, withdrawing effected routes

2014 Cisco and/or its affiliates. All rights reserved. 28

2014 Cisco and/or its affiliates. All rights reserved. 29

BGP Challenges Neighbor

Route Storage and Processing 66.59.190.221 4

Source: University of Oregon Route Views Project

2014 Cisco and/or its affiliates. All rights reserved. 31

Faster failure detection

2014 Cisco and/or its affiliates. All rights reserved. 32

Default keepalive interval: 60 seconds

timers bgp keepalive holdtime [min-holdtime]

2014 Cisco and/or its affiliates. All rights reserved. 34

Tears down EBGP session as soon as link to neighbor fails

Interface configuration (for more granularity):

2014 Cisco and/or its affiliates. All rights reserved. 35

Very fast failure detection

2014 Cisco and/or its affiliates. All rights reserved. 36

bgp nexthop {trigger delay seconds | route-map map-name}

2014 Cisco and/or its affiliates. All rights reserved. 37

bgp additional-paths select best-external

2014 Cisco and/or its affiliates. All rights reserved. 38

bgp transport path-mtu-discovery