IP MPLS Virtual Private Networks

Presented by:

Chris Chase

MPLS Concept

Outgrowth of IP Switching (e.g., MPOA, Epsilons IP Switching,

Ciscos tag switching)

Key concept:

Separate routing (the selection of paths through the network) from forwarding/switching plus an abstraction of aggregation

Non-MPLS Routing
Hierarchical topology - edge and backbone routers Forward packet - lookup route at each hop
CR

PER

PER BFR BFR PER

A.1
PER

CR
PER

CR

BFR PER

CR PER

PER

Provider Router Network

BFR - Big Fast Router PER - Provider Edge Router CR - Customer Router

Packet forwarded by hop-byhop route lookup

Routes chosen using OSPF interior routing protocols

3

Routing with MPLS

Interior routes are assigned Labels that identify a connection/path Called a Label Switched Path (LSP) instead of a PVC

CR

PER LSR LSR PER CR

PER

PER

PER

CR
PER CR

LSR PER PER

LSR - Label Switch Router PER - Provider Edge Router CR - Customer Router

LSP: Route lookup once and associated label assigned to packet

Routes chosen using OSPF interior routing protocols

Routing with MPLS

Interior routes are assigned Labels that identify a connection/path Called a Label Switched Path (LSP) instead of a PVC
A.1
CR

PER LSR

PER LSR

A.1
PER CR

PER

PER CR PER CR

LSR PER

LSR - Label Switch Router PER - Provider Edge Router CR - Customer Router

PER

LSP: Route lookup once and associated label assigned to packet

Routes chosen using OSPF interior routing protocols

Routing with MPLS

Traffic Engineering can use alternative to the IGP shortest path
CR PER PER

LSR
LSR PER CR PER CR PER CR

PER

LSR
PER

LSR - Label Switch Router PER - Provider Edge Router CR - Customer Router

PER

LSP: Route lookup once and associated label assigned to packet

Routes chosen using OSPF interior routing protocols

Routing with MPLS

Interior routes are assigned Labels that identify a connection/path. Called a Label Switched Path (LSP) instead of a PVC.
CR PER LSR LSR PER PER

CR PER

CR
PER CR

LSR PER

LSR - Label Switch Router PER - Provider Edge Router CR - Customer Router

PER

LSP: Route lookup once and associated label assigned to packet

Routes chosen using OSPF interior routing protocols

7

MPLS: Decouples routing and forwarding

IP packet header only examined at ingress PER Hierarchy of routing/Label Stacking
VPN & Scale

Interior knows nothing about external addresses or routes

Only needs to know how to get between edges (PERs)

Explicit routing in IPv4/v6 is expensive Use explicit route for LSP instead of OSPF route

Enables very efficient explicit routing

Traffic Engineering

History
IP cut through switching
Improve performance and provide QoS to IP
Multiprotocol over ATM (MPOA) Epsilons IP Switching

Ascends IP Navigator
Ciscos tag switching IBMs Aris

1997
Needed alternative to SVC service for FR and ATM
Provider based IP VPN concept conceived

MPLS work initiated at IETF

A technology solution looking for a problem

Killer Applications of MPLS

IP VPNs
Provider Based, Simple, scalable, layer 2 security Overlapping, private addressing plans

Layer 2 VPNs FR, Ethernet, Circuit services Traffic Engineering

Deliver service guarantees similar to FR/ATM Fast reroute

Hierarchical Networks Carriers carrier Universal control plane

Label = Optical (Lambda), Sonet/TDM, Spatial (ports/conduits)

GMPLS and Optical UNI

* Not really an advantage: Performance
10

The Basics

11

Generic MPLS Encapsulation

Layer 2 Header | PID MPLS Label 1 MPLS Label 2

MPLS Label n

Layer 3 Packet

Label (20bits) | CoS (3 bits) | Stack (1 bit) | TTL (8 bits)

MPLS does not define a link layer protocol no framing provided A shim header between link and network protocol New LLC and PID defined for Ethernet, PPP, ATM, FR to carry label Can stack tags/labels. Stack bit indicates end of stack. There is no protocol ID field to indicate type of encapsulated packet. Protocol of encapsulated packet is implied by the label
Indicated when the label is signaled (next slide)
12

Forwarding Equivalence Class (FEC) and Hierarchy

FEC = All packets with the same forwarding requirements
i.e., same path, same QoS (policing, scheduling, discard)
COS bits can modify packet handling

Many different FEC types:

IPv4, IPv6, FR, ATM, Ethernet VLAN

FEC label all packets in this class get the same label

Can stack labels (end of stack bit)

Hierarchy of equivalence classes
Hierarchy of routing VPNs L3 and L2 Traffic engineering
13

Multi-protocol
Forwarding/Switching is content agnostic
Can carry IP, FR, ATM, Ethernet, anything Label represents base common treatment shared by all packets with that label (FEC) Control Plane (Routing and signaling) is content agnostic IP control plane
Routing OSPF, IS-IS, BGP, PIM

Signaling LDP, CR-LDP, RSVP-TE, BGP+ext, PIM+ext CoS Diff-serv

Many Layer 2 technologies, e.g, FR and ATM, have been

fitted to MPLS MPLS is not ATM But ATM switches can be MPLS switches

14

Standards
IETF
First RFCs
2702 (TE reqs), 3031 (arch), 3032 (stack encoding), 3034 (FR), 3035 (ATM VC), 3036 (LDP), 2547 (VPN), 3107 (BGP), etc.

Drafts: GMPLS, BGP, Multicast, Fast Recovery, L2 VPNs, http://www.ietf.org/html.charters/mpls-charter.html L3 VPN:

http://www.ietf.org/html.charters/ppvpn-charter.html

draft-ietf-ppvpn-rfc2547bis-04.txt

L2 VPN
http://www.ietf.org/html.charters/pwe3-charter.html

Additional ITU work, MPLS and ATM Forum

15

Layer 3 MPLS VPN The Next Generation IP WAN

Based on 2547 draft Another tool in the WAN toolbox for the network architect

16

Traditional Point-to-Point WANs

Rely on a hub architecture
H

C C C C

C
17

Dual Star - Redundancy

H H

C
C C C

18

Aggregation/Distribution Layer Scaling through hierarchy

H H H

C C C C
19

Domains of Enterprise WANs

Private lines FR/ATM VC
Private line replacement Hub-and-spoke Very reliable, trusted, common Site-to-site Internet VPN (i.e., IPSEC tunnels) Point-to-point topologies (typically hub-and-spoke) Extranets (also SSL) Remote access Footprint Outsourced versus do-it-yourself L3 MPLS VPN
Layer 3 IP routing outsourced to carrier

Following slides

They complement each other

20

L3 MPLS VPN 2547 style

Provider-based VPN
Vis--vis CPE-based tunneling VPN, e.g., L2TP with IPSEC Others: Virtual router VPNs, Layer 2 MPLS VPNs

IP MPLS VPN defined as a set of interfaces

Interface: PPP, FR/ATM VC, Ethernet Vlan, LT2P

VPN membership assigned when provisioned

Customer interface: standard IP, no MPLS VPN appears as an Autonomous System (AS)
Customer router peers with this AS - a transit only AS in between customers sites
Private - separated from other VPNs

Like having your own little Internet

MPLS VPN Layer 3 IP Architecture

MPLS Network
PER = Provider Edge Router CER = Customer Edge Router LSR = Label Switch Router

LSR

LSR LSR

OSPF
PER IBGP CER PER

BGP or other protocol or Static Routes

Access IP serial link Encapsulated in PPP or FR/ATM PVC or Ethernet CER

MPLS VPN Value Adds vs. Other VPNs

Any to Any IP Connectivity
Optimal Routing without SVCs
Improved delay by avoiding tandem routing through a hub Offload hub router

Any IP address scheme - Intranets and extranets Circuit Consolidation eliminate aggregation layer Diversity via IP routing - simplified DRO Ease of network expansion Access technology agnostic FR, ATM, PPP over DS0-OC48; Ethernet IP Class of Service Provider-based IP VPN

No CPE-based tunneling and encryption equipment/software nor PKI management.

23

Combined Services
MPLS VPN and Edge VPN (IP-VPN)

CER Internet Cable, DSL,dial

FW
CER IP MPLS VPN Service GW
Access: FR/ATM/DSL Ethernet/P-L PPP

Generic GW
Remote Access Network

CER

International MPLS VPN CER

dial
24

IPSEC tunnel L2TP (optionally IPSEC)

From VPN toward customer

A CE5 PE PE

Load Balancing:

MPLS VPN

Link1, BGP

CE1
Cust site Network A

CE6

Link2, BGP

PE
PE CE3

CE2

CE4
Pt-to-pt link
All flows from remote CEs (3-6) matching route A will load balance across Links 1 and 2 (even from CE4). Note: the load balancing decision is made (using MPLS) at the ingress to the network.

25

Outbound Route Filtering (ORF)

Allowing a CER to communicate route filter to PER
Dynamically transferred through BGP
PER=Provider Edge Router CER=Customer Edge Router

Inbound Permit 0/0 Deny all

Out bound Permit 0/0 Deny all

AC=Access Connection BGP=Border Gateway Protocol MPLS=Multi Protocol Label Switching

AC
CER PER

MPLS

BGP
BGP Route Refresh message carries any inbound prefix-based filter

Any inbound prefix based filter is applied as out bound filter to PER
26

Class of Service Concepts

The ability for user to differentiate traffic
A provider could differentiate in many ways:
Isolation - keep traffic in different classes from unfairly impacting each other Performance

Service

Bandwidth Delay Discard

Availability Support As opposed to the marketing/management view around perception

Network engineer view as a toolset to manage traffic

27

IP Header Class of Service Marking

old
new
0
Version IHL

Prec 3b | D | T | R | x | x

DSCP 6b | x | x
8
Type of Service/

16
Total length

24

32

Diffserv codepoint
Identification TTL Protocol Flags Fragment Offset

Header Checksum Source Address Destination Address

28

CoS via IP Packet Marking

CE classifies traffic per packet via marking
IP Diffserv Codepoints (Precedence bits) Marking interpreted

Separate queuing per class Per class resource scheduling, e.g.,

Priority queuing Bandwidth scheduling (WFQ)

Drop differentiation
Packets marked discard eligible above class bandwidth Transmitted when not congested

29

VPN CoS
Classification (application/policy level), Session control

Gatekeeper
Per Class Policing

Class Servicing

LSR CER PER

PQ Bursty

LSR PER
LSR
PQ Bursty

CER

CER

Port MPLS LSP Policer Session Control (H.323, SIP) Interface Trunk
30

CoS Marking Transparency Using MPLS

Users dont want packet markings to change
And some older systems TCP breaks

Provider can indicate reclassification by marking label

instead of remarking IP packet

IP, DSCP=AF11|

IP, DSCP=AF11|

Label | CoS=AF11 | ...

Policer
IP, DSCP=AF11| Label | CoS=AF12 | ...
31

RFC2547 Constrained Route Distribution

Route Targets (RT) are used to constrain connectivity
Keeps VPNs separate Creates topology within VPN Concept of Hub and Spoke route policies used as building blocks Hub and Spokes have a certain RT import and export list Hub sites can see other hub and spoke routes Spokes see only hub routes Combine/compose to create VPN topology Union of multiple hubs and spokes create arbitrary topologies These slides dont show the explicit RT import/export lists Hub types are shown as Hi and spoke types as Si

32

Any-to-Any Topology

H0

VPN
H0

H0

Hub and Spoke Topology

S2 S2 H2

S1

VPN
H1

S1

H1

Hi = hub interfaces, Si = spoke interfaces. The term hub and spoke just refers to how routes are constrained.

Si can only exchange routes with Hi. Hi exchanges routes with all Hi and with Si. Specifically in terms of route targets (RTs), Hi exports RT_Hi and imports {RT_Hi, RT_Si}, while Si exports RT_Si and imports RT_Hi.
34

Hub and Spoke Topology

S2 S2 H2

S1

VPN
H1

S1

H1

Here we combine connectivity policies. Using H0, all hubs talk to each other. By taking such unions of policies completely arbitrary bi-directional connectivity graphs can be realized (in fact completely arbitrary uni-directional graphs could be achieved which might be applicable for something like a firewall).
35

Note RTs only Constrain Routes

This does not use access lists that filter packets!
BGP MPLS VPN technology constrains route distribution (i.e., connectivity), they do not require per packet manipulation (which does not scale nor manage well). Packets always follow routes (in the reverse direction of route flow)

But constraint of a specific route is not sufficient to constrain the reachability of a destination matching the route!
Overlapping routes (e.g., aggregates or defaults) can cause problems.

36

Tradeoffs of L3 MPLS VPN

For all the IP values-adds the drawbacks are:
Have to route with provider!
Troubleshooting is more difficult than L2 WANs L2 has clear demarc connection up or down. Convergence is slower Route changes have to propagate through provider routers Certain routing problems are more difficult to solve Some problems are more easily solved with direct topology manipulation E.g., hub connectivity based on source Peering model rather than flat model a different paradigm

Only IP: no IPX, SNA, DECNET, Appletalk

Have to tunnel

Technology not as mature

37

Customer Support
Layer 2 services are easier to support
A customer doesnt call if his FR-connected routers arent seeing the same set of routes

Layer 3 VPN
Customer calls I cant see my route. Help me troubleshoot my network.

Customer visible provider-based tools can help sectionalize and show customer whether there is a problem with provider network without getting a technician on the line.
38

Comments about CER-PER Protocol

2547 VPNs are a peering architecture!
Static
Stable, but fill out order form Only can detect local link failure

BGP
Many policies; geared towards multihoming; peering
Load balancing and ORF

OSPF
Changes intra-area to inter-area backdoor always preferred

EIGRP
Proprietary Without ability to summarize need ability to avoid going active CER acts as stub can loop (count to infinity) without new feature
39

How MPLS and 2547 VPNs Work

Quick BGP intro LDP operation 2547 VPNs
Follow the VPN route and label Follow the packet

40

BGP Basics
TCP Connection

R1
RIB ----

W| s,r

A| z

R2
RIB ----

BGP - fairly simple protocol

Uses TCP for reliable delivery Distributes appearance/change and withdraw/disappearance of routes in route table
Routing Information Base (RIB) = BGP route table

eBGP = between AS iBGP = within AS AS_PATH list of where route has been Next hop Other attributes are about policy, i.e., which route is best
41

Route Reflector scaling IBGP among AS edges

CE PE

IBGP client

PE

RR
CE PE PE CE
IBGP

RR
PE PE
EBGP

CE

CE

42

Route Reflector
Updates in RIB from inbound peer type are sent to outbound peer type in table below Client is a special kind of IBGP neighbor
Any BGP router with a neighbor designated as a client is a route reflector
Outbound Inbound EBGP IBGP Client

EBGP
IBGP Client

X
X X

X
X

X
43

A Label Switched Path LSP

The downstream node assigns label
POP! SWAP! SWAP! PUSH!

data

417 data

666 data

233 data

data

A label switched path

tail end Often called an MPLS tunnel: payload headers are not Inspected inside of an LSP.
44

head end

LSPs are Unidirectional

Destination FEC based
Cant distinguish upstream sender LSPs merge Results in multipoint-to-point LSPs
565 IP2

IP2

417 IP2

912 IP2

IP2

IP1

417 IP1

823 IP1

233 IP1

IP1

LSP merge
45

Penultimate Hop Popping

POP + IP Lookup SWAP SWAP PUSH

IP

417 IP

666 IP

233 IP

IP

Look up the label pop + look up header underneath

Why even bother sending a label?
IP Lookup POP SWAP PUSH

IP

IP

666 IP

233 IP

IP
46

Follow the Route and Follow the Packet

MPLS VPN Cloud LSR3

LSR2 LSR1

PER1

PER2

CR1

Network Z CR2

Site 1
CR1 at Site 1 has a packet addressed to a host in network Z at Site 2. How does it get there?

Site 2
47

Label Distribution in Interior

For links configured for label switching

Node sends out periodic UDP Hello all router subnet broadcast on well-know LDP UDP port Contains IP address for desired LDP session and desired label space Creates a TCP-based LDP session to any node answering Hello Session is initiated by node with lower advertised IP address Node picks a local label for each route in its routing table and advertises this to everyone Called downstream, unsolicited and independent control mode The upstream node only installs into its label forwarding table where the downstream node is the next hop for the route (FEC) Note in this mode if there is a reroute that changes the next hop the label is just rebound locally no signaling upstream Much faster reroute compared to ATM, but local label assignment does not guarantee LSP is in place ATM-LSRs work differently

Each node advertises routes (FEC) and labels to all LDP peers

48

LSP Setup for OSPF Route to PER2

IPFR Cloud L4 L2 LSR3 L2 pop LSR2

L1 L2 LSR1 PER2 L1 PER1

PER2

CR1 CR2

Li - labels requested via LDP from next hop neighbor for each routing table entry LSP for the OSPF route to reach PER2
49

How MPLS VPNs Work

1) Follow the routes Each VPN on a PER has a private routing table
Called a Virtual Routing Forwarding (vrf) table

vrf is assigned attributes that are unique to the VPN

Route Targets (RT) - attached to VPN routes.

only vrfs with common RTs share routes with each other

Route Distinguishers (RD) - appended to routes to ensure uniqueness even if VPNs have overlapping address spaces

Creates a new address family called vpnv4 = RD+ipv4

Note: RTs and RDs are not applied to packets

2) Follow the packet A stack of two labels is used to forward the packet on the interior LSP and then external interface
50

VPN extensions
Route Target (RT)
BGP 64 bit extended community value First 16bit identify as RT type. Other 48 bit is variable
Conventional format ASN:X, i.e., 16b:32b

Route Distinguisher
64 bit, first 16 identify RD type
48 bit selectable with format convention ASN:X, i.e., 16b:32b

51

Distributing Customer Routes

IPFR Cloud

LSR3

PER2 learns Rt Z via BGP or is statically configured with Rt Z.

LSR2 LSR1

PER1

PER2

Network Z
CR1

Li - labels LSP

LNK2 data: vrf1 vrf1: RT1, RD1 table: Rt Z LNK2

CR2

52

Customer Routes Distributed via IBGP with Label IPFR

Cloud LSR3

LSR2 LSR1

PER1

PER2

IBGP msg
CR1
RD1+Z, L4, RT1, PER2

Network Z

Li - labels LSP

LNK2 data: vrf1 vrf1: RT1, RD1 table: Rt Z L4,CR2,LNK2

CR2

53

Only vrfs with Matching RTs Import Route

IPFR Cloud LSR3

LNK1 data: vrf1 vrf1: RT1, RD2 table: Rt Z L4, PER2 PER2 L1, LSR1
CR1

LSR2 LSR1

PER1

PER2 Network Z

Li - labels LSP

LNK2 data: vrf1 vrf1: RT1, RD1 table: Rt Z L4,CR2,LNK2

CR2

54

Purpose of BGP Label

Indicates which vrf and optionally which interface on
the egress PER ways:

Locally, the egress PER will treat labels in two possible

Non-aggregate label is associated with an external route
Will be switched directly to an outgoing interface IP header is not examined

Aggregate label is associated with a locally originated or directly connected route

Packet will be looked up in the vrf context

55

CR1 learns RT Z via BGP (or statically configured)

IPFR Cloud
LSR3

LNK1 data: vrf1 vrf1: RT1, RD2 table: Rt Z L4, PER2 PER2 L1, LSR1
CR1

LSR2 LSR1

PER1

PER2
Network Z

table: Rt Z PER1

Li - labels LSP

LNK2 data: vrf1 vrf1: RT1, RD1 table: Rt Z L4,CR2,LNK2

CR2

56

Packet for Rt Z forwarded by CR1

IPFR Cloud LSR3

LNK1 data: vrf1 vrf1: RT1, RD1 table: Rt Z L4, PER2 PER2 L1, LSR1

LSR2 LSR1

PER1

PER2 Route Z

Z| packet
CR1 table: Rt Z PER1,LNK1

Li - labels LSP

LNK2 data: vrf1 vrf1: RT1, RD1 table: Rt Z L4,CR2,LNK2

CR2

57

Top label is label-switched through interior

IPFR Cloud LSR3

LNK1 data: vrf1 vrf1: RT1, RD1 table: Rt Z L4, PER2 PER2 L1, LSR1
CR1

L1 L2 LSR1

L2 pop LSR2

PER1

L1|L4|Z| packet

PER2 Route Z

Li - labels LSP

LNK2 data: vrf1 vrf1: RT1, RD1 table: Rt Z L4,CR2,LNK2

CR2

58

Top label popped at end of LSP

IPFR Cloud LSR3

LSR2 LSR1

PER1

L4|Z| packet
PER2 Route Z

CR1 LNK2 data: vrf1 vrf1: RT1, RD1 table: Rt Z L4,CR2,LNK2

CR2

Li - labels LSP

59

Inner label determines egress interface and then is popped.

IPFR Cloud
LSR3

LSR2 LSR1

PER1

Z|packet
PER2 Route Z

CR1 LNK2 data: vrf1 vrf1: RT1, RD1 table: Rt Z L4,CR2,LNK2 CR2

Li - labels LSP

60

MPLS in Core Not Needed

MPLS for IGP domain serves as a tunneling method
among PERs

Could use other tunneling methods Advantages to MPLS:

Full mesh of LSP tunnels automatically created Can use MPLS TE

Internet draft to use IP or GRE tunneling

Automatically (treat vpnv4 BGP next hop as a recursive encapsulation)

61

MPLS VPN Security

There is a private routing table for each VPN (vrf) VPN membership Identity associated with each access
connection VPN membership is not determined by IP header, only by interface (e.g., DLCI, VPI/VCI, PPP, VLAN tag). Label and RT for VPN attached to routes advertised for interface. Route and its matching label are only imported by routing tables that match the VPN RT. Impossible for a packet on a PVC in one vrf to spoof its way or jump into another vrf

62

MPLS VPN Security

Requires correct provisioning of connections and RTs
Same as FR/ATM security
Given correct provisioning it is impossible for packet to jump from one PVC to another PVC

If you dont need encryption on FR/ATM/P-L then dont need it here

If you would encrypt over FR/ATM/P-L then you would also encrypt here

63

MPLS VPN Scale for the Carrier

Can the MPLS VPN technology scale to meet the size of
the market?

Can it be managed at scale? Does this have anything to do with the Internet?

64

All Those Routes

Can be a lot of routes in the service
The aggregate over all VPNs There is no summarization of routes!

BUT
No VPN state in backbone LSRs, only in PERs. PER only holds routes for VPNs touching it. Route Reflectors (RR) only handle VPNs they touch

65

Large Corporate Intranet vs Internet

Intranet = Private Corporate Network
aka VPN P/L or FR/ATM VCs
Recent survey - total number of U.S. enterprises using FR at ~35,000

Tens to thousands of sites

95% T1

Internet corporate access from Intranet

10 Corporate gateways T1

66

Modeling Large Enterprise VPNs

Based on customer observations and business nature of
large corps

Large # sites, N, for N=100 10,000 #routes = K*N + C

K = 2, 3, 10
one route always for CER-PER link

Largest FR/ATM customer N ~20,000 sites 95% < T1 BW utilization < 25%

67

Constraints
User Plane forwarding (pps)
vrf/MPLS cost is small Control Plane
Interior (IGP) component is independent of MPLS/VPN

Space (memory)
vrf and BGP session overhead small compared to route space

Signaling/Routing (CPU)
most important in transient situations (e.g., link failures)

Public resources
e.g, registered IPv4 addresses, RTs among partners OSS

68

Divide and Conquer

How to keep dimensions within constraints?
(i) State reduction (ii) Partition

(iii) Distribute
Forwarding Control plane

69

State Reduction
Route Summarization
In middle of customer network not really an option, maybe greenfield intranets

Limit the routes in a VPN

Keep # allowed routes commensurate with number of interfaces purchased
vrf route limit

Use carriers carrier

Providers, non-enterprise, that have lots of routes, few ports

70

Partitioning
Limit VPNs touching a PER
To avoid poor PER utilization aggregate and fan-out
Groom up interfaces rather than push PER toward CE Fan out across PERs in a POP

Limit VPNs touching RR

(i) Via RTs and ORF requires RT assignment strategy (ii) Via communities (iii) Via PER-RR mutually exclusive VPN subsets

PERs and RRs only need to handle the largest

enterprise customer VPN

71

Distributing Forwarding and Control

Distributed forwarding
All modern routers have distributed user (forwarding) plane BUT most have centralized control
Low CER speeds lots CERs/PER more likely to be constrained by control plane limits than packets per second

Distributed control
CER-PER routing and vrf tables limited to necessary interfaces Central controller has no vrf tables, vpnv4 route tables, or CER peering protocols

72

IP Multicast VPN Solution

CE Backbone Multicast CE PIM Adjacency PE PE P PIM Adjacency CE PE PIM Adjacency VRF mVRF Backbone

It is based on Multicast Domain (draft-rosen-vpn-mcast-05.txt) P and PE routers multicast enabled Provider internal multicast routing tables Globally PEs configured to run PIM (global instance) with adjacent P routers

PEs maintain PIM adjacencies with CE devices

Normal PIM configuration in customer network
PIM modes, RPs , multicast addressing
73

Multicast Tunnels and the default MDTs

CE Provider Network CE PE
Per VRF MDT

PE

PE

CE

Per mvrf default multicast distribution tree (default MDT) using traditional PIM

within backbone MDT used to distribute end customer multicast packets and PIM control messages Access to the MDT is via a multicast GRE tunnel interface on PE Each PE in VPN is a leaf _and_ root on the MDT For efficiency (but more state) can launch per session (S,G) MDTs for a VPN
74

Using an MDT
C-packet SRC=PC1 DST=225.1.1.1 PE CE SRC Lo1 MDT GROUP ADDRESS 234.10.10.1 P-packet SRC=Lo1 DST=234.10.10.1 C-packet SRC=PC1 DST=225.1.1.1 PE CE Receiver

Forwarding onto the MDT is done in encapsulated packets off PE

GRE or IP-in-IP

C-Packets - customer control and data packets

P-Packets provider control and data packets

Destination Address = MDT group address for VRF Source Address = IP address of PE M-BGP peering address

C-packet becomes a P-packet when encapsulated MPLS is NOT used!

75

IP Traffic Engineering and MPLS

Improving utilization of backbone resources

76

The Multi-Commodity Flow Problem

ni cij nj
The traffic engineering problem: 1)
Demands d(i,j) from node i to j
Constraints - link capacity b(i,j) Costs, e.g., link costs C(i,j) Path (route) p(k) variables for each demand

Find a feasible solution

2)
3)

Find a min cost solution

Find feasible and min cost solution with single node or link deletion

77

Explicit Routing
Solutions to the arbitrary TE problem require specifying the
explicit route (path) for each demand

Could calculate explicit routes satisfying constraints offline

Then specify explicit routes in network without constraints

78

Constraint-based Shortest Path First (CSPF)

Can let the network enforce constraints CSPF distributed algorithm Given full knowledge of network resource allocation

Route a demand by
1) Pruning network to only feasible paths 2) Pick shortest path

Compromise to solving the full TE problem

79

IP TE
Metric Manipulation
i.e., pick OSPF weights to create feasible solution
Limited in problems that it can solve

Simple Topology and capacity augmentation

Tends to over-engineer or restrict topology

Source Route
IPv4 option that allows explicit route Very costly, not practical No efficient explicit routing nor knowledge of network resource allocation
80

Making it Fit with Plain IP Routing

D 2

A 1

3 C B 4

Link size = 1, d(1,2) = 0.75, d(1,3) = 0.5, d(1,4) = 0.5

Cant pick OSPF weights that work
81

ATM TE
ATM routing (PNNI) has knowledge of resource usage
Bandwidth booked per trunk

Performs CSPF to find feasible path for

New demand
Rerouted demand Feasibility referred to as Call Admission Control

82

Making it Fit with ATM switching

D 2

ATM switch

A 1

3 C

B 4

Link size = 1, d(1,2) = 0.75, d(1,3) = 0.5, d(1,4) = 0.5

83

IP over ATM
The way to build ISP backbones not too long ago Allowed efficiently utilizing a limited number of costly
facilities shared among routers backbone routers

Typically a full mesh of ATM PVCs is created among the

PVCs sized to router endpoint demand

But
Led to N^2 IP peering
IP router investment outstripped speed of ATM

84

MPLS TE for IP
Provides efficient explicit routing for IP Can communicate resource constraints But not an overlay routing design
Routers not in a full peering mesh

Uses IP-based control plane protocols rather than a

different protocol RSVP-TE uses extensions of RSVP to carry labels and additional constraints

85

How RSVP-TE Works

PATH downstream contains explicit hops and bandwidth RESV upstream contains labels

A
1

3 C
51 pop

18

pop
LDP 3 L9

PATH <A, B, C, 3> 0.5Mbps

RESV with labels

86

Online CSPF with OSPF-TE

Can use RSVP-TE without resource reservation
Calculate constrained paths offline

For online CSPF need:

Knowledge of resource assignment in network
Add resources to OSPF link states

i.e., bandwidth available per class (diff-serv)

Flood changes in resource allocation

Unlike normal OSPF which just floods when link up/down changes

Now use RSVP-TE with non-zero reservations per class (diffserv) Similar to ATM PNNI

87

MPLS Fast Reroute

Using MPLS TE to improve availability
RSVP-TE creates backup tunnels On failure of protected LSP, packets are shoved down backup LSP tunnel Switchover is faster than waiting for CSPF to calculate and signal a new LSP

For local repair (link or node) can recover ~100ms or

better
Backup LSP is already in place, so as soon as the failure is detected locally the headend just needs to reprogram the label FIB

88

Link Protection
Create backup LSP around link to Next Hop With or without reservation
Can also backup normal LDP LSP

Backup tunnel. Pushes label 51 onto tunnel

2
pop

A
18

C
51

45

Protected LSP
89

Node Protection
Create backup tunnel LSP for two hops away (next-next
hop)

Backs up RSVP-TE tunnel

Learns labels from RESV recorded route of protected tunnel

Backup tunnel. Pushes label 45 onto tunnel

2
pop

A
18

C
51

45

Protected LSP
90

Path Protection
Create an end-to-end diverse backup tunnel Slower than local protection have to wait for headend to
detect failure

Backup LSP

2
pop

A
18

C
51

45

Protected LSP

91

What are Layer 2 VPNs?

Defined at the IETF PPVPN and PWE3 groups
http://www.ietf.org/internet-drafts/draft-ietf-ppvpn-l2vpn-requirements-00.txt http://www.ietf.org/html.charters/pwe3-charter.html

Point-to-point
Virtual Private Wire Service (VPWS) Offers FR, ATM and Ethernet pvc-like services
Nothing new here have been available for many years

Multi-point Ethernet Bridging

Virtual Private LAN Service (VPLS)
Similar to the Transparent LAN Services

Around for a while using standard Ethernet switching But VPLS is more scalable over the WAN

92

So what?
IP or MPLS as the multi-service carrier core
Was ATM, but ATM didnt keep up with IP investments On one core network carrier can put
Internet, Voice (trunking and service), FR, Ethernet, ATM, L3 IP VPN, IPSEC VPN
Finally, network convergence for the carrier??

New market for struggling carriers

Some newer providers only built fiber transport and IP backbone for Internet service and no ATM backbone
They are eager to go after the Enterprise WAN business L2 VPNs can be built on their existing IP infrastructure

For Customers nothing really new, just more competitors

for their WAN
93

Business Communications Review, Jan 2002 chart from Vertical Systems

94

Tunneling
PE-to-PE tunnel
L2TP MPLS

Multiplexer field
One tunnel, many connections called Pseudo Wires

Control field
Optional sequence number (detect out of order packets) Protocol specific control bits (e.g., DE, FECN, CLP, PTI)

95

Encapsulations
IP header (20B) Payload Session ID (4B) Cookie (8B) Control word (opt)

L2TPv3 purely connectionless with IP header

No new technology in carrier IP backbone but Spoofable
Cookie provides no strong verification

No QoS other than diff-serv

MPLS
Less overhead Can use MPLS TE
Tunnel label (4B) Payload cont VC Label (4B) Control word (opt) Payload
96

L2 MPLS VPN: Example FR

Directed LDP between PE pair exchanges FEC and label
for a particular pseudo wire

DLCI 200

CER PER
DLCI 300

MPLS Network
PVCs within tunnels

DLCI 100

PE

CER

CER

PER

L1 | L2 | Cw | FR PDU
FR PVC from DLCI 100 to 300
97

VPLS Virtual Private LAN Service

Multipoint to multipoint service
Any-to-any

Does LAN bridging, MAC address learning While Ethernet frame based
IT IS ACCESS TECHNOLOGY AGNOSTIC
Dont have to use GigE Can use Ethernet bridging over other access types

i.e., bridged over FR/ATM/PPP for NxDS0, T1, NxT1, T3 or bridged over SONET

Any protocol not just IP [e.g., IPX, DECNET]

No routing with carrier! but

More than a few dozen sites on a VPN (single LAN)? No Spanning Tree, so just connect routers
98

State of the Art

OAM work still needs a lot of work
Fault detection/isolation, performance measurement, probing

Little Call Admission Control

How to map bandwidth resources and classes onto tunnels

No Multi-AS implementations Minimal legacy interworking

Just glue connections at dumb interconnect to ATM

99

MPLS to Prem or in the Enterprise?

Can run MPLS to CER
By running BGP CER-CER
Can create own VPNs (vpnv4) on top of providers

Tenant service
3rd tier ISP backbone outsourcing (carriers of carrier)

Hierarchy of ipv4 routing

But dont need MPLS for tunneling CER-CER

Use IP tunneling with transparent interoperability with carrier

MPLS in private network

Create own VPNs (essentially an internal carrier) For traffic engineering IP

100

Completed PHASE I- MPLS Please Continue to the Next Phase

101

Performance Engineering in MPLS-based VPNs

Susan Hilton

Enterprise Network Consultant

Performance Engineering

Rationale CoS Foundation Technologies Service Implementation Applied Performance Engineering

103

Not All Traffic is Equal

APPLICATIONS
Interactive Data 3-Tier ERP Bulk Transfer Interactive Voice

SERVICE METRICS

BANDWIDTH

LOW

MEDIUM

HI

MEDIUM

DELAY

LOW

MEDIUM

HIGH

LOW

JITTER

MEDIUM

MEDIUM

HIGH

LOW

104

Multi-Application Networks
Mixing applications with similar traffic characteristics and
similar performance requirements is simply a sizing exercise Statistical multiplication

Mixing applications with conflicting traffic characteristics

often causes some to not meet Response Time requirements Even with sufficient bandwidth deployed!

105

Traffic Profiles and basic QoS requirements Voice, Video and Data
Voice Video-Conf Data

Smooth Drop Sensitive Delay Sensitive UDP Priority

Bursty Drop Sensitive Delay Sensitive UDP Priority

Smooth/Bursty Drop Insensitive Delay Insensitive TCP Retransmits

Bandwidth per call depends on codec and samplingrate

Latency 150 ms Jitter 30 ms Loss 1%

Similar performance requirements as VoIP, but radically different traffic patterns

Latency 150 ms Jitter 30 ms Loss 1%

Traffic patterns for Data vary among applications (and even among different versions of the same application)

One-way requirements

One-way requirements

Data Classes: Mission-Critical Apps Transactional/Interactive Apps Bulk Data Apps Best Effort Apps (Default)
106

Application Class

Example Applications

Data Classifications: Application Examples

Application / Traffic Properties Packet / Message Sizes Highly Interactive applications with tight user feedback requirements. Average Message Size < 100 B Max message size < 1 KB bytes

Interactive

Telnet, Citrix, Oracle Thin-Clients, AOL Instant Messenger, Yahoo Instant Messenger, PlaceWare (Conference), Netmeeting Whiteboard

Transactional

SAP, PeopleSoft - Vantive, Oracle Financials + Internet Procurement + B2B + Supply Chain Mgmt + Application Server Oracle 8i Database, Ariba Buyer, I2, Siebel, E.piphany, Broadvision, IBM Bus 2 Bus, Microsoft SQL, Lotus Notes, Microsoft Outlook, BEA Systems, Email Download (SMTP), DLSw+

Transactional applications typically use a client-server protocol model.

User initiated client based queries followed by server response. Query response may consist of many messages between client and server. Query response may consist of many TCP and FTP sessions running simultaneously (e.g. HTTP based applications)

Depends on application. Could be anywhere from 1 KB to 50 MB

Bulk

Database Syncs, Network based Backups, Video Content Distribution, Large ftp file transfers All non-critical traffic, HTTP Web Browsing + Other Miscellaneous traffic

Long file transfers Always invokes TCP congestion management

Average message size 64 KB or greater

Best-Effort

107

Performance Engineering
Includes: Network Engineering

Capacity Planning Traffic Engineering Bandwidth management

congestion management/avoidance to ensure the availability of high priority traffic and at the same time increase the network efficiency

What is performance engineering? The process of engineering a network to assure that applications attain their required performance.

Provide defined service metrics for different applications

108

Service Metrics
Bandwidth: Delay/Latency:
Time it takes a packet to travel from origination to destination Distance, switching, insertion, queuing

Jitter (Variability in Delay):

Latency that is unpredictable; 1st packet 10ms delay, 2nd packet 30ms delay. (Early/late packets)

Packet Loss:
Buffer Overflows, Selective discards, Line Errors

109

Increase in:

Technology Evolution - General Attributes

Most connectivity Most delay Least cost

I VPNS L2
VPN
ATM

Connectivity Shared
Resources (<$?)

Path Variance Delay Delay Variance

Decrease in:

FR

FR

Per connection
engineering

PL

Least connectivity Least delay Most cost

110

Technology Evolution - Future Direction

I
COS

I VPNS L2
VPN
FR
ATM

Connectivity Shared Resources

(<$?)

VPNS
COS

L2
VPN COS

PL

Class-of-Service features added to improve: Path Variance Delay Delay Variance

FR
COS

ATM PL

111

QoS / CoSWhats the Difference?

QoS Quality of Service
Absolute Metrics, Contracted parameters Each flow must be engineered independently Addresses the service requirements of different applications in order to provide more than best effort service for specific applications

CoS Class of Service

Relative treatment of contending flows Implies that flows can be categorized or differentiated! The implementation that provides QoS

Used interchangeably in this session

112

Is Bandwidth the Answer?

Just deploy more bandwidth

Queuing Delay is f(link speed) Bandwidth is cheap QoS is complicated

MAYBE for LAN, MAN Maybe even for WAN backbone WAN edge will remain a bottleneck for foreseeable future

113

Why IP QoS is Needed

Enterprise networks are migrating toward IP transport Best effort is not good enough

Engineered performance is required for enterprise applications.

Emerging applications (VOIP, Streaming) are highly sensitive
to delay, jitter (delay variation), and packet loss.

Need performance/reliability of private networks

with ubiquity/cost advantage of Internet.
One approach MPLS

114

CoS for IP VPNs

Traditional techniques do not work for mesh topologies as we
will see.

Egress port speed is still a bottleneck

The Service needs to participate in CoS solution.

115

CoS Foundation Technologies

Advanced Queuing
Queue Management Traffic Shaping

Classification and Marking

Fragmentation

116

Advanced Queuing
Advanced Queuing is any technique that transmits
I.e. Not FIFO

packets in a different order than they were received.

These techniques only kick-in when there is congestion.

(I.e. if there is no queuing, then there is no advanced queuing.

Advanced queuing only makes sense where there is a speed mismatch.

I.e arrival rate is greater than departure rate.

117

Priority Queuing
In
Prioritization allows
specified traffic to preempt competing traffic.

Can be multiple
(4 in Cisco)

levels of priority.

Higher priority

traffic can starve lower priority traffic.

Out

118

Bandwidth Allocation
Also called:
Each traffic type gets a relative allocation of the bandwidth. bits, bytes, packets
Custom Queuing Weighted Round Robin

In

10

Out
119

Bandwidth Allocation Considerations

Cycle Time
All buckets are served in each cycle. (No priority) More buckets longer cycle time

Unused allocation shared proportionately across

remaining traffic types.

Cisco de-queuing quirk

Always serve at least 1 packet, even if it is bigger than allocation
No concept of credit or deficit.

120

Fair Queue
Not all packets are equal
Large and small packets Part of sparse or heavy peer to peer More or less time sensitive to application

Scheduling algorithms basic

When a packet within a flow arrives, calculate when it would get served as part of each flow Process packets in this order (not necessarily at this time)

3 5 4 2 1

121

Weighted and Flow Based

Simple algorithm works well to give everyone a fair share.
Some problems Sparse flows must wait even though they require little bandwidth (high jitter) High priority packets must wait

Assign weights to scheduled times based on

Precedence

Sparse or heavy flow (Cisco Flow Based WFQ)

Other

122

No defined traffic classes

Weighted Fair Queuing

Packets hashed to 1 of 64 queues Hash f (SourceIP, Source Port, DestIP, DestPort) Possibility of bulk and interactive with same hash

Weighted Fair means each queue has equal weight (1)

Each Packet is scheduled at arrival time
Schedule Time = Queue Tail + (Weight * Length) De-Queue based on Schedule Time
Calendar Queuing

0
125

1
Hash

1000

2 3
300
200 310 250
240 180

150

64

300

120

60
123

Flow Based WFQ more detail

Detects bandwidth of layer 4 flows (also know as
conversations)

Classifies traffic into as many as 64 bins Allocates bandwidth equally across all flows Light flows get the bandwidth they need, heavy flows share
the remaining bandwidth

On by default in Cisco low speed interfaces

124

Class-Based Weighted Fair Queuing

A Hybrid
Class-Based
Defined Classes instead of hashing

Weighted Fair Queuing

Weighted Fair Queuing with defined weight
Schedule Time = Queue Tail + (Weight * Length)

Class-Mapping

Weight

0
400
100 bytes

A B

2
10
1200

600
150 bytes
600

300
150 bytes

60 bytes

60 bytes
125

Class-Based Weighted Fair Queuing

Weight of a traffic class is implied by Bandwidth
Weight ~ 1/(BW Percent)

Behavior is opposite of WFQ

Higher BW = Higher Priority

Example: Class A FTP Class B Telnet

BW 10% Weight 10 BW 90% Weight 1.111

Class-Mapping

Weight

0
15,000
1500 bytes

10

B
1.1

244, 198, 132, 66

60 bytes each
126

 Unlike WFQ that applies relative priority, CBWFQ enables absolute

guarantees

CBWFQ - Cisco

Assigns flows to classes

Allocation to a class can be based on almost anything

Made up of many parts

FQ ~ Fair Queue, nobody gets it all

W ~ Weighs applied to queues CB ~ Class Based, uses class to define queues

None of this applies unless there is a queue to manage. When no
congestion, no priority is assigned.

Only available Cisco solution for high speed router ports (above E1, with
possible exception of DS3 Frame Relay)

Weights are applied per class Generally uses Flow Based WFQ within a class Includes a special class Low Latency Queue (LLQ)

127

 Policy-map defines the classes Class-map assigns packets to a class Service-policy invokes the policy on an interface
class-map class1 match access-group 101 class-map class2

Configuring CBWFQ (Cisco)

match input-interface s0
! policy-map policy1 class class1 bandwidth 50 queue-limit 100 class class2 bandwidth 20 queue-limit 35 class class-default fair-queue interface atm0.1 point-to-point ip address 10.10.10.1 255.255.255.252

pvc atlanta 1/105

vbr-nrt 40000 72000 32 service-policy out policy1
128

Low Latency Queue - LLQ

A special queue defined by the policy-map Applies strict priority up to the bandwidth specified will
not serve any other queue until the LLQ is empty queue depth do not apply bandwidth command

Drops packets above the specified bandwidth WRED and

Invoked by using the priority command in place of the

129

Queue Management
Queue Depth is specified in policy-map CBWFQ defines how queues are served but what happens
when a particular queue gets too big?

Packets are discarded

Tail Drop drops all packets arriving after the queue is full Weighted Random Early Detection (WRED)

130

 Tail Drop, Global Synchronization, WRED

Queue Management Tail Drop

Tail drop tends to affect all flows in a queue. This effect is called global synchronization. All flows crank up their windows, congestion occurs, tail drop drops all arriving packets, all widows reset.

Total BW Utilization

Queue Full
Average Utilization After tail drop.

Individual TCP Sessions

131

Queue Management Weighted Random Early Detect

Effective for large number of flows, questionable for enterprise. Can specify differing WRED threshold based on IP Prec.

- WRED

WRED drops a few random packets before congestion reaches the queue depth threshold. This causes a small number of flows to reset their TCP/IP window; while the remaining flows continue to use available bandwidth.

Queue Full
Total BW Utilization

WRED Thresh
Average Utilization After WRED drop.

Individual TCP Sessions

132

Tail Drop vs. WRED

Tail drop tends to affect all flows in a queue. This effect is
called global synchronization. All flows crank up their windows, congestion occurs, tail drop drops all arriving packets, all widows reset.

WRED drops a few random packets before congestion

reaches the queue depth threshold. This causes a small number of flows to reset their TCP/IP window.

133

Traffic Shaping

Traffic shaping is a tool to move a queue from one place in a network to another. Queues only occur where there is a speed mismatch.

If arrival rate > departure rate -> queue

Traffic Shaping forces a speed mismatch in a router to prevent a speed mismatch in a network.

134

Traffic Shaping
T1 In / 64K Out Queue in the Network

T1 Port

64K Port

Router

FR Network

Router

Traffic Shape to ~ 64K No Queue in Network

Queue is in Router, where advanced queuing can be used
135

Mesh Based Shaping

In mesh topologies, where is the choke point? That is
where a queue will build!

Router

Router

Can we control this queue at the high speed

end? NO, because of other remote sites.

136

Value of MPLS Traffic Shaping

Unlike mesh PVCs, MPLS services can apply shaping in
the cloud to manage the queue

Router

Router

Q
137

Policing / Shaping Two Sides of the Same Coin

A service subscription has an implied traffic contract
Service provider Polices arriving packets against the contract Customer Shapes traffic to assure conformance to the contract

138

Policing
Policing
Enforce a traffic contract Pass all traffic within contract

Out of contract
Drop Or mark as out of contract

DE Bit in frame relay networks CLP in ATM networks IP Precedence or DSCP in IP networks

139

What happens to Non-Conforming Traffic

Mark but do not discard, unless congestion
OR

Drop

140

Classification and Marking

Need to identify packets in order to determine what
service level is required (classification) Supported by marking or coloring Marking is done in the IP header
Ver Hdr ID TTL Protocol
Type of Service

Length Flag Fragment Offset

Header Checksum

20 bytes

Source IP Address
Destination IP Address Options Data

141

 Type of Service provides 8 bits

Bits 0-2 IP Precedence
Value 0 1 Bits 000 001

Marking IP Precedence

Name Routine Priority

2
3 4 5 6 7

010
011 100 101 110 111

Immediate
Flash Flash Override Critical Internet Control Network Control
142

 Start with IP Precedence 3 bits (Class Selectors) Add Drop Precedence Levels of 3 bits
Precedence 7 6 Samenetwork control Sameinternet control

Marking DSCP

5
4 3 2 1

Express Forwarding (EF)

Class 4 Class 3 Class 2 Class 1

Best Effort
143

 The second three bits are used for Per Hop Behavior or
Drop Probability Applies to Class 1-4 or Assured Forwarding (AF) Provides more flexibility
Class 1 Low drop 001010 AF11 DSCP 10 Medium drop 001100 AF12 DSCP 12 Class 2 010010 AF21 DSCP 18 010100 AF22 DSCP 20 Class 3 011010 AF31 DSCP 26 011100 AF32 DSCP 28

Marking DSCP

Class 4 100010 AF41 DSCP 34 100100 AF42 DSCP 36

High drop

001110
AF13 DSCP 14

010110
AF23 DSCP 22

011110
AF33 DSCP 30

100110
AF43 DSCP 38
144

Fragmentation
On low speed (<768K) ports, queueing is not enough Insertion (serialization) delay is an issue Insertion = packet size (bits)/line speed
Example (1500*8)/56 = 214 msec

Objective for voice is 10 msec insertion delay per packet

145

Insertion Delays
Fn(line speed)
100 bytes 100 bytes

Fn(packet length)
500 bytes

56Kbps t 14ms t

128Kbps 64Kbps
6.25ms 12.5ms t

128Kbps 64Kbps 31.25ms 62.5ms

800bits /56K = 14ms 100bytes / 7 = 14ms

56Kbps=7bytes/ms

(8*100)*(1/128K + 1/64K) = 18.75ms

(8*500)*(1/128K + 1/ 64K) = 93.75ms

146

Fragmentation & Compression

Even with optimal queuing treatment, performance may not
be acceptable.

The best treatment that can be obtained with strict

I.e. Best case, prioritized packet gets transmitted immediately.

prioritization is queuing delay of O(1/2 of a packet).

Worst case, prioritized packet has to wait for the currently transmitting packet to finish. On average, wait time for prioritized packet is of a non-prioritized packet.

This is still substantial delay (and jitter) for low speed ports. Fragmentation & Compression is a means to make the low
priority packets smaller; so O(1/2 packet delay is smaller)

147

Head of Line Blocking

Real time traffic arrives but a 1500 byte packet is just
starting transmission

LLQ/CBWFQ gives only priority if the line has not started

to send the packet There is no preemptive capability.. 64-kbps access. Real time traffic have to wait. This is HOL Blocking

The 1500-byte frame takes 187.5 ms to serialize on a

Link Fragmentation and Interleaving (LFI) is the mechanism

fragment large data frames into regularly sized pieces and to interleave small real time packets into the flow.

148

Head of Line Blocking

149

Fragmentation

MTU Dangerous many apps set Do Not Fragment Can be done at source, but no very practical

FRF.12 Fragment data packets Prioritize voice packets

Voice only, not suitable for priority data

No LFI ATM Interworking is complicated

ML-PPP Best generic solution

More overhead than FRF.12, but OK with compression

150

Fragmentation
Packet Size (bytes)
50 1536 768 512 256 192 128 64 56
0 1 1 2 2 3 6 7

100
1 1 2 3 4 6 13 14

250
1 3 4 8 10 16 31 36

500
3 5 8 16 21 31

1000
5 10 16 31

1500
8 16 23

Link Speed

63 71

42 63 125 143

47 63 94 188 214

T (Transmit 1 Packet) mS
151

Why Compression?
IP header UDP header RTP header

Real time protocol, synchronizes packets

IP Header UDP Header RTP Header 2 Voice Samples 20 bytes 8 bytes 12 bytes 20 bytes

Total = 60 bytes, 63% is overhead! 60 bytes * 8 bits/byte * 50 PPS = 24KBPS!

152

Why Compression?

153

How does it work?

Compressor and decompressor share consistent state : that includes
fixed fields, first order differences and second order difference fields (delta encoding) index.

There is Context ID (CID) that identifies flows and used as database Flows are hashed against IP source&destination address and UDP
source&destination ports and assigned CID at compressor

Decompressor use CID as database index .. There is sequence number to detect packet loss

Bandwidth vs CPU

154

How it all works together!

155

Service Implementation
Example AT&Ts IP Enabled Frame
Relay/ATM
Marking Shaping Queuing

Profiles
Policing Futures

156

Service Architecture

CE
MPLS Core
PER PER
Access PVC (CDR)

Port

Port

Frame Relay or ATM

CE
157

 CER marks TOS or DSCP bits

Real Time 101 110 or 101 000
Dropped if rate exceeded

Marking

Bursty High
011 010 or 011 000 in contract Remarked to 011 100 if out of contract

Bursty Low
010 010 or 010 000 in contract 010 100 out of contract

Best Effort
000 000

158

Shaping
Based on egress port speed
Contracted rate not a factor Shaped to egress port speed

Moves queue into the network router

159

IPFR/ATM CoS Implementation

AT&T uses Low Latency Queueing (LLQ) and Class Based
Weighted Fair Queueing (CBWFQ) in the IP FR/ATM network to implement CoS

LLQ has strict prioritybest for delay sensitive applications like

voice CBWFQ allows critical data classes to get more bandwidth allocation than other classes
LLQ drained completely whenever packets queued (up to Max BW during congestion) Other classes typical CB-WFQ
V V V V V V 1 1 Interface Class 2 (FIFO)

LLQ (FIFO)

1 1 1 1

Class 1(FIFO)

2 2

2 2 2

3 3

3 3 3

Transmit Queue (FIFO)

Class 3 (FIFO)

160

Queuing
4 Queues (Classes of Service)
Real Time--LLQ Bursty-HiCBWFQ with WRED

Bursty-LoCBWFQ with WRED

Best EffortWFQ with WRED Mapping based on IP Precedence or DifServ marking

161

 AT&Ts IPFR/ATM Service provides for 4 separate classes

Real Time Bursty High

CoS Profiles

Bursty Low
Best Effort

The 4 classes can be thought of as a priority hierarchy The hierarchy is controlled by bandwidth allocation
Simply, the more bandwidth that is allocated, the higher the priority Real time class is given a strict bandwidth allocation Other classes are assigned a percentage of CDR bandwidth

162

Real Time Class

Best for voice, maybe videonot both if port speed is
<768K

Strictly policedexcess over the allocated bandwidth is

dropped

Must be purchased in increments of 20% of CDR Determine real time allocation based on call
requirements

163

Policing Real Time class

RT
Ingress Police to contracted value
Drop excess
No burst

Egress Police only when port is congested

Drop excess
No burst

164

Policing Bursty Data

Ingress
Remark Excess (out of contract)
Contract based on allocation profile

Burst Size?

Egress
No policing function Queuing treatment of out of contract identical to in contract Lower drop thresholds

165

Futures
MPLS EXP bits used for CoS in the backbone
Will no longer remark customer packets

MPLS Traffic Engineering (TE)

166

Applied Performance Engineering

Optimizing response times
Identify traffic classes & requirements Implement policies at network bottlenecks.

Calculate expected behavior.

167

Applied Performance Engineering

Remember the relevant parameters:
Bandwidth

Delay
Jitter
Loss

168

Performance Engineering Guidelines

Potential queuing (Bottlenecks) exists at any point where the
arriving rate can exceed the departing rate.

i.e. find the speed mismatch points

The dominant bottleneck is at the slowest link in the end to end
connection.
TCP protocols tend to adapt to the available bandwidth. This means that the only place where there can be a sustained congestion is at the slowest link. Any faster links will only experience transient congestion.

For Low Speed ports, priority is more important For High Speed resources, BW allocation is sufficient

169

Frame Relay Performance Engineering

FR, ATM Speed Mismatch Network Buffering
App1

App2

Router

Router

App3

Q
170

Application Inventory
List all applications Include administrative apps
Routing DNS OS

Establish Requirements
Response Time, Bandwidth

171

What other applications are out there? Applications that have low delay tolerance (sub second)
Telnet Citrix

DLSW
TN3270

Applications that have multi-second response times

ERPs (SAP, Peoplesoft, Siebel) Credit card authorizations

Reservations
HTTP applications

Background applications
Email FTP Database synchronization

172

Response Time Requirements

Group applications according to
(response time requirements)

delay sensitivity.

P.O.S Telnet

Web

Voice

My-SAP PeopleSoft 7

FTP E-Mail

Very Delay Sensitive

Not Delay Sensitive

173

Grouping
If voice is present Put it in RT Group adjacent application classes into available bins.
Use caution if RT is to be used for data apps You dont have to use all available classes

Real Time

Bursty-Hi

Bursty-Lo

Best Effort

VOICE

P.O.S Telnet DNS

ERP Web

Mail FTP
174

Capacity Planning
Establish BW / Application
To meet stated requirements

Determine required port speed

175

 Only one type of data applicationsBursty High Mix of sub-second and background Sub-second Bursty High

What is the best profile?

Background Best Effort

Mix of multi-second and background Multi-second Bursty High

Background Best Effort

Mix of all types Sub-second Bursty High

Multi-second Bursty Low Background Best Effort

This process will help to identify the best profile!

176

Profile Selection
COS Package Multimedia High RT% of CDR 80 60 60 Multimedia Standard 40 40 20 20 20 10 10 10 Critical Data 0 0 BH% of CDR 20 40 20 60 40 80 60 40 80 60 40 100 80 BL% of CDR 0 0 20 0 20 0 20 40 10 30 50 0 20 BE --------------

0
Business Data Economy 0 0

40
0 0

60
100 0

--100

177

Insertion Delay Exercise

Given a 1500 byte packet, what is the insertion delay
o at 56K?
o at 128K?

If you wanted to run voice on the same 128K port as the

1500 byte packet, what size would you recommend fragmenting the packet to minimize jitter? (hint10 msec should be the insertion delay of the fragment)

178

CoS Exercise 1
Need to support 3 simultaneous calls with a G.729 codec Rest of data is web surfing
What port size do I need? What profile should I pick?

179

CoS Exercise 2

No voice requirements Lots of telnet traffic Some http-based ERP applications Rest is web surfing and email

What profile should I consider? Why?

180

Completed PHASE II Congratulations Completed All Phases

181