Mikrotik Data Center MUM 2014 KevinMyers 4 by 3

www.iparchitechs.
com
1-855-MIKROTIK
Network Architecture
Validated designs utilizing
MikroTik in the Data
Center
P R E S E N T E D B Y:
K E V I N M Y E R S , N E T W O R K A R C H I T E C T / M A N AG I N G PA R T NE R
I P A R C H I T E C H S M A N AG E D S E R V I C E S
1-855-MIKROTIK
www.iparchitechs.com
Background
Kevin Myers
16 + years in IT/Network Engineering
Designed and implemented networks in Service Provider,
Enterprise, Ecommerce and Government environments
Areas of Design Focus:
MikroTik integration with multi-vendor networks
Design of BGP/MPLS/OSPF Service Provider Triple-Play
networks
Design of large enterprise Data Center networks
Certifications
MTCINE #1409INE006
Certified CCNP, CCNA, MCP, MTCRE, MTCTCE, MTCNA
24/7/365 MikroTik TAC
Nationwide Private 4G LTE MPLS
Proactive Network Monitoring
Design / Engineering / Operations
1-855-MIKROTIK
IP ArchiTechs Managed Services

Exhibitor at 2013 and 2014 MUM Please stop by our exhibitor booth and register to win
an RC Helicopter !
The first Carrier-Grade 24/7/365 MikroTik TAC (Technical Assistance Center)
Three tiers of engineering support
Monthly and per incident pricing available
1-855-MIKROTIK or support.iparchitechs.com
AirMPLS - Private Nationwide 4G LTE MPLS backbone
Partnership with Verizon Wireless - available anywhere in the Verizon service area
Not Internet facing privately routed over our MPLS infrastructure
Multiple Deployment options to carry public and private traffic including L2 adjacency
Proactive Monitoring / Ticketing / Change Control / IPAM (IP Address Management)
Carrier-Grade Network Engineering / Design in large (100,000+ nodes) environments
1-855-MIKROTIK
Introduction The MikroTik enabled Data

Center
Role within the Data Center
Layer 3 Core Designs using CCRs with 10 Gbps interfaces
Top of rack / End of Row L3 options for core connectivity
External / Internal Firewall Internet reachability / protect critical internal
networks with multiple layers (PCI)
VPN Aggregation Multiple Vendors / Remote Mgmt Access
MPLS P/PE router - Segregation of traffic within Data Center
Role between Data Centers

MPLS L2 VPN VLAN extension between Data Centers for VM mobility
MPLS L3 VPN Segregate traffic as it routes between data centers
VLAN Rewrites Used to deal with VLAN overlap between two or more sites
Multiple Gateways for the same subnet at more than one site
1-855-MIKROTIK
Conventional Data Center
1-855-MIKROTIK
Multi-Million dollar DCs - Where does MikroTik fit in ?
MikroTik routers can be used in different areas of the Data

Center and compete with mainstream vendors like Cisco,
Juniper and HP within a specific set of design parameters.
The goal of this presentation is to display the design
elements required to build a Layer 3 infrastructure
capable of up to 320 Gbps forwarding with off the shelf 10
Gig switches
Why 320 Gbps?
Relies on ECMP (server side) assuming 16 BGP paths
16 Paths is a conservative value for ECMP some go as high as 128
paths
16 CCRs with 20 Gbps LACP channels = 320 Gbps
Design validation was tested with 2 CCRs which yielded 40 Gbps
between servers
1-855-MIKROTIK
Multi-Million dollar DCs - Where does MikroTik fit in ?
Why?
The business case for MikroTik in the Data Center
CAPEX (Capital Expenditure) Savings
Lower hardware replacement cost when a node fails
Cisco Nexus Deployment for 320 Gbps
$2,000,000 to $5,000,000 CAPEX
MikroTik Deployment for 320 Gbps
$50,000 to $100,000 CAPEX
1-855-MIKROTIK
The MikroTik enabled Data Center
1-855-MIKROTIK
Part 1 Desigining for High Availability

99.999% uptime
Getting to five 9s isnt easy can only have 5 minutes of unplanned
downtime per year maintenance windows arent included
HA design elements
Stackable switches enables multi-chassis LAG for CCRs and servers to provide
survivability in the event of a failure of one of the switches
LACP channeling/bonding at Layer 2 allows devices to aggregate speeds as well as
prevent routing topology changes when a link fails
Load Balancers Provide the ability to use multiple CCR chassis as a single firewall
without breaking state. The LB has the ability to return traffic to the same source
and track that relationship dynamically.
BFD Bi-Directonal Forwarding Detection allows a network t o converge much more
quickly than standard timers
Multiple Internet BGP Peers When used along with BFD if the upstream carrier
supports it, multiple tier 1 peerings provide a level of redundancy to ensure Internet
traffic is uninterrupted
1-855-MIKROTIK
Part 2 Achieving 320 Gbps throughput

Two mechanisms for achieving high throughput
Method 1 - ECMP
Equal Cost Multipath (ECMP) on the CCR RouterOS is capable of up to 128 gateways .
Example below shows 16 gateways for one route
Routes can be installed by either OSPF or Static. BGP can also be run on top of OSPF and utilize
ECMP as well.
ECMP Route with 16 Gateways
1-855-MIKROTIK

ECMP Continued
Using multiple gateway allows traffic egressing the router to balance

along multiple paths but what about ingress?
Server side ECMP is the key to scaling throughput when using

independent routers.
Support in multiple operating systems Microsoft and Linux both
support ECMP in static routes along with OSPF and BGP
1-855-MIKROTIK
Achieving 320 Gbps throughput
1-855-MIKROTIK

Utilizing BGP and OSPF at the server for dynamic ECMP
Role of OSPF
Converges quickly using adjusted standard timers (1 second hello 3 second dead)
Converges even faster with Bidirectional Forwarding Detection (BFP)

Provides Loopback reachability for BGP
Is needed to implement ECMP dynamically through MikroTik routers until ECMP
is added to BGP.
Role of BGP
Advertise data center subnets for servers, databases, web apps, etc, to the 16
router CCR core
Can be utilized for traffic management
1-855-MIKROTIK
Achieving 320 Gbps throughput
1-855-MIKROTIK

Method 2 Offset VRRP Gateways
Each CCR is the VRRP master for one or more VLANS
Requires setting priority for each VLAN/CCR
Can be used in conjunction with ECMP when servers cannot be setup for ECMP
CCR 1 Master for VLAN 100 Backup for VLAN 200
CCR2 Backup for VLAN 100 Master for VLAN 200
1-855-MIKROTIK

LACP Channels for Routers and Servers
LACP is an open standard for aggregating Layer 2 links 802.3ad
Referred to as Channeling, Bonding, Teaming, Link Aggregation
Can be trunked with multiple VLANs and multiple Layer 3 gateways
Can be used with VRRP
20 Gigabit LACP channel on CCR-1036-8G-2S+
1-855-MIKROTIK

Microsoft LACP example:
1-855-MIKROTIK

Linux LACP example:
1-855-MIKROTIK

Final Result 40 Gbps throughput
Only 2 CCRs in the Core with 16 CCRs, the throughput will be roughly 320 Gbps
1-855-MIKROTIK
Part 3 Multiple Data Centers
1-855-MIKROTIK
Using MPLS in the Data Center

CCRs can be used as MPLS edge routers to connect Data Centers.
Used to segregate traffic within and between Data Centers
L2VPN (VPLS ) Provides Layer 2 Connectivity and isolation
L3VPN Provides Layer 3 connectivity and isolation
VRF (Routing Marks) Used to separate customer routing tables so
that more than one customer can use the same subnet without
overlap
1-855-MIKROTIK

MPLS Customer Isolation at Layer 3
1-855-MIKROTIK

EoIP provides Layer 2 Connectivity and will allow MPLS to function across an
encrypted internet link. Either EoIP or VPLS can be used for L2 connectivity.
1-855-MIKROTIK
VLAN Rewrites
Problem:
Data Center 1 uses Vlan 100 for web Servers on 10.1.1.0/24
Data Center 2 uses Vlan 100 for storage replication on 192.168.222.0/24
When extending the VLAN between Data Centers, one side must be
rewritten
CCRs can do this via bridging
MikroTik routers with switch chips can use /switch to perform vlan
rewrites
1-855-MIKROTIK
VLAN Rewrites change VLAN 100 traffic to

VLAN 3100
1-855-MIKROTIK
VLAN Rewrites change VLAN 100 traffic to

VLAN 3100
Create VLAN 100 and 3100 interface VLANs
Create Bridge and add VLAN interface ports
1-855-MIKROTIK
Dual VRRP Gateways

Problem when extending VLANs between Data Centers, If there is
not a local gateway for hosts in that subnet, traffic must go all the
way to the other Data Center via Layer 2 to hit the default gateway
Solution: Dual VRRP gateways
Data Center 1 VRRP GW 100.64.100.1/24
Data Center 2 VRRP GW 100.64.100.2/24
These are duplicate IPs How can this work?

Because VRRP uses MAC addresses derived from the VRRP Group
Number Hosts will always find the gateway in their own data center
before going to the other Data Center
1-855-MIKROTIK
Dual VRRP Gateways

Add input filter for VRRP on both edge routers IP Protocol 112 to prevent
either gateway from becoming master for the other (bridges must be set to use
IP Firewall)
Add VRRP Gateway for 100.64.100.1 in both Data Centers
1-855-MIKROTIK
Dual VRRP Gateways
1-855-MIKROTIK
2014 Pittsburgh MUM RC Heli Giveaway !!

4 To Give Away!!!
17 RC Helicopters
24/7/365 MikroTik TAC | Nationwide Private 4G LTE MPLS | Proactive Network Monitoring | Design / Engineering / Operations
1-855-MIKROTIK
Questions?
The content of this presentation will be available at mum.iparchitechs.com
Please come see us at the IP ArchiTechs booth in the Exhibitor Hall
Email: kevin.myers@iparchitechs.com
Office: (303) 590-9943
Web: www.iparchitechs.com
Thank you for your time and enjoy the MUM!!

Mikrotik Data Center MUM 2014 KevinMyers 4 by 3

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Mikrotik Data Center MUM 2014 KevinMyers 4 by 3

Загружено:

Авторское право:

Доступные форматы

www.iparchitechs.

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

IP ArchiTechs Managed Services

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Introduction The MikroTik enabled Data

Role between Data Centers

24/7/365 MikroTik TAC

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Conventional Data Center

24/7/365 MikroTik TAC

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Multi-Million dollar DCs - Where does MikroTik fit in ?

MikroTik routers can be used in different areas of the Data

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Multi-Million dollar DCs - Where does MikroTik fit in ?

24/7/365 MikroTik TAC

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

The MikroTik enabled Data Center

24/7/365 MikroTik TAC

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Part 1 Desigining for High Availability

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Part 2 Achieving 320 Gbps throughput

ECMP Route with 16 Gateways

24/7/365 MikroTik TAC

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Part 2 Achieving 320 Gbps throughput

Using multiple gateway allows traffic egressing the router to balance

Server side ECMP is the key to scaling throughput when using

24/7/365 MikroTik TAC

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Achieving 320 Gbps throughput

24/7/365 MikroTik TAC

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Part 2 Achieving 320 Gbps throughput

Converges even faster with Bidirectional Forwarding Detection (BFP)

Can be utilized for traffic management

24/7/365 MikroTik TAC

Nationwide Private 4G LTE MPLS

Proactive Network Monitoring

Design / Engineering / Operations

Achieving 320 Gbps throughput