Nexus 5000 Architecture

Nexus 5000 Architecture
Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential 1
Agenda
 System Hardware Overview

 Internal Architecture
 Fabric Data Path
 Lossless data path
 Forwarding and Policy Enforcement
Nexus 5000 Solution Components
Standards
Wire speed 10GE Data Center Virtualization
Access Switch Ethernet (DCE) FCoE
Networking
LAN SAN A SAN B
LAN LAN LAN
MAC MAC
A B A&B C
Active-Active
End nodes
Access N5000
N5000 Layer N5000
MAC MAC
A C
MAC
B
• 10GE L2 non-blocking, • L2 Multi-pathing for • Unified fabric for LAN, • VM-Optimized network
wire-speed switch increased bandwidth, SAN, HPC/IPC services
• Competitive price- scalable L2 domains • Enables FC connectivity • End Port Virtualization
performance • Priority Flow Control – across more servers • Unified I/O with 10Gb
• Low latency < 3.2s based Lossless fabric • Fewer switches, fewer and FCoE
• Congestion Management points of management • Delivers increased,
• Lossless
• DCE auto-negotiation • Significant cable and security operational
• Unified fabric agility, and better
adapter reduction
• Future proof  Lossless fabric & Greater utilization of network
Data Center Scale • Lower cost, power, cooling
assets
Eco System Partners
System Hardware
Overview
Nexus 5000 Product Portfolio
Industry’s First I/O Consolidation Virtualization Fabric for
Enterprise Data Center
28-Port 1RU Switch 56-Port 2RU Switch
Expansion
Modules FC + Ethernet Ethernet Fibre Channel
• 4 Ports 10GbE/FCoE • 6 Ports
• 4 Ports 1/2/4G FC • 8 Ports 1/2/4G FC
10GE/FCoE
Eco System Partners
OS Cisco NX-OS
Mgmt Cisco Fabric Manager and Cisco Data Center Network Manager
Rear Panels
NX5020 All 10GE ports are FCoE capable!
Support for 1 GE Support for crypto
10/100/1000 Expansion Power

Console Base 10GE
Out of Band Mgmt Module(s) Entry
Cables connect in the rear for ease of server wiring
Front Panels
NX5020
Dual redundant
N+1 redundant fans
power supplies
Replaceable components on the front for easy access
Power
NX5020
Power Supply
Maximum Power – 750W

Typical Operating Power – 480W
AC Input - 110/208 Volts
Efficiency - 82-88%
Protection - 110%-150% max load
Fully redundant, load sharing and hot swappable
Expansion Modules
Ethernet Six 10G Ethernet

Expansion Module
Four 10G Ethernet

Combination
Expansion Module Four 1/2/4G
Fibre Channel
Fibre Channel
Expansion Module Eight 1/2/4G
Fibre Channel
Cooling
NX5020
Cooling Module
Max RPM - 12K

Failover - N+1
Op Temp - 0 to 40 C
Humidity - 95% non-condensing
Elevation - 10K feet
SFP+ Transmission Media
SFP+ Cu
•Low power consumption
•Low cable cost
•Low transceivers latency
•Low error rate (10 exp-17)
Power Transceiver
Technology Cable Distance (each side) Latency (link)
SFP+ CU
Twinax 10m ~0.1W ~0.25 s
Copper
SFP+ USR MM OM2 10m

1W ~0.1 s
ultra short reach MM OM3 100m
SFP+ SR MM OM2 82m

1W ~0.1 s
short reach MM OM3 300m
Cat6 55m ~8W 2.5s

10GBASE-T Cat6a/7 100m ~8W 2.5s
Cat6a/7 30m ~4W 1.5s
Internal Architecture
Hardware Architecture
10 GE Interfaces 10 GE 1/2/4 Gbps Fibre Channel
Interfaces to Storage Network
SFP+ SFP+ SFP+ SFP+ SFP+ SFP SFP

xcvr xcvr xcvr xcvr xcvr xcvr xcvr 10 GE
10 GE SFP+ SFP+ SFP+ SFP+ SFP+ SFP SFP & FC
xcvr xcvr xcvr xcvr xcvr xcvr xcvr
Memory
Intel 3100 PCI Controller

Unified Unified Unified Unified
Port Port Port Port
Controller Controller Controller Controller
FLASH
Intel
LV Xeon
(1.66 GHz)
Unified NVRAM
Crossbar
Fabric
Serial RS-232 Console
Unified Unified Unified PCIe

Port Port Port
Controller Controller Controller
1GE
SFP+ SFP+
xcvr
XAUI
SFP+ SFP+
xcvr
XAUI XAUI
SFP+
Dual NIC Dual NIC
xcvr xcvr
xcvr
SFP+ SFP+ SFP+ SFP+
xcvr xcvr xcvr xcvr SFP+
xcvr
XFI XFI XFI
10/100/1000 Management
10 GE Interfaces
Supervisor Details
CPU 1.66 GHz Intel LV Xenon - LF80538KF0281M
IO Chip Set Intel 3100 South Bridge for embedded applications
DRAM 2 GBytes of DDR2 400 (PC2 3200) in two DIMM slots
Program Store 1 GBytes of USB based (NAND) Flash
Boot/BIOS 2 Mbytes of EEPROM with locked recovery image
On-board Fault Log 64 MBytes of Flash for failure analysis

Kernel Stack traces, boot record and fault logs
NVRAM 2 Mbytes of SRAM – Syslog and licensing information
Secure Keystore Renesas AE46C1 – Credentials and secure RNG
Management Interfaces RS-232 console port – console0

10/100/1000BASE-T – mgmt0 partitioned from inband VLANs
Unified Crossbar Fabric
 58 port crossbar and scheduler
3 unicast and 1 multicast crosspoints
 Central tightly coupled scheduler
Request, propose, accept, grant, acknowledge
semantics
Packet enhanced iSLIP scheduler
Total SRAM 24.6 Mbits
 Distinct unicast and multicast schedulers Gates 12.4 Million
 Eight classes of service Transistors ~200 Million
Metal Layers 7
Egress buffer credits
Signal Pins 1286
DWRR class of service
SerDes 232 @ 3.75Gbps
DWRR ingress interface
Unified Port Controller
 Media Access Controllers
1/10G Ethernet and 1/2/4G Fibre Channel
 Packet Buffering and Queuing
Total of 1.875 MBytes used in four slices
 Forwarding Controller
Ethernet, Fibre Channel
Total SRAM 35 Mbits
Layered policy engine
Total TCAM 1 Mbit
 Four data path slices Logic Gates 18 Million
One 1/10G Ethernet or two 1/2/4G Fibre Channel ports Transistors ~300 Million
Connects to one Altos port Metal Layers 7
All switching done in Altos crossbar Total Pins 900
480 KBytes of buffering SerDes 32 @ 3.75Gbps
Switch ASIC Architecture 10GE LAN Uplink
Fibre Channel
SAN Uplinks
1/10GE Attached Server
SAN B
Transceiver Transceiver
XAUI – 10 Gbps 1/2/4G Fibre Channel XAUI – 10 Gbps
4 @ 3.125G 1 @ 1.0625/2.125/4.25G 4 @ 3.125G
FC FC
1/10G MAC 1/10G MAC
MAC MAC
Forwarding Forwarding Forwarding

Parsing & Parsing & Parsing &
Editing ? Editing Editing ?
Virtual Virtual Virtual

Queues Queues Queues
Packet Egress Packet Egress Packet Egress
Buffer Queues Buffer Queues Buffer Queues
Slice 1 Slice 2 Slice 3 Slice 4 Slice 1 Slice 2 Slice 3 Slice 4
Unified Port Controller Unified Port Controller

4 @ 3.75G – 12Gbps 4 @ 3.75G – 12Gbps 4 @ 3.75G – 12Gbps
Fabric Buffer Fabric Buffer Fabric Buffer

Unicast and
Multicast
Schedulers
58 source busses in total
Unified Crossbar
Presentation_ID© 2006 CiscoFabric
Systems, Inc. All rights reserved. Cisco Confidential 17
Switch Fabric Data
Path
Data Path Deep Dive
10GE Attached Servers
 Media Access Controllers

1/10G MAC 1/10G MAC
 Crossbar operation
Parsing &
Forwarding
Parsing & Unicast
Editing ? Editing
Multicast
Virtual
Queues
Virtual
Queues  Latency
Packet Egress Packet Egress
Buffer Queues Buffer Queues
Port 1 Port 4
Gatos
Fabric Buffer Fabric Buffer

Unicast and
Multicast
Schedulers

Altos
Media Access Controllers
 Each Unified Port Controller slice has…
One 1 Gigabit Ethernet MAC
One 10 Gigabit Ethernet MAC
Two 1/2/4 Gigabit Fibre Channel MACs
 Two of the slices in each Gatos have an 802.1AE LinkSec

encryption engine
 Integrated Flow Control handling
Ethernet – 802.3X “PAUSE” and Cisco Priority Flow Control
Fibre Channel – BB_credits
Crossbar Overview
 Tightly coupled scheduler and crosspoint
20% link speedup
12 Gbps
 Unicast Scheduler
Virtual Output Queuing
3x fabric speed up
3 crosspoints
Multiple frames transferred per scheduling event
“Superframing”
 Multicast Scheduler
System Class queuing
Separate crosspoint
Unicast Virtual Output Queuing
Egress Port
VOQ 1
Packet
Buffer
VOQ N
Ingress Port
Scheduler
Q1 Q1
Egress
Queue
Q8 Q8
Packet
Buffer Q1 Crossbar
Q8 Fabric VOQ 1
Q1 Packet
Egress
Buffer
Queue
Q8 VOQ N
Q1
Egress
Q8 Queue
Egress Port
Day in the Life of a Unicast Frame
Ingress Port Egress Port
VOQ 1 Scheduler VOQ 1
3
1 Packet
Buffer 4 Crossbar Packet
8 Buffer
VOQ N Fabric VOQ N
2 5 7
Q1
6 Q1
Egress Egress
Queue Q8 Egress Fabric buffer Q8 Queue
1. Frame arrive into Packet buffer

2. Frame pointer posted to Virtual Output Queue
3. VOQ posts request to Scheduler
4. Scheduler arbitrates and grants access
5. Frame sent to Fabric Buffer
6. Fabric Buffer sends to egress queue
7. Egress port sends frame on wire
8. Egress buffer indicates freed buffer resources
Scheduler overview
Accept
VOQs IF1
Egress Ingress
Scheduler Scheduler
VOQs IF2
Egress Ingress
Scheduler Scheduler
VOQs IF3 Egress Ingress

Scheduler Scheduler
Request Grant
Proposal/mandate
Unicast scheduler algorithm
 Egress Scheduler
A priority is selected
Fixed priority, or
DWRR
An ingress is selected within that priority
Highest priority “current preferred” ingress is given a
“mandate”
iSLIP maximally matches remaining requesters
 Ingress Scheduler
Egress schedulers make a proposal
Ingress scheduler selects an egress
Fixed Round Robin selection
The selected Egress Scheduler updates its own “Current preferred”
In multi-pass scheduling, this step happens only for first-
pass selections
Multicast
Multicast MAC lookups
 MAC table
32K entries total (unicast, multicast, Fibre Channel)
1K entries (software setting) for multicast
 Populating multicast MAC table

IGMP snooping
Static
 Multicast MAC lookup miss

Source only multicast (for L3 multicast)
Forward frame to interfaces linked to Multicast Routers
Learned via PIM snooping
Flooding (for L2 multicast)
Multicast, fabric replication
Ingress Fabric Egress

Mcast
U-VOQ
U-VOQ
A
BB
BM U-VOQ
U-VOQ
Mcast Mcast
Ucast
U-VOQ
U-VOQ
Mcast A A
B CC
C M-VOQ
M-VOQ AA
Mcast
Use cases
• Ethernet multicast
Multicast Class Queuing
Port 2
MC Ingress
Queues
Priority
Queues
Egress
Priority
Port1 Port3
MC Ingress MC Ingress
Priority Priority
Queues Queues
Egress Egress
Priority Priority
Queues Queues
MC Ingress
Port 4
Queues
Queues
Priority
Priority
Egress
Multicast Scheduling Algorithm
…..
DWRR priority Ingress 3

selection Scheduler
Request vector Proposal vector

Match
Ingress selected
Preferred ingress = X
Global RR pointer
Egress
2 1 Egress
Egress
1
2 Egress
Egress 2
3 Egress
Egress
57
58
…
Scheduler Scheduler
Scheduler Scheduler
Scheduler Scheduler
Scheduler
Multicast Egress Multicast Egress Multicast Egress

Fabric Buffer Buffer Fabric Buffer Buffer Fabric Buffer Buffer
Free? Free? Free? Free? Free? Free?
Latency
10GE Attached Servers  3.2 µsec port-to-port
First-In-First-Out
Full featured forwarding
 6.7 µsec kernel to kernel
1/10G MAC 1/10G MAC Stateless offloads
no DDP
Parsing &
Forwarding
Parsing &
1.4 µsec host send
Editing ? Editing
2.1 µsec host receive
Increases with OS, interrupt,
Virtual Virtual and transport overheads
Queues Queues
Packet
Buffer
Egress
Queues
Packet
Buffer
Egress
Queues  Socket layer app-to-app
Port 1 Port 4 Linux 2.6
Gatos Raw – 10.1 µsec
UDP – 11.2 µsec
Fabric Buffer Fabric Buffer TCP – 11.8 µsec
Unicast and
Multicast
Schedulers

Altos
Lossless Data Path
QoS Processing Flow
IfIf buffer VoQs for unicast
buffer usage
usage crosses
crosses threshold:
threshold:
Central
•• Tail
Tail drop
drop for
for droppable
droppable class
class (8 per egress port) Scheduler
Trust
Trust CoS
CoS •• Assert
Assert PFC
PFC signal
signal to
to MAC
MAC
L2/L3/L4
L2/L3/L4 info
info with
with ACL
ACL for
for no-drop
no-drop system
system class
class
STOP
Per-class
Per-class
MAC Traffic
Traffic Ingress
Ingress MTU
MTU
MAC Buffer
Buffer usage
usage
Classification
Classification policing
policing checking
checking Monitoring
Monitoring 8 muticast VoQs
Truncate
Truncate or or drop
drop
packets
packets ifif MTU
MTU is
is violated
violated
PFC ON/OFF signal

Egress Queues
Egress
Marking
Marking Scheduler
Mark
Mark packet
packet with
with Strict
Strict priority
priority++
CoS
CoS value
value DWRR
DWRR scheduling
scheduling
Class Based Data Path
 Different classes of traffic require different treatment, e.g.
FC class of traffic requires lossless or no drop treatment
Market Data Ethernet class traffic may also require no drop
Remaining Ethernet Data may only require best effort
 Nexus 5000 data path resource and features are all per class
based; for example,
Per class VOQs and egress queues, buffers, MTU, drop behavior
 Per Class behavior should be consistently configured system wide
and network wide
System Class
 Nexus 5000 supports Modular QoS CLI (MQC) for all QoS configuration
 System is a new target introduced at the global cfg level
System classes are instantiated within a system policy
System policy is a service-policy attached to the ‘system’ target
 At FCS, parameters configurable under system class:
MTU
Drop, no Drop
 At ingress, packets are classified into a system class
At FCS, classification can be based on .1p or interface
 Once classified, this class assignment travels with the packet through the entire system to
select per class treatment at every step
 At Egress, 802.1p rewrite is supported. 802.1p value can then be consistently used throughout
the network to select the same system class treatment
PFC and BB_Credits
 IEEE 802.3x Pause provides no drop flow control

similar to BB credits for FC
 Priority Flow Control is a finer grained mechanism of

flow control over standard pause or link level BB credits
 Priority Flow Control uses .1p CoS value mapping to a
system class to send appropriate pause to previous hop
Priority Flow Control
Priority based Flow Control
• Enables lossless behavior

for each class of service
• PAUSE sent per priority
when buffers limit exceeded
Priority based bandwidth management
Priority based
Bandwidth Management
• Enables Intelligent sharing of

bandwidth between traffic classes
control of bandwidth
• 802.1Qaz Enhanced Transmission
Forwarding
Data Path
10GE Attached Servers
1/10G MAC 1/10G MAC
Forwarding
Parsing & Parsing &
Editing ? Editing
Virtual Virtual
Queues Queues
Packet Egress Packet Egress
Buffer Queues Buffer Queues
Port 1 Port 4
Gatos
Fabric Buffer Fabric Buffer

Unicast and
Multicast
Schedulers

Altos
Forwarding Pipeline
Parsed Packet
Vlan Translation  Wire rate “fixed” latency

Collect Interface Table (4K)
Virtual Interface
Configuration and
Table (512)
State Vlan State Table  Parsed frame fields,
(1K)
configuration, and control
Determine
Fibre Channel
Switch Table (4K) plane state are evaluated to
Destination
(ingress only) Multicast Vector
determine destination(s)
Station Table Table (4K)
(16K)
 Policy engine filters based on
Ethernet
Learning configuration, bindings, and
RBACL Label Table layered ACLs
(2K)
ACL Search Engine

(2K)
Policy Enforcement
Binding Table
(2K)
 Layered equal cost multi path
Zoning Table
expansion
(2K)
Fibre Channel
Multipath
Fibre Channel PortChannel Table
Multipath Table (1K)
Expansion
(ingress only)
(16) EtherChannel/PortChannel
Editing Instructions &

Virtual Output Queue List
Parsing ethernet IP packets
Destination address
Source address
Ethertype = .1Q CoS d VLAN
Ethertype = 2
Ver IHL TOS Total len

Identification Flg Frgm offset che
ck
TTL Proto Header cksum che sum Parsed Packet
Source address
ck Vlan
Translation
Virtual
Table (4K)
Destination address Interface Table
(512)
Interface State
Vlan State
Table (1K)
IP options Fibre Channel

Switch Table
Forwarding (4K)
(ingress only) Multicast
Vector Table
Src port Dst port Station Table
(16K) (4K)
Seq number Ethernet

Learning
Ack number RBACL Label
Table
Hdr len Flags Win size ACL TCAM Policy
(2K)
Binding Table
(2K) Enforcement (2K)
Cksum Urgent ptr Zoning Table
(2K)
Fibre Channel Multipath PortChannel

TCP options and data FC Multipath Table Expansion Table
Sc (1K)
(ingress only) (16)
hec
FCS k Virtual Output
Queue List
Parsing FCoE packets Fibre Channel frames are
FCoE encapsulated prior to
forwarding
Destination address
Source address
Ethertype = .1Q CoS d VLAN
Ethertype = FCoE Ver
Reserved
SOF Parsed Packet
Vlan
r_ctl d_id Virtual
Translation
Table (4K)
Interface Table Interface State
cs_ctl s_id (512) Vlan State
Table (1K)
type f_ctl Fibre Channel
Switch Table
seq_id df_ctl seq_cnt Forwarding (4K)
(ingress only) Multicast
Vector Table
ox_id rx_id Station Table
(16K) (4K)
Parameters Ethernet
Learning
RBACL Label
Table
(2K)
Policy
Payload ACL TCAM
(2K) Enforcement
Binding Table
(2K)
CR
Cc Zoning Table
(2K)
hec
CRC k Fibre Channel Multipath PortChannel
FC Multipath Table Expansion Table
EOF Reserved
Sc (1K)
(ingress only) (16)
hec
FCS k Virtual Output
Queue List
Unified Fabric & FCoE
FC over Ethernet (FCoE)
FCoE Benefits
 Mapping of FC Frames over  Fewer Cables
Both block I/O & Ethernet traffic co-exist on
Ethernet same cable
 Fewer adapters needed
 Enables FC to Run
 Overall less power
on a Lossless
Ethernet Network  Interoperates with existing SAN’s
Management SAN’s remains constant
 No Gateway
Ethernet
Fibre
Channel
Traffic
FCoE Standard Specification Status
FCoE Frame Format
 An extension of FC over Bit 0 Bit 31
lossless Ethernet Destination MAC Address
 FCoE Standard Specification Source MAC Address
in ANSI INCITS FC T11.3 IEEE 802.1Q Tag

ET = FCoE Ver Reserved
Frame Format agreement Aug. Reserved
2007 Reserved
Reserved SOF
Addressing scheme ratified
Feb. 2008 Encapsulated FC Frame
(Including FC-CRC)
Target completion 2H08
EOF Reserved
24 companies behind the FCS
standard
Byte 0 Byte 2197

Ethernet
Header
Header
Header
FCoE
CRC
EOF
FC Payload
FCS
FC
I/O Consolidation
Today I/O Consolidation with FCoE
 I/O consolidation Phase 1  Today:
(Mid 2008) Parallel LAN/SAN Infrastructure
LANReduction ofSAN A adaptersSAN B
server LAN
Inefficient SAN B
SAN A Infrastructure
use of Network
Simplification of access layer & cabling 5+ connections per server – higher adapter
and cabling costs
Gateway free implementation – fits in
Adds downstream port costs;
installed base of existing LAN and
cap-ex and op-ex
SAN
Each connection adds
L2 Multipathing Access – Distribution additional points of failure in
Lower TCO the fabric
Fewer Cables Longer lead time for server provisioning
Nexus 5000
Multiple fault domains – complex diagnostics
Investment Protection (LANs and
SANs) Management complexity
Consistent Operational Model
Enhanced Ethernet and FCoE Ethernet FC

Nexus 5000: I/O Consolidation
LAN SAN A SAN B LAN SAN A SAN B
8 4
2
2
• Half the Cables

• Half the Adapters
• Power & Cooling
Savings
• Consistent
Nearly twice Management
the Cables
16 Servers Enet FC Total 16 Servers Enet FC Total
Adapters 20+ 20 40* Adapters 20 0 20
Switches 2 2 4 Switches 2 0 2
Cables 40 40 80 Cables 40 0 40
Mgmt Pts 2 2 4 Mgmt Pts 2 0 2
Menlo: I/O Consolidation Network
Adapter
 Off the shelf NIC and HBA ASICs
from: Qlogic, Emulex 10GE/FCoE
Dual 10GE/FCoE ports
 Support for native drivers
and utilities
Customer certified stacks
 Replaces multiple adapters
per server
 Consolidates 10GE and FC
on a single interface
 Minimum disruption in existing 10GE FC
customer environments
 Supports PFC & DCBX
 Linux (SLES & Redhat) and
Windows versions
PCIe Bus
 June 2008 availability
Menlo ASIC
Cisco/Nuova designed multiplexer and FCoE offload protocol engine

Nexus 5000 Architecture

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Nexus 5000 Architecture

Загружено:

Авторское право:

Доступные форматы

Nexus 5000 Architecture

 System Hardware Overview

LAN LAN LAN

Eco System Partners

28-Port 1RU Switch 56-Port 2RU Switch

Eco System Partners

Support for 1 GE Support for crypto

10/100/1000 Expansion Power

Cables connect in the rear for ease of server wiring

Replaceable components on the front for easy access

Maximum Power – 750W

Fully redundant, load sharing and hot swappable

Ethernet Six 10G Ethernet

Four 10G Ethernet

Max RPM - 12K

SFP+ USR MM OM2 10m

SFP+ SR MM OM2 82m

Cat6 55m ~8W 2.5s

SFP+ SFP+ SFP+ SFP+ SFP+ SFP SFP

Intel 3100 PCI Controller

Unified Unified Unified PCIe

CPU 1.66 GHz Intel LV Xenon - LF80538KF0281M

IO Chip Set Intel 3100 South Bridge for embedded applications

DRAM 2 GBytes of DDR2 400 (PC2 3200) in two DIMM slots

Program Store 1 GBytes of USB based (NAND) Flash

Boot/BIOS 2 Mbytes of EEPROM with locked recovery image

On-board Fault Log 64 MBytes of Flash for failure analysis

NVRAM 2 Mbytes of SRAM – Syslog and licensing information

Secure Keystore Renesas AE46C1 – Credentials and secure RNG

Management Interfaces RS-232 console port – console0

 Eight classes of service Transistors ~200 Million

 Four data path slices Logic Gates 18 Million

Connects to one Altos port Metal Layers 7

All switching done in Altos crossbar Total Pins 900

480 KBytes of buffering SerDes 32 @ 3.75Gbps

Forwarding Forwarding Forwarding

Virtual Virtual Virtual

Slice 1 Slice 2 Slice 3 Slice 4 Slice 1 Slice 2 Slice 3 Slice 4

Unified Port Controller Unified Port Controller

Fabric Buffer Fabric Buffer Fabric Buffer

58 source busses in total

 Media Access Controllers

Fabric Buffer Fabric Buffer

58 source busses in total

 Two of the slices in each Gatos have an 802.1AE LinkSec

1. Frame arrive into Packet buffer

VOQs IF3 Egress Ingress

 Populating multicast MAC table

 Multicast MAC lookup miss

Ingress Fabric Egress

DWRR priority Ingress 3

Request vector Proposal vector

Multicast Egress Multicast Egress Multicast Egress

58 source busses in total

PFC ON/OFF signal

 IEEE 802.3x Pause provides no drop flow control

 Priority Flow Control is a finer grained mechanism of

• Enables lossless behavior

• Enables Intelligent sharing of

1/10G MAC 1/10G MAC

Fabric Buffer Fabric Buffer

58 source busses in total

Vlan Translation  Wire rate “fixed” latency

ACL Search Engine