CFP380 StudentGuide

Internal Use Only
Internal Use Only
Corporate Headquarters - San Jose, CA USA

T: (408) 333-8000
info@brocade.com
European Headquarters - Geneva, Switzerland

T : +41 22 799 56 40
emea-info@brocade.com
Asia Pacific Headquarters - Singapore

T : +65-6538-4700
apac-info@brocade.com
© 2010 Brocade Communications Systems, Inc. All Rights Reserved.
Brocade, the Brocade B-weave logo, Fabric OS, File Lifecycle Manager, MyView, Secure Fabric OS,
SilkWorm, and StorageX are registered trademarks and the Brocade B-wing symbol and Tapestry are
trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries.
FICON is a registered trademark of IBM Corporation in the U.S. and other countries. All other brands,
products, or service names are or may be trademarks or service marks of, and are used to identify,
products or services of their respective owners.
Notice: This document is for informational purposes only and does not set forth aniy warranty,
expressed or implied , concerning any equipment, equipment feature, or service offered or to be offered
by Brocade. Brocade reserves the right to make changes to this document at any time, without notice,
and assumes no responsibility for its use. This informational document describes features that may not
be currently available. Contact a Brocade sales office for information on feature and product availability.
Export of technical data contained in this document may require an export license from the United
States government.
Revision: May, 2010
CFP 380
CFP380 Internal Use Only Course Introduction
Revision 0110 1- 1
Course Overview
• CFP380 is a blended learning class based on Fabric OS v6.3.0 that
combines instructor-led training and web-based training (WBT) eStudy
materials and consists of:
- 3 days of Instructor-led training for more hands-on time to reinforce all blended
learning (web-based and instructor-led)
- 4 hours of Web-based training eStudy materials provide additional material that
you can access after this class is complete
• This course, along with Virtual Fabric training will help you prepare for the
Brocade Certified Fabric Professional (BCFP) certification exam
#143-070
• The CFP 380 blended learning course will NOT cover all of the BCFP
content. Candidates interested in preparing for the BCFP exam should
also take:
- AFS141 Introduction to Brocade Virtual Fabrics web-based training (6 hours
WBT)
C 2009 Brocade Commun ca1 ans Systems Inc 2

Al I R ghTS Reserved
CFP380 web-based training does not have to be completed before the instructor-led
training
Revision 0110 1- 2
Course Objectives
After completing this course, attendees should be able to:
• Implement Brocade Fibre Channel to Fibre Channel Routing (FC-
FC Routing)
• Discuss Brocade Fibre Channel over Internet Protocol (FCIP)
• Configure Brocade Adaptive Networking
• Discuss virtual connectivity using NPIV and Brocade Access

Gateway
• Implement Fabric OS security policies
• Manage advanced features using DCFM

~
0
0
The information on the slide above reflect the objectives of both the instructor-led
training section and the web-based training section of the CFP380 blended learning
course.
The CFP380 web-based training course objectives are noted with (WBT).
Revision 0110 1- 3
Course Objectives (cont.)
• Identify Brocade switch and director hardware platforms (WBT)
• Describe Fabric OS long distance options (WBT)
• Describe Fabric OS interoperability options (WBT)
~
0
0
The information on the slide above reflect the objectives of both the instructor-led
training section and the web-based training section of the CFP380 blended learning
course.
The CFP380 web-based training course objectives are noted with (WBT).
Revision 0110 1- 4
Instructor-Led Course Agenda

• Module 1 - Course Introduction
• Module 2 - FC-FC Routing Theory
• Module 3 - FC-FC Routing Administration
- Lab: FC-FC Routing
• Module 4 - Fibre Channel over IP (FCIP) Theory
• Module 5 - Fibre Channel over IP (FCIP) Administration
- Lab: FCIP Interactive Demo
• Module 6 - Adaptive Networking Traffic Management
- Lab: Adaptive Networking Traffic Management
• Module 7 - Adaptive Networking Fabric Profiling
- Lab: Adaptive Networking Fabric Profiling

This course also includes Web Based Training components.
Revision 0110 1- 5
Instructor-Led Course Agenda (cont.)

• Module 8 - Virtual Connectivity - NPIV and Access Gateway
- Lab: Access Gateway
• Module 9 - Fabric Security
- Lab: Security

This course also includes Web Based Training components.
Revision 0110 1- 6
Web-Based Training Agenda

• Switch Hardware
- Identifies the B-Series family of Fibre Channel switches and adapters
including FCoE/CEE and extension products
- Describe other FC related hardware including cables and SFPs
• Director and Backbone Hardware
- Identifies the B-Series family of Fibre Channel Directors and
Backbones including FCoE/CEE and extension blades
• FC Long Distance Solutions
- Describe the use of each long distance mode
- Differentiates long distance hardware such as DWDM and LW SFPs
• FC Interoperability
- Describe interoperability options available between B-Series Fabric OS
and M-Series M-EOS

The scope of this course does not cover th,e FCoE related products. For
information regarding the FCoE family of products, refer to the Brocade FCoE 101
Introduction to Fibre Channel over Ethernet (FCoE) course.
Revision 0110 1- 7
Course Prerequisites
• Before taking this course, attendees should have completed:
- 6 months work experience associated with SCSI storage and SANs
- Introduction to Fibre Channel Concepts course or equivalent
knowledge
- Introduction to L2 Administration and Theory course or equivalent
knowledge

Revision 0110 1- 8
What's Your BCFP Study Plan?

• To be well-prepared for the BCFP exam, your study plan should
include the following steps:
- Understand lecture material and steps performed in the lab
- Understand AFS141 Introduction to Brocade Virtual Fabrics web-based
training (WBT) material (Not included in this course1 )
- Take the free online knowledge assessment (KA): CFP380-KA
- Get some hands-on time with your equipment
- Heavily review the BCFP in a Nutshell self-study guide
•The guide addresses exam topics not in the course
• We strongly recommend using the Nutshell guide
- Attend a BCFP in a Nutshell Virtual Classroom Training Event
C 2009 Brocade Co-nrn.J"'I cat ans Sys!ems Inc 9

All R1gn1s Reserved
Footnote 1: Remember, the CFP 380 blended learning course does NOT cover all
of the BCFP content. Candidates interested in preparing for the BCFP exam should
also take the AFS141 Introduction to Brocade Virtual Fabrics web-based train ing
which must be purchased separately.
Revision 0110 1- 9
Brocade Certification and Accreditation Program

• Recognizes your expertise in implementing industry-leading
technologies
- SAN Certifications: Professional, Administrator, Designer, Manager,
FICON Architect
- SAN Accreditations: Server Connectivity, Technical Support
- IP Network Certifications: Network Engineer & Professional, Layer 4-7
Engineer & Professional
- IP Accreditations: lnternetworking
• Learn more about the program at our website:
http://www.brocade.com/education/certification-accreditation
c 2009 Brocade Communica11ons Systems Inc 1O

A ll R ights Reserved
Brocade Education website:
Revision 0110 1 - 10
Registering for a Certification Exam
PEARSON
'\J\J~
• Pearson VU E is our chosen certification test delivery vendor. They
operate 5000+ testing centers worldwide. To register for an exam
or locate the nearest testing center:
Visit http://www.pearsonvue.com/brocade
Call 866-361-5817 toll-free in North America
Visit http:l/www.pearsonvue.com for other contact numbers worldwide
(some locations may not have toll-free numbers)

In countries where English is not the primary language, examinees are given an additional
30 minutes. The following is a list of countries where VUE considers English to be the
primary language: Australia, Belize, Bermuda, Canada, Ireland, New Zealand, South Africa,
the United Kingdom and the United States.
No student may take the exam more than 2 times in a two week period. Pearson VUE
accepts many of the major world currencies. All examinees are required to accept a non-
disclosure agreement. This agreement means the examinee will not discuss or disclose any
of the questions or exam contents. Failure to comply with the agreement may result in
forfeiture of certification status and benefits.
Pearson:
Visit
Call 866-361-5817 toll-free in North America
Visit for other contact numbers worldwide (some
locations may not have toll-free numbers)
Registering for an Accreditation Exam
Webassessor
Online Secured Testing Technology
• Kryterion's Webassessor tool is our chosen method for delivering

web-based accreditation exams. To register for an exam:
Visit
https://www.webassessor.com/wa.do?page=publlcHome&branding=BROCADE

No student may take the exam more than two times in a two week period. Kryterion accepts
credit cards for payment. All examinees are required to accept a non-disclosure agreement.
This agreement means the examinee will not discuss or disclose any of the questions or
exam contents. Faillure to comply with the agreement may result in forfeiture of certification
status and benefits.
Webassessor:
g-BROCADE
Going Green and Taking Notes

• Courseware materials are
}
distributed in electronic format
• Adobe Reader is required to
AddStldty!fole
ShowC-t&~Tool,!ar
Clrl-16 I
; hw,._.........
open the included PDF
courseware and lab exercise
files
• To take notes in the PDF files
we have enabled commenting

Footnote 1: Use "CTRL" and "6" as a shortcut to create PDF notes.
Training Facility and Training Policies

• Facility information and lab policies
• Start and stop times
• Regular breaks
• Please ...
- Be open to see what will happen
- Have fun
- Set your cellular telephone and digital pager to "silent" or "vibrate"
• Please feel free to share non-compromising SAN experiences
Introductions
• Please take a moment and share with us:
- Your name
- Your employer
- Your location
- Your background with switches, SANs, Fibre Channel, storage,
systems, and networking
- Your goals in attending this training course
- An interesting fact about yourself
CFP380 Internal Use Only FC-to-FC Routing Theory
Revision 0110 2- 1
Objectives
• After completing this module and associated lab exercises,
attendees will be able to:
- Describe Fibre Channel-to-Fibre Channel routing (FC-FC routing) and
related hardware
- Discuss Fibre Channel-to-Fibre Channel routing terminology, theory

and concepts
- Compare physical frame flow versus logical frame flow between routed
fabrics
- Identify additional FC-FC routing features

Revision 0110 2-2

FC-FC Routing
• Fabric OS provides L3
Fabric OS Fabric OS
Fibre Channel-to-Fibre Fabric 1 Fabric 2
Channel routing (FC-FC
Routing) between
fabrics
• Allows device access

between two or more
fabrics without merging
the fabrics
• FC-FC routing is
supported between the
following fabric types: M-EOS
Fabric 3
- Fabric OS-to-Fabric OS
- Fabric OS-to-M-EOS
- M-EOS-to-M-EOS
C 2009 Brocade Co'Timun ca1 ans Systems nc 3

Al 1R ghts Reserved
Revision 0110 2- 3
FC-FC Routing (cont.)

Fabric OS
• Physical connectivity Fabric 1
is accomplished
through the use of a
Fibre Channel router
(FC router)1
• Logical connectivity
is accomplished
through the use of
Logical Storage Area
Networks (LSANs ),
by creating uniquely
named zones called
"LSAN zones" •Host WWN 10:00:00: • Host WWN 10:00:00:

Al 1R ghts Reserved
FC-FC routing was introduced in Fabric OS v5.1 on the Brocade 7500 and Brocade
FR4-18i blade . FC-FC routing is also known as, FCRS (Fibre Channel Routing
Service), FC-to-FC routing , FCR, FC routing and routed SANs.
Footnote 1: Includes implementing and configuring the underlying physical

connectivity between the fabrics that will share devices using EX_Ports/lnter Fabric
Links (IFL).
The FC router in effect enforces an implied "DENY_ALL", and the administrator
must configure the "PERMIT" entries (ACLs) via LSAN zoning.
Revision 0110 2-4

BROCADE
EDUCATION SOLUTIONS
Revision 0110 2- 5
FC-FC Routing Terminology

• Edge Fabric : A fabric that is attached to one or more FC router ports
- Maintains its own fabric settings
- Can communicate with devices in other edge fabrics and devices in the
backbone fabric with the use of LSAN zones
• Backbone Fabric (BB): Edge Fabric 1 Edge Fabric 2
The interconnection
point for edge fabrics,
containing at least one
FC router
- The simplest backbone
fabric is a one FC
router fabric, with no
direct or indirect E Port
connectivity to other
FC routers Simple Backbone Simple Backbone
Fabric 1 Fabric 2

Al 1R ghts Reserved
Above, each router represents a "simple backbone fabric" in the routed fabric. There
is no E_Port to E_ Port connectivity between the two routers
Revision 0110 2-6

FC-FC Routing Terminology (cont.)

• Complex Backbone: Allows scalable routed fabrics by allowing multiple
FC routers to be networked together using E_Ports in the backbone fabric
- All ISLs that participate
Edge Fabrics 1 - 7
in the backbone fabric
are configured as
E Ports
- Any combination of
Fabric OS switches
and routers can make
up the backbone
Complex Backbone 1 Complex Backbone 2

Revision 0110 2- 7

• Fabric ID (FID): Uniquely identifies each fabric participating in routed
fabrics1
- Every edge and
Edge Fabric 1 Edge Fabric 2
backbone fabric
requires a unique
FID
- A total of 128 FIDs

can be assigned
(1-128)
( BB FIO = 100 ) ( BBF10 = 120 )
- Both backbone and

edge FIDs .are
configured on the Simple Backbone Simple Backbone
FC router2 Fabric 1 Fabric 2

Al 1R ghts Reserved
Above, each router represents a "simple backbone fabric" in the routed fabric. There
is no E_Port to E_ Port connectivity between the two routers.
Footnote 1:
The FC router backbone fabric and Virtual Fabric base switch share the same FID.
Edge fabrics use separate FIDs for VF and FC routing. The FC router in the
backbone assigns an edge fabric FID with the edge fabric having no awareness of
the FC routing FID assigned to it while a VF enabled edge fabric would have the
FID configured locally within the edge fabric.
Footnote 2:
Backbone fabric IDs (FID) can be administratively configured using
fcrconfigu r e . It is necessa ry to invoke fosconfig --disable fer prior to
fc r conf igu r e and the switch must be disabled using swi tch d is a b le .
Edge fabric FID values can only be viewed from the swi tchshow output of the
router (not on the edge fabric). Each EX_Port will show the assigned FID value.
Revision 01 10 2-8
Terminology Review (cont.)
The following output does not correspond to the diagram.

router : admin> switchshow
<Truncated Output>
11 id N2 Online EX PORT 10 : 00:00:60 : 69 : 80 : 5b : 5f
"RSL DCX " (fabric id = 1)
12 id N2 Online EX PORT 10 :00: 00 : 60 : 69 : 80 : 5b : 5f
" RSL DCX " (fabric id = 1 )
13 id N2 Online EX PORT 10 :00: 00 : 60 : 69 : 51:3a : bf
" RSL 5300 " (fabric id = 11)
14 id N2 Online EX PORT 10 :00: 00:60 : 69 : 51:3a : bf
"RSL 5300 " (fabric id = 11)
To set the FID for an edge fabric use port cf gexport :
r oute r : adrnin> portcfgexport

Usage: portcfgexport <port#> [ -a ad.min] [-f fabricid
(1. . 128) ]
<Truncated Output>
*Set t he preferred FID value for each Edge Fabr ic by passing

the -f toggle:
Revision 0110 2-9


• EX_Port: A type of E_ Port used to
( FID = 7 )
connect an FC router port to an Edge Fabric
edge fabric without merging the two 1

- EX Ports on a router connect to
E_ Ports in an edge fabric
- Use the po r t cf g expo r t command
to configure router EX_Ports2
Backbone Fabric
• Inter-Fabric Link (IFL): IFLs are ~· ( FID = 100) ~

~
DID =4
configured/enabled between the ( DID = 5
edge fabric E_ Ports and the FC ~

j ~ EX_Port
router backbone fabric EX Ports3 \Q E_Port

From the point of view of a switch in an edge fabric, an EX_Port is virtually

indistinguishable from any other Brocade E_Port. It follows all applicable FC E_Port
standards and thus a switch connecting to an EX_Port runs in standard E_Port
mode.
Recall that nothing is required to change in the existing edge fabric (SAN Islands) to
allow FC routing to occur. CU commands such as fabricshow display all domains
participating in the edge fabric .
Footnote 1:
When using FCIP, the port is called a VEX_Port (Virtual EX_Port). A VEX_Port
communicates with a standard VE_Port (Virtual E_Port) on the other side of an IP
Network.
Footnote 2:
router : adrnin> portcfgexport
Usage: portcfgexport [SlotNumber/ ] PortNumber
[-a 1-enable 2-disable] [ -f fid( l. . 128)]
[- r r a tov] [-e e_d_tov] [-d domain]
[ -p 0-native 1-core 2-extended edge]
[-m 0-Brocade 1-0pen 2-McDATA Fabric , 3-McDATA Fabric
Legacy ]
[-t 1-Enable 2-Disable]
The portcfgexport command is used to configure a FC port as an EX_Port. The
command has a single required argument - the port on which the command is to
operate - and several optional parameters:
-a : Sets the port to be enabled (1) or disabled (2).
-f : Fabric ID (1 -128) for the edge fabric attached to this port; default value is
the port number divided by 3 plus 2 rounded down.
-r : R_A_ TOV used for port negotiation, in msecs (2000 - 12000). Default:
10000.
-e: E_D_TOV used for port negotiation, in msecs (1000 - 60000). Default:
2000.
-d: Preferred front phantom domain ID (1-239). Default: 160.
-p: Port ID format of the edge fabric (1-core, 2-extended edge, 3-native).
Default: Core PIO.
-t : Auto-negotiate fabric parameters (1-enable, 2-disable). Default: enabled .
The command displays the currently configured values for the specified port when
no optional parameters are specified.
Footnote 3:
The addition of an IFL (creation of an EX_Port) between the router and the edge
fabric will not cause the fabrics to merge.
When all IFLs are removed, FOS detects this condition and forwards an RSCN to
the applicable remote fabrics.
The removal of an IFL will not introduce a fabric reconfiguration on the edge fabric.
The Owner IFL has a special role. It is the IFL that performs the ROI (request
domain ID) forthe Translate Phantom domain when it comes online. The Owner IFL
can be viewed via the OwnerDid field in the fcrxlateconf ig command.
All exported devices from one remote routed fabric are "hanging off' just one
phantom translate domain (xd) on the local fabric. For each remote routed fabric
that shares (imports) physical devices, a separate and distinct translate (xlate)
domain (virtual domain) is created in the local fabric.
Revision 01 10 2 - 11

• LSAN: a logical storage Fabric OS
Fabnc 2
area network that spans
multiple physical fabrics
- Allowing devices in different
fabrics to communicate with
each other
• LSAN Zone: Zones that
define which devices are to
be shared between fabrics
- Defined in each fabric that
will share devices (edge or
M·EOS
backbone) Fabric 3
- Is a traditional zone created

using normal zoning tools
but uses a special naming •Host INVVN 10:00:00: • Host INVVN 10:00:00:
prefix lsan_
Al 1R ghts Reserved
An LSAN is implemented using LSAN zoning which includes devices from two or
more routed fabrics.
An LSAN zone defines device communication between autonomous SANs but only
allows designated devices in those SANs to communicate. They are defined in
each fabric, whether edge or backbone, that will share devices (devices that will be
exported/imported).
Zone names are not case sensitive, e.g. "LSAN_", or "lsan_", or "LSan_", ...
Configured the same way standard zones are and subject to normal zoning
enforcement
LSAN zones are compatible with Fabric OS and M-EOS. FC router uses LSAN
defined zones to determine which devices need to be imported (phantoms) into
which routed fabrics. LSAN zones must be configured in fabrics where the physical
devices exist. The router performs zoning enforcement for edge fabrics at the
ingress router EX_ Port.

• Front Domain (fd): A logical domain created in the edge fabric when
edge fabrics are connected to backbone fabrics1
• Translate Domain (xd): A logical domain created when routed fabrics
share devices2 Backbone Fabric
- Sharing is accomplished
through the creation and
enabling of LSAN zones
,,,,..
- This logical! domain
is where the imported
devices log1ically exist
(
fe EX_Port
jO E_POll
- - FO
- xo
• Storage WNN 20·00:00: • Host WNN 10:00·00:

Al 1R ghts Reserved
Footnote 1: Front Domain: A front domain represents the router in an edge fabric.
Front domains are not created in a backbone fabric. Instead, they are a tier domain
between the translate domains (xd) and the edge fabric. Imported devices are NOT
attached to front domains, they are attached to translate domains . FD's do not
have a scalability effect. Virtual links between front and translate domains do not
count as hops in hop-count limitations.
Footnote 2: Translate Domain: A logical domain created in routed fabrics that

share devices. They are created in edge or backbone fabrics, but only created when
physical devices in both fabrics requiring an xd are online and are part of an LSAN
zone in two or more fabrics . Only one xd exists for each remote routed fabric. Xds
exist in the backbone when devices from an edge fabric are imported into the
backbone fabric (backbone-to-edge routing). FC-NAT is used to logically "attach"
imported devices to the xd. A preferred domain ID (DID) is the requested DID but
not an insistent domain ID. The standard RDI (Request domain ID) process occurs
and if the preferred (xlate) domain ID is not already assigned to some other domain,
the principal switch will assign the requested domain ID. Otherwise, the principal
switch will reply with the next available domain ID. It seems reasonable, but not
required , to configure a preferred translate domain ID that is the same value as the
FID value of the remote edge fabric that the domain ID is representing. This would
require planning, and is more readily achievable in green-field installations.

• Exported Device: A physical device defined in an LSAN zone that
the router export out of a fabric (edge or backbone)
Backbone Fabric
FID • 100
• Imported Device: Logical Device

Imported Logical Device
A logical device Imported
X024
defined in an LSAN
zone that represents
a physical device in a ~~~·•I
different routed fabric r; ~p:1
- - FD
vv
Physical Devlee
(edge or backbone) - XO Exported
Physical Device
Exported

Al 1R ghts Reserved
The terms Export and Import are based on the view of the FC router. Physical
devices within a fabric need to be exported out of a fabric and the logical device
representing the physical device needs to be imported to the remote fabric.
BB BSl : admi n> lsanzoneshow -s

Fabr i c I D: 1 Zo ne Name : LSAN_ Edge l
10 : 00 : 00 : 05 : le : 57 : 7c : 79 EXIST
22 : 00 : 00 : 20 : 37 : dd : d9 : 29 I mported
Fabri c I D: 2 Zone Name : l san edge2
1 0 : 00 : 0 0 : 05 : l e : 57 : 7c : 79 Imported
22 : 00 : 00 : 20 : 37 : dd : d9 : 29 EXIST
Note: EXIST= local physical device to be exported and Imported= remote fabric
device that was imported.

• Fibre Channel Router Protocol (FCRP): A Brocade-authored L3 routing
protocol with two distinct components:
1. FCRP Edge
Fabric Protocol
2. FCRP Backbone
Fabric Protocol
. .
Backbone Fa bric FC Routers "talk" aaoss I Backbone Fabric
~B fabric using FCRP J
C 2009 Brocade Co'Timun ca1 ans Systems nc i 5

Al 1R ghts Reserved
In addition to FC routers being able to communicate using FCRP within the

backbone fabric, EX_Port on the router enable FCRP Edge Fabric protocol
communication across the EX_ Port IFLs, e.g. the Brocade routers communicate via
Layer 3 FCRP protocol through the edge fabrics IFLs.
BROCADE
EDUCATION SOLUTIONS
Physical Frame Flow
Backbone Fabric
FlO • 100
• EX_Port
f
0
- -
l-
E_Polt
FO
XO Edge Fabric
Edge Fabric

Al 1R ghts Reserved
Logical Frame Flow
Backbone Fabric
FID "' 100
Edge Fabric
Edge Fabric

Al 1R ghts Reserved
BROCADE
EDUCATION SOLUTIONS
Integrated Routing
• Condor2 and GoldenEye2 ASICs provide FC-FC Integrated
Routing (IR) on a per-port basis1
• An Integrated Routing license is required to allow configuration of
FC-FC routing capable ports called EX_Ports2
- License enforcement is checked on configuration and when enabling
the EX Port3
• Integrated Routing is currently supported on:4
- Brocade FC8, FX8 and FS8 blades in the DCX and DCX-4S chassis5
- Brocade 5100, 5300, 7800 and Brocade Encryption Switch (BES)
• Supports trunking on EX_ Ports
• Integrated Routing is disabled by default6

Footnote 1: The web-based training Hardware module for this class has additional
information on ASICs and hardware platforms
Footnote 2: When an FR4-18i blade is ins,erted into a DCX/DCX-48, all Fibre

Channel ports are persistently disabled. This is not the case with 8 Gbps Fibre
Channel blades.
If FR4-18i EX_Port is online, Condor2 EX_Ports cannot be configured and enabled.

The reverse is also true, where FR4-18i EX_ Ports cannot be configured and
enabled if online Condor2 EX Port are enabled on the DCX/DCX-4S chassis.
Footnote 3: If the license is removed while EX_Ports are online,

Condor2/GoldenEye2 EX_ Ports will continue to function until the next fabric rebuild ,
switch disable or port offline event.
rou t er : admin> licenseshow
I ntegrat ed Routing license
AXQYN3mMKa3DXBAmJRWfFPTa HRNtRZPFB7ZAR :
<truncated output>
Footnote 4:
Not supported on:
• DCX and DCX-4S ICL ports
• FC 10-6 1O Gbps blade
• 8 Gbps blades in the Brocade 48000
• Brocade 300
• All 4 Gbps switches and blades except the Brocade 7500 and FR4-18i
• All embedded switches both 4 and 8 Gbps
Footnote 5:
FC8 are 8 Gbps port blades, FX8 is the distance extension blade and FS8 is the
encryption blade. FC routing over FCIP ports, VEX_ Ports , is not supported on the
FX8-24 blade but is supported on the Brocade 7800.
Footnote 6:
Use the fosconf ig - - enable fer to enable FC-FC routing.
Integrated Routing Considerations

• Only Fa bric OS switches are allowed in the backbone f abric
- Backbone fabric must be set for Brocade Native Mode
(lnteropmode 0)
• Routing services must be disabled to change the backbone

fabric ID
• Integrated Routing and TopTalkers in Fabric mode are not

concurrently supported 1
- TopTalkers in F_Port mode is supported with IR
• Integrated Routing EX_ Ports are only supported in a Virtual

Fabric Base Switch with Virtual Fabrics enabled

Footnote 1: The Fabric OS To pTalkers feature will be discussed in a later module.
Other FC-FC Routing Platforms

• The 4 Gbps Condor-based 7500, 7500E and FR4-18i director
blade also support FC-FC routing 1
- No license is required, functionality is directly associated with the
hardware
- Only Fabric OS switches are allowed in the backbone fabric2

• Backborie fabric must be set for Brocade Native Mode (lnteropmode 0)
- FC-FC routing functionality is enabled by default
- EX_Ports are not supported in the Base Switch with Virtual Fabrics
enabled 3

Footnote 1: The web-based training Hardware module for this class has additional
information on ASICs and hardware platforms
Footnote 2: M-EOS switches are allowed in the edge fabric only.
Footnote 3: Only Condor2 and GoldenEye2 based EX_Ports and VEX_Ports are
supported in a Virtual Fabric Base Switch.
FR4-18i is supported in a DCX with the following restrictions:

• FR4-18i EX_ Port or Integrated Routing EX_ Port
- Cannot use both on the same chassis
-Can use both in the same Fabric
• FR4-18i VEX_Ports within the same chassis as IR EX_Ports must go to
separate edge fabrics
DCX
VEX Port FCIP Link

Edge
EX Port
Edge
Replacing FR4-18i blade with an FC8 blade retains EX_ Port configuration for the
first 16 ports and all other ports are cleared.
Configured EX_ Ports do not affect behavior. In other words, offline FR4-18i I 7500
EX_ Ports do not affect ability to configure Condor2/GoldenEye2 EX_ Ports and vice
versa. If a Condor2 blade is replaced with another Condor2 blade, all configuration
information remains.
BROCADE
EDUCATION SOLUTIONS
EX_Port Trunking
• Fabric OS v6.1 and v6.2 use master port trunking of EX_Ports
- When a master port goes offline, slave ports in the trunk go
offline/online to select the new master port1
• Fabric OS v6.3 EX_ Port support masterless trunking EX_Ports

- On master port offline, slave ports stay on line and one takes over the
responsibilities of the master port1
• Masterless EX_Port trunking considerations:

- Only supported on 8 Gbps switches and blades
- On Brocade DCX and DCX-4S, masterless EX_Port trunking is only
supported with VF enabled
- On Brocade 5300 and 5100, the masterless EX_Port trunking is
supported with VF disabled or enabled

Footnote 1: Lowest backend port number becomes new master port not the lowest
front end switch port number (normal user port). A backend port is an internal
switch port number and may be higher then the user port number.
LSAN Zone Binding

• LSAN zone binding is an optional feature that increases the FC router
scalability in very large environments
- Provides device exporting/importing control at the edge fabric level
• Without LSAN zone

binding every FC router
in the backbone fabric
Edge Fabric 1 Edge Fa bric 3
maintains the entire
LSAN zone and device
state database1
• With LSAN zone Edge Fabric2 Edge Fabric 4
LSAN Binding
binding each FC router • Edge 1 to Edge 2
• Edge 1 to Edge 3
stores only the LSAN zone • Edge 3 to Edge 4
entries for selected edge
fabrics 2
C 2009 Brocade Commun ca1 ans Syst ems Inc 27

FC Router 1 will process all LSAN zones for Edge Fabric 1, 2, and 3 but not 4. FC
Router 2 will process all LSAN zones for Edge Fabric 1, 3 and 4 but not 2.
Footnote 1:
The size of this database limits the number of FC routers and devices. Without
LSAN zone binding, the maximum number of LSAN devices is 10,000. With LSAN
zone binding, the Fe-Fe routed fabric can import more than 10,000 devices and the
backbone fabric can support more FC routers and CPU consumption by an FC
router is lower.
Footnote 2:
LSAN zone binding uses an FC router matrix, which specifies pairs of FC routers in
the backbone fabric that can access each other, and an LSAN fabric matrix, which
specifies pairs of edge fabrics that can access each other.
You set up LSAN zone binding using the fcrLsanMatr i x command. This
command has two options: - fer and - lsan . The - fer option is for creating
and updating the IFC router matrix, and the - lsan option is used for creating and
updating the LSAN fabric matrix. The FC router and LSAN fabric matrix databases
are automatically distributed to all FC routers in the backbone fabric_
router : admin> fcrlsanm.a trix --add -fer <WWNl> <WWN2>
r outer : admin> fcrlsanmatrix --add -lsan <FIDl> <FID2>
r ou ter : admin> fcrlsanmatrix --apply -all
Revision 01 10 2 - 27
LSAN Tagging
• FC routers with Fabric OS 6.2 and greater support two types of
optional LSAN tags used for special processing 1
• Enforcement tags: Only LSAN zones with names containing a

configured enforcement tag name are imported by the FC router
• Speed tag: Used when FC router needs to import targets to a
routed fabric before the host is online (e.g. boot over SAN)
• There are two parts to setting up LSAN tags:

1. FC Router: LSAN tags are configured and enforced independently
on each FC router in the backbone fabric2
2. Fabric: LSAN tags are then appended to the LSAN zone names in
the fabrics sharing devices (edge or backbone)
• Rather than "lsan_ ", LSAN zone names use "lsan_ <tagname>_ "

Footnote 1: Supported on both 4Gbps and 8Gbps router platforms running Fabric
Os v6.2 or later.
Footnote 2: LSAN tags are added, removed and viewed from the FC router using
the f cr l san command:
router : admi n> fcrlsan --help
Usage : fc rl san [ --add - enforce I - speed <tag>]
[-- remove - enforce I - speed <tag> ]
[--show - enforce I - speed I - a l l]
[--h e l p)
Create an enforcement LSAN tag:

fc rl san --add - en f orce < t ag>
Create a speed LSAN tag

fc r lsan --add - speed <tag>
Delete an enhancement or speed LSAN tag

fcr l san - -delete - enforce I speed <t ag>
LSAN Enforcement Tagging

• Only devices in locally defined enforcement tagged LSANs are imported
- Provides device exporting/importing control at the LSAN zone level
- Improves scalability and reduces FC router CPU utilization
- None defined
enforcement tag LSAN Zones
• LSAN_Enf1_F1-F2
and normal LSAN • LSAN_Enl2_F1·F3
• LSAN_Enf4_F3-F4
- _Edge Fabric 3
- Supports up to 8
enforcement tags ----~---~-
· ~_12_
. E_
n~-
. E~
n~~
per FC router
~~
Edge Fabric 2
• LSAN_Enf4_F3-F4

FC Router 1 has enforcement tags Enf1, Enf2 and Enf3 defined and will only
process LSAN zones that use the Ent1, Enf2 or Enf3 tag . FC Router 1 will only
process a subset of LSAN zones from Edge Fabric 3 and 4. FC Router 2 has
enforcement tags Enf2, Enf3 and Enf4 defined and will only processes LSANs that
use the Enf2, Enf3 or Enf4 tag1. FC Router 2 will only process a subset of LSAN
zones from Edge Fabric 1 and no LSAN zones from Edge Fabric 2.
Steps to setup this configuration:

1. On Router 1 create three enforcement tags, in this example they are called Enf1 ,
Enf2 and Enf3.
2. On Router 2 create three enforcement tags, in this example they are called Enf2,
Enf3 and Enf4.
3. In Edge Fabric 1 create LSAN zones using the three enforcement tags Enf1 ,
Enf2 and Enf3, in this example they are LSAN_Enf1_F1 -F2, LSAN_Enf2_F1 -F3
and LSAN Enf3 F1-F4.
4. In Edge Fabric 2 create an LSAN zone using the enforcement tag Enf1 only, in
this example it is LSAN_ Enf1 _F1-F2.
5. In Edge Fabric 3 create LSAN zones using the two enforcement tags Enf2 and
Enf4, in this example they are LSAN_ Enf2_F1-F3 and LSAN_Enf4_ F3-F4.
6. In Edge Fabric 4 create LSAN zones using the two enforcement tags Enf3 and
Enf4, in this example they are LSAN_ Enf3_F1-F4 and LSAN_Enf4_ F3-F4.
Revision 01 10 2 - 29
LSAN Speed Tagging

• Targets in speed tagged LSAN are imported to remote edge fabrics
as soon as they come online, independent of the on line status of
the initiator1
- Typically devices are i mported after both the remote and local devices
within an LSAN are online
- Used for boot over SAN
- One speed tag per FC router
- Can be used with standard LSAN zones or Enforcement Tag LSAN
zones Host FC Router Storage
...____, Edge Fabric 1 Edge Fabric 2

Footnote 1: Speed tags: Certain hosts are very sensitive to timeout and retry
during target discovery process. FC router tends to take a long time, more than 5
seconds, to present proxy devices and setup path for proxy devices. Due to many
constraints of hardware and protocol, FC router is unable to improve the
import/export process to satisfy those sensitive hosts. FC router treats speed
tagged LSANs differently by always importing these targets to the hosts. The status
of these targets in the speed tagged LSANs remains Imported and the name server
in the host fabric will always retain a PIO for them. This allows sensitive hosts to do
discovery faster for these targets.
Steps to setup this configuration:

1. Create a speed tag on the FC router, In this example FAST.
2. In Edge Fabric 1 create LSAN zone containing the host and target, in this
example LSAN_z onel
3. In Edge Fabric 2 create LSAN speed tag zone containing the hoist and target,
in this example LSAN_ FAST_ zonel (Target, Host)
When the target comes online in Fabric 2, the proxy target is immediately
imported into Edge Fabric 1 - no matter the host state
Revision 01 10 2 - 30
Create an enforcement LSAN tag:

fcrlsan --add - e n force <tag>
Create a speed LSAN tag

f crlsan --add - speed <tag>
Display the current enhancemenUspeed LSAN tags

f crlsan - -show - e n force I speed <tag>
Delete an enhancement or speed LSAN tag

fcrlsan - -delete - enforce I speed <tag>
Delete all enhancement or speed LSAN tags

f crlsan - -delete - enforce I speed
Enforcement and Speed Tag Considerations

• All FC routers must have Fabric OS v6.2.0 or later
• The FC-FC routing feature must be disabled before configuring
enforcement tags (not required for speed tags)
• LSAN tags are persistently saved in the configuration file
- By default no enforcement or speed tags are defined
• RASlog messages indicate the addition or removal of an
enforcement or speed tag
• Per FC router, up to 8 ,enforcement tags and 1 speed tag
- A tag name can be either enforcement or speed but not both
- Tag names can be up to 8 characters in length
• Can use LSAN zone binding along with the LSAN tagging to
achieve better scalability and performance

Additional Speed Tag Considerations

• Use the same speed tag value across all FC routers in the
backbone fabric to ensure proper handling across the routed fabric
• A total of 500 speed tag LSAN zones can be defined across the
routed fabrics
• Speed tags are only supported in fabrics where the targets are
local and the initiator is remote

FC-FC Routing with MEOS Fabrics

Fabric OS
• M-EOS edge fabric Fabric 1
support1
- M-EOS v07.00 and
higher
- No VEX Ports
- M-EOS not
supported
in the backbone
fabric
M-EOS
Fabric 3

Al 1R ghts Reserved
Footnote 1: Fabric OS 6.3 EX_Ports can also connect to edge fabrics using M-i10K
directors in McDATAOpen Mode operating in 239 Domain mode.
For the latest list of Brocade tested and approved platforms, firmware revisions, and
scalability guidelines, visit www.brocade.com.
These requirements reflect testing performed by Brocade, and may be different
from those specified by your Brocade switch provider. As always, your switch
provider sets the guidelines and definitions that you should follow.
Summary
• FC-FC routing services allows device access between two or more fabrics
without merging the fabrics
• An edge fabric is a fabric that is attached to one or more FC router ports
• A backbone fabric is the interconnection point for edge fabrics, consisting
of FC routers and perhaps L2 switches
• FID uniquely identifies each fabric participating in routed fabrics
• EX_ Port is an FC router port used to connect to an edge fabric
• IFLs are the links between edge fabric E_ Ports and the FC router
EX Ports
• An LSAN is a logical storage area network that spans multiple physical
fabrics
• LSAN zones are used to define which devices are to be shared between
fabrics

Summary (cont.)
• FC routers export physical devices from a fabric and import logical device
to a fabric
• Front domain are logical domain created in the edge fabric when
EX Ports are enabled
• Translate domain are log1ical domain created when routed fabrics share
devices by defining LSAN zones
• Increases the FC router scalability by controlling which devices are
exported and imported:
- LSAN Zone Binding at the edge fabric level
- Enforcement tagging at the LSAN zone level
• Use speed tagging when FC router needs to import targets to a routed
fabric before the host is online

BROCADE
EDUCATION SOLUTIONS
Destination Router Port Selection

• A Destination Router Port (DRP) is the EX_ Port a frame is sent to in order
to reach an edge fabric
• Pre Fabric OS 6.2 destination router ports on different remote FC routers
within a backbone are load-balanced , without regard for the underlying
transport
• In the diagram below, frames from Fabric 1 would be load balanced
between DRP1 on FCR2 and DRP2 on FCR1
Fabric2 Fabric1 ;
~-
Ex_3 e DRP2
FCIP
FCR2
BS Fabric
• Ell_P0<1
~~, '...~, VE_Pon

In the above topology, FCR2 has two DRPs (EX_2 and EX_3) to route the frames
from Fabric 1 to Fabric 2. Prior to Fabric OS release 6.2, FCR2 would use both
DRP1 (EX_2) and DRP2 (EX_3) across FCIP link to route the frames to Fabric 2 . In
Fabric OS v6.2.0 and higher, FCR2 uses DRP1 (EX_2) only, as that DRP is the only
local DRP.
Revision 01 10 2 - 39
Destination Router Port Selection (cont.)

• With Fabric OS v6.2.0+, FSPF cost is used to select the DRP
- DRPs with an FSPF cost of less than 10000 (E_Ports) are considered local
- DRPs with an FSPF cost of 10000 (VE_Ports) are considered remote1
- Remote DRPs are used only if there are no local DRPs
- Although intended for long distance or FCIP ISLs, any ISL or IFL can be
configured to affect the DRP selection
• In the diagram below, frames from Fabric 1 will only use DRP1 on IFCR2
Fabric 2 Fabric 1
Ex_3 • ORP2
/
FCR2
88 Fabric _,,,,
e EX_Poon
~~, '~-i VE_Pon

Footnote 1: Default link cost for FCIP is 10,000.
Destination Router Port Selection (cont.)

• Configure the cost metric with the f crrout erportcost (IFL) or
linkcost (ISL) commands
• The fcrfabricshow command indicates remote DRPs with an *
at the end of the line
85l : admin> fcrfabricshow
FC Router WWN : 10 : 00 : 00 : 05 : le : 41 : 59 : 81 , Dom I D: 2 , Info : 10 . 33 . 36 . 8 , "851 "
EX_Port FID Neighbor Switch Info {enet IP, WWN , name)
------~~----~--~~~;;~;~~;~-------~~~~~~~~~~~~~:~;:~~~~~~~~~::,~~ Local
FC Router WWN : 10 : 00 : 00 : 05 : le : 39 : 51 : 67 , Dom ID : 5 , Info : 10 . 33 . 36 . 96 , " FA4-18 "
EX_Port FID Neighbor Switch Info {enet IP , WWN, name)
------------------------------------------------------------------------
151 2 10 . 33 . 35 . 80 l0 : 00 : 00 : 05 : le : 38 : 0 l : e "810 3 " * ~ Remote

R11 -ST10-851 :admin> fcrfabricshow

FC RouterWWN: 10:00:00:Q5,:1e:9a:71:af, Dom ID: 98,
Info: 10.255.240.76, "R11-ST10-B51"
EX Port FID Neighbor Switch Info (enet IP, WWN, name)
3 10 10.255.240.75 10:00:00:05:1e:Ob:ba:7e "R11 -ST10-B30"
CFP380 Internal Use Only FC-to-FC Routing Administration
Revision 0110 3- 1
Objectives
- Discuss FC-FC routing implementation using CLI and DCFM
- Identify commands and DCFM tools used to verify routing and

connectivity
- Demonstrate management commands used in Brocade FC-FC routing
- Identify common troubleshooting topics related to FC-FC routing

Revision 0110 3-2

BROCADE
EDUCATION SOLUTIONS
Revision 0110 3- 3
Configuring FC-FC Routing from CLI

• Configuration example:
Edge FID 10 Edge FID 100
Domam 3 Domain 98
_....,._ex
....-...,P_orte ,.:=-...:::: ~--~~
300 5100
:: 2009 Brocade Commun ca! ens Sys'e'TIS l"IC 4

Al R1g~·s Reserved
Revision 0110 3-4

Preparing Fabrics For FC-FC Routing

• Before configuring the backbone fabric, Interoperability mode must
be disabled in the fabric1 2
BB_B5l : aclmin> i nter opmode

InteropMode : Off
usage : I nteropMode (0 1213 [ - z McDataDefaultZone ) (- s McDataSafeZone] J

0 : to turn interopMode off
2 : to turn McDATA Fabric mode on
Valid McDataDefaultZone : 0 (disabled) , 1 (enabled)
Valid McDataSafeZone : 0 (disabled) , 1 (enabled)
3 : t o turn McDATA Open Fabric mode on

Footnote 1: The swi tchshow command also displays the switch interoperability
and secure mode information.
Footnote 2: Management Server Platform database must also be disabled in the

fabric· Disabled by default for 8 Gbps platforms but older switches where enabled
by default.
Use the msplatshow and msplmgmtdeactivate commands to display and, as

needed, deactivate the Management Server (MS) platform service. Here is an
example:
7500_1 : admin> msplmgmtdeactivate
MS Platform Service is currently enabled.
This will erase MS Platform Service configuration
information as well as database in the entire fabric .
Would you like to continue this operation? (yes , y , no , n) :
[no ] y
Request to deactivate MS Platform Service in progress ..... .
*Compl eted deactivating MS Pl atform Service in the fabric !
Revision 01 10 3- 5
Preparing Fabrics For FC-FC Routing (cont.)

• Verify that the backbone and edge fabrics are not configured in
access control List (ACL) strict mode
- ACL strict mode is not supported in a FC-FC router environment
• ACL's will be covered in further detail in the security module

Use the fddcfg (Fabric Data Distribution configuration) command to verify and
specify the fabric-wide consistency policies:
BB_B5l : adrnin> fddcfg --showall

Local Switch Configuration for all Databases : -
DATABASE Accept/Reject
sec accep t
DCC accept
PWD accep t
• Tolerant policies display: "SCC ; DCC"
FCS accept • Strict policies display: "sec : s ; occ : s"
AUTH accept •A strict SCC and a tolerant DCC policy
IPFILTER accept
output displays: "sec : S ; DCC"
Fab ric Wide Consistency Policy : - 1111
Revision 01 10 3-6
Preparing Fabrics for Routing (cont.)
To manage the consistency of the sec and DCC databases across the fabric, there
is a Fabric Wide Consistency Policy. This policy defines whether the Switch
Connection Control (SCC) and Device Connection Control (DCC) databases are·
distributed automatically or manually when a database changes or a new switch
joins a fabric. The SCC database determines which switches will be allowed to join
the fabric. The DCC database determines which devices will be allowed to attach to
a switch. These databases are called ACLdatabases'.
There are three levels of fabric-wide consistency that can be specifi,ed for the SCC
and DCC databases:
1. Not defined (absent): Fabric-wide consistency policy is not defined (default). A
switch that has an absent fabric-wide policy can have ACL databases. These
AGL databases can be changed by a manual distribution from another switch.
2. Tolerant: Switches are not required to have the same ACL databases. Switches
with absent and tolerant policies can be part of the same fabric. This provides
greater flexibility for pre-Fabric OS v5.2 and non-Fabric OS switches. Switches
can have the same, different, or no ACl databases. The switch SCC policies in
each fabric must contain all switches in the combined fabric. The switch DCC
policies in each fabric must contain all the devices attached to expected
switches in each fabric. Given the above-If a switch has a different database
from the rest of the fabric, it remains in the fabric. SCC and DCC database
distribution is .automatic; when a database is changed on any switch, that
database is automatically distributed to the rest of the fabric.
3. Strict: Switches in the fabric always have the same ACL databases/ Ensures that
SCC and DCC policies are consistent on all switches in a fabric. To join a fabric,
a new switch must have exactly the same sec and DCC databases as the
rest of the fabric - or no database at all. sec and DCC database distribution is
automatic. If a new switch joins the fabric with no database, the ACL database in
the existing fabric is automatically written to the new switch. When a database is
changed on any switch, that database is automatically distributed to the rest of
the fabric. If one switch in a fabric has a strict policy, all switches in the fabric
must also have a strict policy.
Revision 0110 3-7

Configure Backbone Fabric ID

• Before setting the fabric ID in the backbone fabric, disable the FC routing
service with the foscon f ig command (default setting)1
- All EX_Ports on the switch must be disabled prior to executing this command
- The switch must also be disabled using swi tch d i sab le
• Nlext, set the fabric ID of each backbone switch with the f crcon f igure
command
- Fabric ID must be unique for each edge and backbone fabric
• Finally, enable the FC routing service with the f osconf ig command
BB 51 : admi n > s witeh d i sable
BB_B51 : admin> foseonfig --d isable f er
FC Rou ting 5ervice i5 already di5abled
BB_B5 l : admi n> f c r conf igur e
FC Rou ter parame t er set . <er> to skip a paramet er
P l ease make s u re new Backbone Fabric I D does not conflict wi th any confi gured
EX- Port ' s Fabric ID
Backbone fabric ID : (1- 128) [1] 100
BB_ B5l : admin> f oseonfi g --enable fer
FC Rou ting service is enabled

Footnote 1: The fosconfig --d i sabl e fer command disables the upper layer
FC Routing Service in Fabric OS while the Layer 2 switching remains enabled. The
command has no arguments or optional parameters. In the example above, the FC
Routing service was in the default disabled state, as indicated in the command
output.
When using the f oseonfig --disable fer command, keep these
considerations in mind:
• All EX_Ports on the switch must first be disabled (portdisab l e or
b l adedisable).
• The swi tehdis able command must also be run before the FC routing
service can be disabled.
• Display the current state of the FC Routing service with the familiar
swi tehshow command.
The fc r confi g u re command configures the fabric ID of the backbone Fabric. The
command is menu-driven, and has no arguments or optional parameters. The
Fabric OS default fabric ID value is 1; as shown above, the fabric ID has been set to
100.
The fos e onf ig --enable f er command enables the FC Routing Service in
Fabric OS.
With VF enabled, the backbone fabric and default switch use the same FID.
Revision 01 10 3- 8
Configure EX_Port and Set Edge Fabric ID

• Configure an EX_ Port with the port c f gexport command 1
- If needed, disable the port using portcfgpersistentdisable 2
- Use - m option for M-EOS edge fabrics 3
- Best practice is to manually set the FID - f and front domain ID4 - d
BB B5l : admin> portcfgpersistentdisable 3

BB B5l : admin> portcfgexport 3 -a 1 -f 10 -d 120
( EdgeFl0100

Footnote 1:
The example on this slide persistently disables port 3 and then configures it as an
EX_Port. When configuring routing over an FCIP tunnel, the p ortc f gve xpo rt
command is used.
Footnote 2:
The p o rtdisab l e command can also be used but is not recommended. All FC
ports on the Brocade FR4-18i and Brocade 7500 are persistently disabled at the
factory.
Footnote 3:
Port modes include interoperability modes. See -m port mode parameter below.
Footnote 4:
If the front domain ID is not specified the default domain ID assigned to the first FD
in an edge fabric is 160.
Revision 01 10 3-9
The portc f gexport command is used to place an FC port into EX_Port mode
portcfgexport [slotnumber/ Jportnumber [- a admin] [=f
fab ricid] [ -r r atov] [ -e edt ov ] [-d dornain i d ] [ - p p i dforrnat]
[ -t fabric_parameter] [ -m po rtrnode ]
Required argument: slotnumber /po r t nurnber .
Optional arguments:
- a adrnin Specify whether to ( 1 -enable , 2 -disable) this port as an
EX_ Port. If 2 is specified, the port will not be disabled, but will no longer be
configured as an EX_Port. portcfgdef aul t can also be used to disable
EX Port mode.
-f fabricid Specify the fabric ID. Valid values are 1-128.
- r ratov Specify the R_A_ TOV used for port negotiation. Valid values are 2000
- 120000.
-e edtov Specify the E_ D_ TOV used for port negotiation. Valid values are 1000
- 60000.
- t fabric parameter Specify whether to (! - enable , 2 - disable )
negotiation Of the fabric parameters RA_ TOV and ED_ TOV.
-d dornainid Specify the preferred domain ID. Valid values are 1-239.
-p pidforrnat Specify the Port ID format. (0-native , 1-core , 2-
extended edge ). This operand is applicable only when port mode is set to 0
(Brocade Native mode).
-rn portmode Specify the Port mode (0: Brocade Native mode, 1: M-Series
Open Fabric 1.0 mode (and Brocade lnterop mode), 2: M-Series McDATA Fabric
mode used when the neighboring M-Series switch is running OS version such .as
6.0.2 or later, 3: M-Series Fabric Legacy mode, for the legacy M-Series ED5000
platform)
If no optional arguments are specified, the current port configuration will be
displayed.
Note: The portcfglongdistance command may be used to place an EX_Port in

long-distance mode. An Extended Fabrics license is required.
Revision 01 10 3 - 10
Enable EX Port
• Enable the EX_ Port and verify EX_ Port configuration:
BB_B51 : admi n> p ortcfgpersiste ntena ble 3
BB B51 : admin> p o r tsh ow 3 Changes from OFFLINE to

portName :
HEALTHY after
portHealth : HEALTHY
portcfgpersistentenable
Authentication : None
EX Port Mode : Enabled

!Fabric ID : io I
F ront Phantom : S tate : OK ["CU"r Dom ID : iJ{] WWN : 50 : 00 : 51 : e7 : e2 : 62 : ee : Oa
Pr Switch I nfo : Dom ID : 3 WWN : 10 : 00 : 00 : 05 : 1e : Ob : 96 : 8f
!Fabric params : R A TOV : 10000 E D TOV : 2000 PID fmt : corel
<truncated output>

In the example above, port 3 o n the Brocade 5100 is configured with the following
settings:
EX Port Mode : Enabled
Fabric ID : 10
Front Phantom :
State : OK WWN is assigned from a
Current Domain ID : 120 poll of WWNs by the FC
router to represent the
WWN : 50 : 00 : 51 : e7 : e2 : 62 : ee : Oa
EX Port front domain
Fabric parameters :
R A TOV: 10000
E D TOV : 2000
PID format : core
Revision 01 10 3 - 11
Verify Connectivity- Edge

• Use the swi tchshow command to verify E_ Ports1
Edge_B30 : admin> switch show I ·grep e-port -i
8 8 030800 id NS Online FC E- Port 50 : 00 : 5l : e7 : e2 : 62 : ee : Oal " fcr _ fd_ l20 "
(downstream)
• Use the fabricshow command to view front domains2

Edge_B30 : admin> f abricshow
Switch ID Wor l dwide Name Enet IP Addr FC IP Addr Name
3 : fffc03 10 : 00 : 00 : 05 : 1e : Ob : 96 : 8f 10 . 255 . 240 . 31 0 .0 .0 .0 >"Edge_ B30 "

120 : fffc76 50 : 00 : 51 : e7 : e2 : 62 : ee : Oa O. O. O. O O. O. O. O ! " fer fd 120 "
The Fabric has 2 switches

Footnote 1: EX_Port trunks will appear as E_Port trunks in the edge fabric. E_ Port
trunking is implemented with the familiar CLI commands: swi tchshow,
trunkshow, portcfgtrunkport, and swi tchcfgtrunk. Please note that
trunking is enabled by default.
Footnote 2: Although multiple IFLs may link a single router to an edge switch, only
one front domain will be presented to the edge fabric on behalf of that router.
Revision 01 10 3 - 12
Configure LSAN Zones

• LSAN zoning must be enabled in all fabrics that share devices
- Edge-to-edge routing: edge fabrics
- Backbone-to-edge routing: backbone and edge fabrics
• LSAN zones are zones that begin with the characters LSAN
Lsan , lsan , etc.
• LSAN zone names do not need to match between fabrics sharing

devices
- Best practice is to match LSAN zone names between fabrics sharing
devices to make cross referencing easer
• LSAN zone members must be identified by their PWWN

Configure LSAN Zones (cont.)
• Create LSAN zones using whatever tool you normally use (DCFM,
Web Tools or CLl) 1
Edge Fabric (Fabric ID 10)

Edge_B30 : admin> cfgactvshow
Effective configuration :
cfg : Edge_CFG
zone : LSAN_Backbonel_Edgel
10 : 00 : 00 : 05 : 1e : 57 : 7c : 79
22 : 00 : 00 : 20 : 37 : dd : d9 : 29
Backbone Fabric (Fabric ID 100)

BB_BSl : admin> cfgactvshow
cfg : BB_CFG
zone : lsan_backbonel_edgel
10 : 00 : 00 : 05 : 1e : 57 : 7c : 79
22 : 00 : 00 : 20 : 37 : dd : d9 : 29

Footnote 1: The tools used to configure LSAN zoning are irrelevant - use your
favorite tool. The important point is that the LSAN zones exist in each fabric, and
are being enforced within the fabric as part of the Effective configuration (Fabric
OS) or active Zone Set (M-EOS).
Verify FC-FC Device Routing

• Once the LSAN zones are active, determine which devices actually
exist in the fabric, and which ones are imported
• Executed from the FC router in the backbone
BB B51 : admin> lsanzoneshow -s
Fabric ID : 10 Zone Name : LSAN_Backbonel_Edgel
10 : 00 : 00 : 05 : le : 57 : 7c : 79 EXIST
22 : 00 : 00 : 20 : 37 : dd : d9 : 29 Imported
Fabric ID : 100 zone Name : lsan_backbonel_edgel
10 : 00 : 00 : 05 : le : 57 : 7c : 79 Imported
22 : 00 : 00 : 20 : 37 : dd : d9 : 29 EXIST
• Can specify additional options for selective display:

- f option for a specific Fabric Id (FID)
- w option for a specific WWN
- z option for a specific zone name

The lsanzone show command will display all currently-active LSAN zones that the
backbone fabric is enforcing.
l sa n zoneshow [- s] [ - f fa b ricID] [-w wwn] [- z zonename]
Search parameters - f , - w, and - z allow searching for LSAN zones based on fabric
ID, WWN of an LSAN zone member, or LSAN zone name.
- f fabriclD: Display LSAN zones in the specified fabric.
-w wwn: Display LSAN zones containing the specified port
WWN. (Format XX:XX:XX:XX:XX: XX:XX:XX)
- z zonename: Display LSAN zones with the specified zone name.
- s state: Display state information for the device, valid states include:
Configured - Device is configured to be in an LSAN, but the device is not
imported nor does it exist in this fabric.
EXIST - Device exists in this fabric (the fabric of the zone entry).
Ini t ializing - Device is in an intermediate state. It is not yet imported into
the fabric.
Imported - Device has been imported (proxy created) into this fabric.
In this example, you can see which devices actually exist in the fabric listed (EXI ST)
and which ones are projected into that fabric ( I mpo rted).
Revision 01 10 3 - 15
Verify Translate Domains

• The translate domain (xd) should now appear in the fabricshow
output of both fabrics:
Backbone Fabric:
BB B5l : admin> fabricshow
Switch ID Worldwide Name Enet IP Addr FC IP Addr Name
1: ffcOl 50 : 00 : 5l : e7 : e2 : 64 : ef : 02 0 . 0 . 0 . 0 0.0. 0.0 " fer xd 1 10"

: f 0 : 00 : 00 : 05 : le : 7e : 26 : 2e 10 . 255 . 240 . 33 0.0. 0.0 >' BBB "
Unless the domain ID is specified

• Why is there no Front domain?1 during configuration, the lowest
available domain ID will be negotiated

Footnote 1: When devices are shared with the edge fabric using LSAN zones, a
translate domain will be added to the local backbone fabric representing the remote
edge fabric but front domains are not needed as the backbone fabric is router port
aware.
Revision 01 10 3 - 16
Verify Translate Domains (cont.)
Edge Fabric:
Edge B30 : admin> fabricshow
ffcOl 50 : 00 : 51 : e7 : e2 : 64 : ef : Ol 0 . 0 . 0 . 0
c03 10 : 00 : 00 : 05 : 1e : Ob : 96 : 8f 10 . 255 . 240 . 31
8 50 : 00 : 5l : e7 : e2 : 62 : ee : Oa 0 . 0 . 0 . 0
switches
Xlate domain ID = 1 represents a

phantom switch in this edge fabric that
all the phantom devices from the
remote fabric (FID=100) hang off of.
Front domain ID = 120

The xlate domain and front domains do not have an Ene t I P Addr assigned. This
w1·11_aways
I b_e th .
__ e case since . I, not ph_ys1ca_
th ey are Iog1ca_ . I doma1ns.
.
Verify Connectivity- Backbone

• Verify that you have established a connection to an edge fabric with
the swi tchshow command:
BB B5l : admin> switchshow
<Truncated Output>
zoning : ON (BB_CFG)
switchBeacon : OFF
FC router : ON
FC router backbone Fabr ic._ ::
ID:.:_
: _:.l:;
O::._
o _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _
E-Port 50 : 00 : 5l : e7 : e2 : 64 :ef : 02 " fer xd 1 10 "

<Truncated Output>

In the example 0111 the next page, the swi tchshow command output includes the
following information related to FC Routing functionality and the newly-created
EX Port:
•The switch is a Brocade 5100 (switch Type : 66 . 1).
• The FC router service is enabled (Fe router : ON).
• ThefabriclDforthebackbone(FC rou ter back bone Fabric ID : 10 0).
• Port 3 is online and configured as an EX_Port connection to the edge fabric
(10 : 00 : 00:05 : 1e : Ob : 96 : 8f " Edge B30 " fabric id = 10 ).
• Because devices are shared between the backbone and the edge a translate
domain (XO) is created in the backbone fabric to display the shared (imported)
devices from this edge fabric. (E- Port 50 : 00 : 51 : e7 : e2 : 64 : ef : 02
" fcr_ xd_l_lO ")
To create a trunk, additionally configure and enable ports as EX_Ports to the edge
fabric following all normal trunking requirements. Again verify output with the
swi tchshow command.
EX_Port Trunking is administered using the same CLI commands as E_ Port
trunking: swi tchshow, trunkshow, portcfgtrunkport, and
swi tchcfgtrunk.
Revision 01 10 3 - 18
Verify Connectivity - Backbone (cont,)
BB BSl : admin> switchshow

switchName : BB B51
switchType : 66 . 1
switchState : Online
switchMode : Native
switchRole : Principal
switchDomain : 98
switchid : ff fc62
switchWwn : 10 : 00 : 00 : 05 : le : 7e : 26 : 2e
zoning : ON (BB_ CFG )
switchBeacon : OFF
FC Router : ON
FC Router BB Fabric ID : 100
Index Port Address Media Speed State Proto
0 0 620000 NS No Module FC
1 1 620100 id N4 Online FC Loopback- >Port 1
2 2 620200 id N4 Online FC Loopback->Port 2
3 3 620300 id NS Online FC EX-Port
10 : 00 : 00 : 05 : 1e : Ob : 96 : 8f " Edge- B30 " (fabric id = 10 )
E- Port 50 : 00 : 5l : e7 : e2 : 64 : ef : 02
" fer xd 1- 10"
4 4 620400 id NS In_ Sync FC Disabled
5 5 620500 id NS Online FC Loopback->Port 10
6 6 620600 id N2 Online FC L- Port 4 public
7 7 620700 NS No Module FC<Truncated Output>
Verify Connectivity- Backbone (cont.)

• Display backbone fabric information with the f crf abr icshow
command
- Each FC router : WWN, Domain ID, IP Address, Switch name
- Each EX_ Port: Fabric ID, Neighbor Switch WWN, IP Address, and
Name
BB B5l : admin> fcrfabricshow
FC Router WWN : 10 : 00 : 00 : 05 : le : 7e : 26 : 2e , Dom ID : 98 , Info : 10 . 255 . 240 . 33 , " BB_B51 "
EX Port FID Neighbor Switch Info (enet IP, WWN, name)
3 10 10 . 255 . 240 . 31 10 : 00 : 00 : 05 : 1e : Ob : 96 : 8f " Edge_B30 "

In the example above, the fcrfabricshow command output indicates that the
backbone fabric includes one router, with the following information:
• WWN = 10:00:00:05:1e:7e:26:2e
• Domain ID = 98
• IP address= 10.255.240.33
• Switch name= BB B51
This router has one EX_Port with the following information:
• EX Port Number= 3
• FID=10
• IP address= 10.255.240.31
• WWN = 10:00:00:05:1e:Ob:96:8f
• Switch name= Edge_B30
Verify Physical Devices - Backbone

• Display the physical devices with the f crphydevshow command:
88_ 851 : admin> fcrphydev·show
Device WWN Physical
Exists PID
in Fabric
10 10 : 00 : 00 : 05 : le : 57 : 7c : 79 030100
100 22 : 00 : 00 : 20 : 37 : dd : d9 : 29 6206e4
Total devices displayed : 2
• In the command output we see the two physical devices that are currently
being shared across the backbone fabric:
- Edge fabric (FIO 10) has one physical device PIO 030100
- Backbone fabric (FIO 100) has one physical device PIO 6206e4

Verify Proxy Devices - Backbone

• Display proxy devices with the f c rpro xydevshow command:
BB_B51 : admin> f c rpro x yde vsho w
Proxy WWN Prox y Device Physical State
Created IPI D Exists PIO
in Fabric in Fabric
10 22 : 00 : 00 : 20 : 37 : dd : d9 : 29 OlfOOl 100 6206e4 Imported

100 10 : 00 : 00 : 05 : 1e : 57 : 7c : 79 OlfOOl 10 030100 Imported
Total devices displayed : 2
• In the command output we see the two proxy devices that aire currently
being shared across the backbone fabric:
- The format of the proxy address is XXfYYY, with xx indicating the translate domain ID,
and YYY is a value beginning at 001 1

In the f crproxyde vshow command output above, we see the two proxy devices
that are currently being shared across the backbone fabric:
• In the edge fabric (Fabric ID 10), there is one proxy device (WWN -
22:00:00:20:37:dd:d9:29), matching the physical device attached to the
backbone fabric (Fabric ID 100). The FC address of the proxy device
(Ox01f001) confirms that it is the first proxy device connected to translate
domain 1.
• In the backbone fabric (Fabric ID 100), there is one proxy device (WWN -
10:00:00:05: 1e:57:7c:79), matching the physical device attached to the edge
fabric (Fabric ID 10).
Note: Besides using CLI commands to verify devices, ports, zones, proxies, etc.,
verification can be achieved using SAN Health and DCFM
Footnote 1: The YYY portion of the PIO numbering increments as follows 001 , 101 ,
201 , 301 , 401 , ..., f01 , 002, 102, 202, 302, 402, ... This incrementing scheme is
used to better utilize the VCs when the frame traverses an ISL in the edge fabric.
Note, the device area field begins with -!!fand AL_ PA field is not 00, if using core
PIO.
Revision 01 10 3 - 22
Verify Proxy Devices - Edge

• Use nsallshow to display the 24-bit Fibre Channel addresses
or Port ID (PIO) of all devices in the edge fabric
30 : admin> nsa11show
Imported phantom
030100
devices "attached" to
translate domain ID 1 2 Nx Ports in the Fabric
• In the example above there is one imported phantom device

- The device area field begins with "f ' and AL_PA field is not 00, if using core
PIO
- The device (01 f001) is imported into translate domain (01) in the local fabric
• The proxy device is attached to the translate domain
- It is a logical device but is recognized as being physically in the local fabric
• The proxy device PIO persist across power failures and reboots

You can identify imported proxy devices in nsallshow output by their 24-bit
address:
1. The Domain ID will be the Domain ID of the translate (xlate) domain used for
representing the -emote" Fabric ID where the imported device physically exists.
2. The Area (port#) will be in the range of OxfO - Oxff.
3. The AL_PA will be non-zero, starting at 01 and ending at FF. Examples: 08f003,
03fd22, 47fe17
Verify Proxy Devices - Edge (cont.)

• From the edge fabric, use nscamsho w to display the proxy device 24-bit
PIO and physical device WWN 1
Edge_B30 : adrnin > nscamshow
Translate domain ID 1
nscam show for remote switches :
Switch entry for 1
state rev owner
Number of imported phantom devices
known v410
"attached" to translate domain ID 1
Typ Pid Node Name
NL OlfOOl 3 22 : 00 : 00 : 20 : 37 : dd : d9 : 29 ; 20 : 00 : 00 : 20 : 37 : dd : d9 : 29 ;
FC4s : FCP
PortSymb : (28 ] " SEAGATE ST336704 DE84 "
Fabric Port Name is created by
Fabr i c Port Name : SO : OO : Sl : e7 : e2 : 64 : ea : 03
the router as the translate
Permanent Port Name :
domain is a logical device
Port Index : na
l swi tch entry

r Front domain ID 120
for:::::fiD --~=========::::;~~-~~~~~~~~~-_J
state rev owner cap_available
known v410 Oxfffc03 1 Note that front domain will
I Device list : count 0
No entry is found !
f -- ==============::=, never have devices attached
------------~

Footnote 1:
The nscamshow command displays the Name Server Cache Manager output. The
Name Server Cache Manager contains a cache of the Name Server information for
all other switches in the fabric including the logical front and translate domains.
Optional LSAN Zone Tagging

• Configure and manage optional LSAN tags using the following
commands:
- Create an enforcement LSAN tag :
f crlsan --add -enforce <tag>
- Create a speed LSAN tag
f crlsan --add -speed <tag>
- Display the current enhancemenUspeed LSAN tags
f crlsan --show -enforce I speed <tag>
- Delete an enhancement or speed LSAN tag
f crlsan - -delete -enforce I speed <tag>
- Delete all enhancement or speed LSAN tags
f crlsan - - delete - enforce I speed
• Configured in the backbone on FC-FC routers only prior to creating
LSANs in the backbone and edge fabrics
BB B51 : admi n> switchdisable

BB B51 : admin> fcrlsan --add -enforce local
Lsan Tag set success f ully .
BB B51 : admin> fcrlsan --add -enforce BRCD
Lsan Tag set successfully .
BB B51 : admin> fcrlsan --add -enforce remote
BB B51 : admin> fcrlsan --remove -enforce BRCD
BB B51 : admin> fcrlsan --show -enforce
Total lsan Tags : 2
enf : loca l
enf : remote
BB B51:admin> fcrlsan --add -speed super
BB B51 : admin> fcrlsan --show -speed
speed : super
BROCADE
EDUCATION SOLUTIONS
Configuring FC-FC Routing from DCFM

• In this section the following steps will be performed:
1. Enabling Fibre Channel Routing Services
2. Configure EX_Port using Port Configuration Wizard
3. Enable the EX Port
4. Changing the front and translate domain (optional)
O 2009 Brocade Commun cal1ons Sys•ems Inc 2 7

Al R1gh's Reserved
Enabling Fibre Channel Routing Services
~..:.10.2SS240.47 - FC Router
Gene<el_J EX-Porto LSANFebrlcs LSANZones LSANDeW:es

..::;view AUi OUM 10 3..7
Fibre Chomel ftOl.(e< Gene<el .,formellon & Conf96ellon
:i."11 fdl 'fjlrN Qjlcovot i;cnii!µ• """""' 1- ~
If.) 0 0:
VlewAI• I
~ + •~-LI 0-. •
©
-bone Flbrlc IO; 1
llo4oc FCR ~ are cw«llly cls-.i on tlis swtoh.
x Waining EJ
.. FCR Set!'llce Is -
Lise< carrd ,,,.,,. FCR relete<I feature.
Do yau wen to..- FCR?
~ Ho

Al 1R ghts Reserved
To enable Fibre Channel Routing Services for a switch:

1. In the DCFM navigation tree right click the switch and select Element
Manager ~ Router Admin, the FC Router window will be displayed
2. On the General tab click the Set Fabric ID button

a) Select the Backbone Fabric ID from the pull-down menu and click OK
3. Click the Enable FCR button then click Yes at the prompt
Note: The switch must be disabled to change the backbone fabric ID, valid range
for backbone fabric IDs is 1-128.
Configuring an EX_
. Port
.!'-210.25 outer
0.Wot EX-Porto I.SAN ' * ' " I.SAN z..- LS.'IN ~

EX-Pert ~ . EX-Port lntormol'°" 8 Conf9'otion
~~ ~Po=~rU-~~=No=--~=
~QJ me P<>r=l Sl=-~~H
«v ce:=lh.~. . :ciiiiiiiiiiiiiiiiiiiiiiiiiiix~ dPcrt
l. Se.lect Port
2. Specify Ports Paramerers
3. Confinnation
4. Report

Al 1R ghts Reserved
To configure an EX port:
1. In the FC Router window click the EX_Ports folder tab
2. Click the New button to launch the Port Configuration Wizard
3. Select the port from the list and click Add
4. Click Next
Revision 01 10 3 - 29
Configuring an EX_
. Port (cont.)
Port Configurcitmn W•zard Ef
1. Select Port
FellrlcD:
2. Specify Polis Parametel'S
• BrocedoNoliYe-
3. Confumation
McOola Febrlc-
,... ()pefl-
'6'
\"V
4. Report
MCOATAfebrlclegacy
The following fabric setting cannot be

configured from the wizard
Preferred front and t ranslate domain ID,
PIO Format, R_A_ TOV and E_D_TOV

Al 1R ghts Reserved
Specify the port parameters:

5. Select the Fabric ID of the port from the pull-down menu
6. Set the lnterop Mode for the EX_Port
7. Click Next
lnterop Mode: Specify the Port mode:

• Brocade Native mode
• M-Series Open Fabric 1.0 mode (and Brocade lnterop mode)
• M-Series McDATA Fabric mode used when the neighboring M-Series switch is
running OS version such as 6.0.2 or later
• M-Series Fabric Legacy mode, for the legacy M-Series ED5000 platform)
Revision 01 10 3 - 30
Configuring an EX_
. Port (cont.)
Porl Conf1gu10.hon Wtz<lrd £1

Steps Specify FC Porameter0o
L SdeclPort
2. Specify Ports Parameurs @

3. Specify fC Parameters
4. Confinnalion
Speed
Ingres: R.ce ~)
Al.Ao
Noe-zed
UtNormol
...
I
5. Report long Distonce -
Dessed Dlstonce (Ion)

Al 1R ghts Reserved
Specify the FC parameters:

8. Specify FC parameters using the Speed, Ingress Rate Limit, and Long
Distance Mode drop-down menus
9. Click Next
Revision 01 10 3 - 31
Configuring an EX_
. Port (cont.)
Port Conr1guration Wtzard El

Steps Confrmetion
l. Select Port Allowed Port Type: E.X-Port

Selected Port: 4
2. Specify Ports Parameters
Fabric Id: 2
3. Specify FC Parameters lnterop Mode: Brocade Nauve Mode
FC Parameters:
4. Confurmalion
Speed: Auto
S. Report tneress Rate Limit: Not lrutialized
Lon& Oisllance Mode: LO:Normal
Click Slllle to complete the Poft Configuration

Al 1R ghts Reserved
Confirm the new configuration:

10. On the Confirmation page review the settings and verify everythi ng is
correct
11. Click Save to continue to the Report page
Revision 01 10 3 - 32
Configuring an EX_
. Port (cont.)
Porl Conftgurahon Wtla1d El

Sleps Repcn
L Select Port IConfigurations are corrqileted successfully

2. Specify Ports Parameurs
3. Specify FC Parameurs
4. Confirmation
5. Report

Confirm the new configuration (cont.):

12. Verify that there are no errors and click Close
Revision 01 10 3 - 33
Enabling the EX Port
~..:-.10.255.240.47 · f( Router l!lll~EJ
- AddlionolPo... fallrlcl>
Olftne 2
:~ 10.255.240.47 · FC Router l!lll~EJ
Oenerel EX-l'orls LSANfallrlcs LSAHZones LSANOevlces

EX-l'orl Exi>iorer EX-l'orl lnlonnelion & Conf9.>'ation
El 098(R11-ST08-BS1) Noyt aenemo YJtrw Oelab Ii.di ConftgU"ollOll Port is now online
Iii Port •
PersjStenl ~
P"'1I ..
4(0x4

Al 1R ghts Reserved
Enable the EX Port:

13. The new EX_Port wi ll now be displayed
14 . To enable the EX Port select it from the list and click the Enable button.
;{view All - OCFM 10.3.2 llllf@E'f

~AN !;.drt YJew Qiscover c;pnfigure Monitor Iools t!etp
fJ 0 0:-+ ~ +• ~ IJtJ Decimal • I Neme •I Neme ~ f.- Seerc "f
~
~
IAn Levels Name ---,(De
_ vic
_ ~T
is 0 RSL11_srs RSL11_srs • j
B 0 R11-ST08-BS1 R11-ST08-B ... • swilc
4 20:0A:00:05:1
Ji 22:00:00:04:C
4 22:00:00:20:3"
® 22:00:00:20:3. J
Ji 20:05:00:05:11 RSL1 1_ST8
@ 0 <.p RSL11_ST8_Edge'RSL11_ST8 ...
~T
R11-ST08-830
RSL11_ST8_Edge1
Changing Front and Translate Domain IDs
.,>; view All · OUM 10.3.2 11111!1£1

~ ~ ll)ew ~ ~figo.re fl!Onl«
a.J 0 ~ ~ -~
. . - - - - - - - - - ; FC~Ch
VltYw /IJI. ,__
s;,£1: Swleh______,
:22:00:00:20;3' FCI' 1...-.iels

:2200-00:20;3' H!tl _..O!Jly F -
20:05.00:05:11 fejlJIC Bindl'lg
8 0 ~ RSt.I 1 _ST8_Edge fer! Fencr.g
Pol1ALCo~
.;p T
DOON
Pot! (lr014>S
FCTr~ •
f'Tr-.ng

Al 1R ghts Reserved
To configure front domains on an edge fabric:

1. From the main DCMF window click Configure ~ Routing Domain IDs
Changing Front and Translate Domain IDs (Cont.)
; 'r Configure Routmg Domain IDs Ef

Avoiatlle Swlches : Selected Swlches
Allevels Swlch Name .... Conflgu'ed tot FCR Swlch 1Nam(! IDomein D
El Q ~ RSL11 _ST8 RSL11 _ST8 fcr_fd_160 R11-ST08-B51 I2 •
~ fcr....xd_1_2 fcr_xd_1_2 Swlch fcr_xd_1_1 R11-ST08-B51 8 •
El Q <J> RSL11 _ST8_Edge1 RSL11 _ST8_...
® tcr_fd_160 fcr_fd_160 Swlch
® tcr....xd_1_1 fcr....xc1_1_1 Swlch

To configure front domains on an edge fabric use the Configure Routing Domain
IDs dialog (cont.):
2. Select the domain in the edge fabric you wish to change (Note: The FD must
be online before you can change it)
3. Click the right arrow to add it to the Selected Switches list
4. In the Domain ID column use the pull-down menu to select the new domain
ID
5. Click OK
BR0CADE
1
EDUCATION SOLUTIONS
Configuring LSAN Zoning

., ; Vt~ All OUM 10 J.2
Cii. D
N.leYoie
El O , . RS1.11_sra
El <::>1111.st0&-1161
,i 20;0AJlllOS.1
~
Z"""41
--r-~----L.._,
01
a220000CMC I-'-"--- ---..
• 2200001113
a22000020;3
a 2005;ooos.11
8 0 • RSl.ll_Slll..Ed!le
fCh~
p TredllMhOCCng
•
•
<s> r
111
RSI.I
...__o .....,,_.,.
Ill
m O q.OS1.11_$1>.-1
-- - .s-
•I

Al 1R ghts Reserved
Once Fibre Channel Routing is configured DCFM automatically creates a scope

specifically for LSAN zoning in the Zoning window.
Configuring LSAN Zoning:

1. Highlight the fabric and then click Configure -+ Zoning -+ LSAN Zoning
(Device Sharing)
2. In the Zoning Scope drop-down menu select LSAN_xxx, where xxx is the
fabric name
Adding Devices to the Zone
Z"'*>g Scq>o LSAN_RSl11 _STe __:J

ZoneDB I
4
• Zones
Pd~-·
Aile.do Nome _ ,o.Mc ~ ~~z g ActiYole...
El 0 <IP
RSl..11 _Srs • l:;::I (FCD:6206e2) SEAOATE TEOflOl.OOV ~ 22;
El 0 R11-STQ8:.8S1 R11.ST08-851 SWli (FCD:6206ell SEAOATE TEOflOl.OOV ~ 22;
10 (FCD:OOOIOO) llr~ Corrmri:8tlons Systems,
6206e8
..!) 6206e2
"' 6206ef
~ s
8 0 ~ RSL11_ST8_Edge1
El 0 R11.STQ8..830 R11.STQ8..830 Swlq ~
.iJ 030100
[ dliil New Zane N e w -...

Adding devices to the zone:

3. Click the New Zone button
4. Set the zone name, DCFM will automatically prepend LSAN_ to the zone
name. To change the name of the zone, double click on the field and type in
the new name
5. Select the devices you want to add to the zone, you can select multiple
entries using CTRL +Click
6. Click the arrow to move the devices into the new zone
7. Click the Activate button to save the zone to the fabrics
Note: The 5 and 10 entries under switch R11-ST08-B51 are loopback ports and
show up because the view has been changed to "Occupied Ports-en a prior
screen.
Activating LSAN Zones

t-~ Activate LSAN Zonl!~ 13
Review !he MmWY lnformetlon for the LSAN ccnfi!ltntion
LSA.N zones con be~ to the preferred febrtcs, llis Is needed IOI the zones wlh o.tflne de'kes.
Febrtcs
Zone Oestno00n febrics 13 ~ RSL11_ST8
8 (i!I LSAN_Newlone RSL11_ST8,RSl11_ST8_Edge1 (i!I LSAN_}lewlone
8 ~ RSl 11_ST8_Ed!le1
(i!I LSAN_}lewlone
You 01e abcM..C 10 ectivale LS.AN zones i1 the febrlcs. For the LS.AN 1ones that corHin offine de\ltces tx.t are
nol ossignod lo #Jlt'f clher f-. tho zones wt be pushed lo tho f-wh«e lho...., devices belorog lo.
tt lhe<e Is no adrYe zone confl!µ'.iion In tho,_ _ the LSAN zones wt be _,hod 10, ozone oonfl!µ'IDon wt be
created h the $Wlch to contan the LSAN zones
Ou'n!l the L S A N - . the cwr.n I.() on the ,..,.ed •Ob1cs wt be ~ed
tt o newty u..,.ed LSAN zone hes the ._,..,,.es an exlslng zone In the ?°'*'II
dalobose, tho new <one defnboo wil ovetwrlo ll>O e>dslng zone
Do you w...c 10"""""'°?

Activating LSAN zones

7. Click Ok to save the configuration to the switches, note that the
LSAN_NewZone has been added to both the edge fabric and backbone
fabric
8. At the DCFM Message dialog click Yes
Note: The LSAN zones will be added to the active configuration of each
fabric involved in the zone. If there is no active zoning configuration in
the fabric one will be created and made active.
9. Click Ok on the LSAN activation confirmation, this will close all dialog boxes
Note: To refresh the Zoning dialog click Ok to close the window, then reopen by
going to Configure -+ Zoning
Revision 01 10 3 - 40
Verifying Zone Configuratiions

• Close the Zoning dialog and reopen to refresh with the new
ZoneDB information
• LSAN zones can be m.anipulated like any other zone
• If a zoning config does not already exist one is created to hold the
new LSAN zone
- Cl x
Zcrilg Sccpe RSl.11_ST6_Edgo1 Zone 06 Fable lone 06 • Zone 06 Opcrtllon Zone 06 Edit •
G ~R11-ST08-B30
Type WNH
lA.es-
•
1' Fild
~ : I
I
•• Zones
IEJINJt:SAN'J
8
lone
1rco:01to01JSEAOATET
ij )FCD:01ft01)SEAOATET
~ (FCD.030100) 6rocecle
9 Fild
:
•• Zone Conf90
Ila&JINJLSAN_New~·I
LSAN_CF0_20091
1ii3J _g AeWete...
Z<dlgPclcieo
1 I
LSAN zones can be added to existing zoning configurations. If the fabric already
has an active zone configuration place the new LSAN zone into that configuration
and write the changes to the fabric.
The most important thing to remember is that LSAN zones are administered just like
any other zone
Zoning Scop RSl.11 ST8 Zone DB falorlc Zone DB H

1
( Zone DB AsNf!J Z
j
•• •• Zone Con~s
I"
Paenllal Members Type WMJ • Zones
AllevelS 1,..,..... ~Fm

fi ~
B ii;n ....._...,..
-4' (FCD:011001) Brocade Convn
~
-;-i J!l;JU..._<>• ..,, , S. Activat. =J
jID LSAN_NewZone
ID 0 R11-ST08-B51
ij [fCD:620!ie2) SE.a.GATE TEC Zonng Polldes I
ij IFCD:6206ef) SEAGATE TEI I
ZoneDB I
Paential Members
•
•Zones
AllevelS ....
l!I 0 0 c,,p RSL11_ST8
~
2..J
r- I
"'"'=l_SAN_,Jl!ew
· ID _ Zone---.[------.1
1611 (fCI0:6206e2] SE.a.GATE TECHNOLOGY~
s. Acliv8te ...
l!I 0 0 ~ RSL11_ST8_Edge1 ij lfCI0:6206ef] SE.a.GATE TECHNOLOGY (!/WY,
l ~""""'""-~!
BR0CADE
1
EDUCATION SOLUTIONS
Verifying EX_Port to E_Port Connectivity

• To verify connectivity between an EX_Port and an attached E_Port
look at the properties for both ports
- The EX_Port will show the Neighbor Domain information and both
ports will s how Port Status as Online
General
Po1No.m>er 4(0x4)
Po1Neme
Po1 proloeol FC
Po1- 20:04:0005:1e:7e:dx63
Po1- SW
Po1 Type
Alowed Po1 Type
~
Po1 Stolus 17(0x11)
F-Pcrl !!di
Febric Id 2 FC
~"'"'''"IOde 20:11:00:05:1e:Clx9f.6f
-
FCR ROl.te< Port Cost 1000 SW
10;00:00;05:1e.lix9t.6f
Nome R11-STOS.B30
Pv4Address 10.255240.46
Pv6Address 8
Backbone Fabric Edge Fabric

EX_Port properties E_ Port properties

To access EX_Port and E_Port properties select Element Manager --+ Ports:
;;; View All - DCFM 10.3.2

~AN ~dlt Y'..iew Q.iscover ~onfigure Monitor Iools t!.elp
Decimal ""' I ""'_.!

_N_a_m_e _ _ _ Name
View All•
B
u
J,
!Aii Levels c;
0
~
~ 1cuc::1 •-
~ Routed Products Grou

~T -
G 0 SJ Virtual Group
B 0 R~~~==~~~"---'-'---------, l!!l!lm!!RB~
Element M~nager ~ t!.ardware
~onfiguration
Firmware Management
~ eorts
~dmin
RSL11_ST8
J
Zoning flouter Admin
Viewing LSAN Fabrics (cont.)

.l"'-?.10.2SS.240.47 FC Router 1!!1~£1
Ooroerll EX.f'olts LSANF...lco LSNIZ..,.. LSNl-

LSAN febriofxl>b« lnt0<-&Ccnfi!l'rtll0not R11.STl)8.830(2)
El LSANF.,.cs !1!«"90LSANfll>ric
mJ SIMIMj)Mi
6.'! R11.sTQ8.851 (1) Oenerll
D ,
Type Edge
febrioSWtch- R11-Sll)8.830
SWtchWll\N 1(UIQ:Oil05. l«Ob;9161
SW.ch PY•
Pod'Wll\N Stole
~Co 1D.OQOQ05 - Exls! Phl""""PO
030100 ~
- - --
ProxyLSAN-.
"
Vendor ""'
~·Tee
P<wt"NNH
220QOQ2& -ed
9.C.e Proxy PO
011101
-- 1
Phy$col Fob Phi"""" Fob Phyalcll PO """"
R11-ST08-B51 6206ei ~
1D.oi
,,
~eTee 220Q002n - 01 !001 1 R11.sT08-B51 6206e1
freePro1. . - - Too1 10.2SS240•7 ADO User-- Role._,

Al 1R ghts Reserved
Open the FC Router window and click the LSAN Fabrics tab
• From here you can view switches involved in LSANs
• Selecting a switch from the navigation pane allows you to view LSAN specific
information for that switch
• LSAN zones
• Physical LSAN devices
• Proxy LSAN devices
Clicking the Manage LSAN Fabric button will launch Element Manager for the
selected switch
Revision 01 10 3 - 44
Viewing LSAN Zones (cont.)

r..: 10.2SS.240.47 FC Rout tr 1!!1~£1
11 fX-l'orls LSAN F- l.SN<Zonn LSAN-

LSAN Zone expor., r>•QmWia(l.p1 I.SAN -Zqne (1)
El LSANZane.s
Eif@fih@il - LSAN_NowZone
F8"'loD 1
F8"'loSWtch- R11-ST-1
FtbneTw>e BactbOr>e
Olfli'loOrric;u
~I.SANO.--.
Venc:IOf _. Port~ StMe
- fob l'hyOiOej Fob- Pl> --
·Soo\111• rec 220000.20" e.bl R11-Sl08-B51 6206el 20"000020"
~·Tee .• 2200.00.~ Ellbl R11-sT08-851 6106e2 20.00.00:20.
v..- ...
Br_ Co
................
10;00:0005 -ed
Stole Proxyfabric:D~
011001
- f o b ~Fob Pl'Y8
2 R11-STOll-830 0301
C 2009 Brocade Co'Timun ca1 ans Syst ems nc .::5

Al 1R ghts Reserved
The LSAN Zones folder tab provides a condensed view of LSAN zones configured
for managed switches
• Selecting a zone from the navigation pane displays information for that zone
display:
• General
•Zone name
• Fabric ID
• Switch name
• Fabric type
• Physical LSAN devices
• Proxy LSAN devices
Viewing LSAN Physical Devices

~ I0.1~U40 41 f( A:outn l!!ll!J CJ
°"*" EX..... LSN<Fwa LS,<HZA>neo j LSAH-.
LSNIO.-E>;lol• '"-Ol ~Oll05,.l6C!!d(2,&ocode°"'""""""S\11-."'l
e!DLSN<DMa o.-•
a l!!i..,._.,._, ,.,,.
...,
..,,.,_
,.,.,,,..-..,.,
,.,,,,
..,~..,":!"
os1«56:dl"'
6)1'4" WW f*tlnt-.e 10'11-s-roe.a>o)
li)22001D'l0.l1 . .0ll _.., ...,,00
6)22:0'.t<D.'10.31t1<t13 NDdll!IVMH .2Q.00.00:051a..S&:cl2d
El&i"°"YO.-.
C!)U):Ol0l05.le.!6:c:t-:
v"""
15 re
- c - -·"'·
rm
1
022m~,, ...,, sur. -
0 22001Xl:l037 •""
LSN<Z•n"
~ • rain: 0 Ftbr'lc Swtt;: rlllft tp
LSAH__,..., l Rt1-STOS-8JO ~
"°"YLSNtl>Ma
Yencb .. Pc.Iv.wt Sl.lle PrcxryF.t:incOPrmyFlbrt: ,ProxyPI> Phylicm!Fatrtcm ~FabrcSWtd'!Nllme fhra:jctlPI) NodlWNrf
&ocodeC. 1000ll005 - I ffll.$W$.8SIOll001 > "11~ ll)Ol()O l0.00000511
C 2009 Brocade Commun cat ens Systems Inc 46

The LSAN Devices tab shows a list of physical and proxy devices. Properties can
be viewed for any device by selecting it from the navigation tree
Physical device properties

• Port WWN, Node WWN, Physical PID
• Vendor
• W hich LSAN zones the device is a member of
• W hich fabrics contain proxies for the device
Revision 01 10 3 - 46
Viewing LSAN Proxy Devices
0..-.1 D"..Jrat1 L..SNtr-..es LSAHZC!Na LSAHC!eYltw

<SAH°""'"e.w" ~~W!J9,0S•~~<$2'1C•.-~-.rcJ
El:JLSAHO.-
a lll~D1Ycn
°""'"
PartWHt
i>•oOOOQ.OS.ta56:c*
e>noooo20S1dit:01
o......nr..-ic
"-YPD .,.,.,
t(fttt..ST-..1)
6) 22.00.CC:ZO.J7 _. 4S Nodt~ 2G.OO:CI005..,e:5&.c8:2cl

El e:i .....
Cl) - ,_,.,,_
_ ...
.,.,..,
, (ft1t..sT08-elO)
C210000Xt'Jl•01 Vtrdlr fll>OClde~SV-. ~

~noooo~'Jl .. u ~:t11W
:!5 '"
"'""'~~
::::...
~'Rl
....I
01'
,..._ ,. r!ID'CD
~-·
C 2009 Brocade Commun cal ans Systems nc ~7

Al 1R ghts Reserved
Proxy device properties

• Proxy PIO
• Port and Node WWN
• Which fabric the device exists in physically
• Physical PID
• Vendor
• Fabric type
• Which LSAN zones the device is a member of
Revision 01 10 3 - 47
BROCADE
EDUCATION SOLUTIONS
Troubleshooting from the Backbone

• Use following commands to verify if devices exi st in the fabric, are
imported and where they are located 1
BB B51 : admi n > lsanzoneshow - s
Device is not imported as the

Fabric other device is not online
Device is configured in an LSAN

but not online or WWN is not
correct in LSAN zone
BB_ B5 1 : admin> fcrproxydevshow
No proxy device found-~-======::::::=====::=:=::=:=::::::::~
Both devices are not online so
BB 51 : admin> fcrphydevshow no devices have been imported
Device WWN Phys i cal
Exists iPID
in Fabr i c Only the device in FID 10 (edge
fabric) is online.
10 10 : 00 : 00 : 05 : 1e : 57 : 7c : 79 010 100

Footnote 1: Example output when both devices are online and correctly
configured .
BB BSl : admin> lsanzoneshow -s

Fabri c ID : 10 Zone Name : LSAN_ Backbonel_ Edgel
10 : 00 : 00 : 05 : le : 57 : 7c : 79 EXIST
22 : 00 : 00 : 20 : 37 : dd : d9 : 29 Import ed
Fabric ID : 100 Zone Name : lsan_ backbonel_ edgel
10 : 00 : 00 : 05 : 1e : 57 : 7c : 79 Imported
22 : 00 : 00 : 20 : 37 : dd : d9 : 29 EXIST
BB B51 : admi n> fcrproxydev show

Proxy WWN Proxy Device Physical State
Created PID Exists PIO
in Fabri c in Fabric
10 22 : 00 : 00 : 20 : 37 : dd : d9 : 29 OlfOOl 100 6206e4 Imported

100 10 : 00 : 00 : 05 : 1e : 57 : 7c : 79 OlfOOl 10 030100 Imported
Total devices d i splayed : 2
The lsanzoneshow command will display all currently-active LSAN zones that the
backbone fabric is enforcing.
lsanzoneshow [-s) [-f fabricID] [-w wwn] [-z zonename]
Search parameters -f, -w , and -z allow searching for LSAN zones based on fa.bric
ID, WWN of an LSAN zone member, or LSAN zone name.
- f fabriclD : Display LSAN zones in the specified fabric.
-w wwn: Display LSAN zones containing the specified port
WWN. (Format XX:XX:XX:XX:XX:XX:XX:XX)
-z zonename: Display LSAN zones with the specified zone name.
-s state: Display state information for the device, valid states include:
Configured - Device is configured to be in an LSAN, but the device is not
imported nor does it exist in this fabric.
EXIST - Device exists in this fabric (the fabric of the zone entry).
Initializing - Device is in an intermediate state. It is not yet imported into
the fabric.
Imported - Device has been imported (proxy created) into this fabric.
In this example, you can see which devices actually exist in the fabric listed (EXIST )
and which ones are projected into that fabric (Imported).
In the f crproxydevshow command output above, we see the two proxy devices
that are currently being shared across the backbone fabric:
• In the edge fabric (Fabric ID 10), there is one proxy device (WWN -
22:00:00:20:37:dd:d9:29), matching the physical device attached to the
backbone fabric (Fabric ID 100). The FC address of the proxy device
(Ox01f001) confirms that it is the first proxy device connected to translate
domain 1.
• In the backbone fabric (Fabric ID 100), there is one proxy device (WWN -
10:00:00:05:1e:57:7c:79), matching the physical device attached to the edge
fabric (Fabric ID 10).
Note: Besides using CU commands to verify devices, ports, zones, proxies, etc.,
verification can be achieved using SAN Health and DCFM
Revision 01 10 3 - 50
Troubleshooting from the Backbone(cont.)

• Additional command used to verify FC router configuration:
- Display routing information: f crrouteshow
- Display or configure preferred translate domain ID: fcrxlateconfig

The fcrproxyconf ig command is sometimes used to define a proxy device

whose PIO does not change. Similarly, the fcrxla t econfig command is
sometimes used to define a xlate domain ID that does not change.
An example edge fabric topologyshow command output:
Brocade : admin> topologyshow
3 domains in t he fabric ; Local Domain ID : 10
Domain : 1
Metric : 10 500
Name : fer xd 1 100
Path Count : 1
Hops : 2
Out Port : 10/4
In Ports : 1/ 1 2 1 /15
Total Bandwidth : 6 . 000 Gbps
Bandwidth Demand : 100 %
Flags : D
<Truncated Output>

• As you begin sharing devices, display the FC router resources
available with the fcrresourceshow command:
BB B5l : admin> fcrresourcesh ow
Daemon Limits :
Max Allowed Currently Used
LSAN Zones : 3000 2

LSAN Devices : 10000 4
Proxy Device Slots : 10000 2
WWN Pool Size Al l ocated
Phantom Node WWN : 24576 4

Phantom Port WWN : 98304 8
Port Limits :
Max proxy devices : 2000
Max NR Ports : 1000
Currently Used(column 1 : proxy, column 2 : NR_Ports) :
216 I 2 l

It is important to note that, when troubleshooting routed SANs, some commands

relate specifically to switches, and others relate specifically to routers. Switch
centric commands would include all the normal Fabric OS commands, some of
which have been updated to include pertinent edge fabric information.
Fabricshow, for instance, displays all front and translate domains in the edge
fabric. router centric commands, for the most part start with 4 ", like
f crresourceshow, and display information specific to the routers, and the
backbone fabric .
In the f crresourceshow command output, you can see the per-backbone and
per-port maximums for the following FC Routing resources:
• LSAN zones
• LSAN devices (proxy or physical devices)
• Proxy device slots (device-to-AL_ PA mappings
• Phantom node WWNs
• Phantom port WWNs (includes ports connecting front and translate domains
(virtual /SLs), translate domain ports for proxy devices, and EX_Ports)
• NR_Ports (stored at every physical port for routing decision purposes)
The scalability limits always override the maximum values in this command output.
Revision 01 10 3 - 52

• Besides reviewing the *show commands, FC-FC routing errors are
reported in the switch error log in the backbone fabric as part of the
FCR error module
2009/11/02 - 21 : 18 : 39 , [ FCR- 1071 ) , 97 , FID 128 , INFO ,
BB_ B51 , Port 4 is changed from non FCR port to FCR port .
• The FCR error module tracks changes in the status of:
- FC Routing service
- Physical and proxy devices in both the edge and backbone fabric
- LSAN zones, EX_Ports, and edge fabric status
- Resource maximums are exceeded
- Port configuration parameters are mismatched

In the example above, the error message indicates that a FC router port 4 changed
from a non FCR port to an FCR port.
The F CR- * error messages are documented in the System Error Message
Reference Manual.
Troubleshooting from the Edge

• For device sharing related issues., use s wi t chs how from the edge
fabrics to verify devices are online:
Edge B30 : admin> switchshow I grep e-port -i
17 -17 031100 id I
NS Online J FC E-Port 50 : 00 : 5l : e7 : ed : b6 : 3e : 02
" fer fd 160 " (downstream) (Trunk master)
• Use the cfgsh ow command from the edge fabrics to verify device
connectivity1:
Brocade : admin> cfgshow
<Truncated Output>
cfg : LSAN CFG 20091103
zone : LSAN Newzone
10 : 00 : 00 : 05 : le : 56 : c8 : 2d
22 : 00 : 00 : 20 : 37 : ef : 43 : 38
22 : 00 : 00 : 20 : 37 : de : Ol : e0

Footnote 1:
The zone s how command will give the same information in slightly different format.
FC-FC routing connectivity can also be verified with the fcping command
Additional Troubleshooting Considerations

• Congestion in a routed fabric is handled much the same as in a
non-routed fabric:
- Add IFLs between the edge fabrics and the routers
- Add ISLs between the switches in the backbone fabric
- Add QoS to prioritize traffic flows across routed links
• With backbone-to-edge routing, some traditional Fabric OS

commands do not display certain information
- From the backbone fabric: issuing isl show does not display IFLs to
the edge fabrics
- From the edge fabric: issues topologyshow displays translate
domains related to backbone-to-edge routing but with very high link
cost

Troubleshooting Data Collection

• The supportsave command collects key information about FC-
FC routing along with other data needed for troubleshooting
- Most FC routing commands are included (see notes)
- Also includes FC routing debug logs
- Should be run on all switches and routers in data path prior to
escalating an issue

The supportsave command output includes information from the following FC

routing-related CLI commands: fcrproxydevshow, fcrphydevshow,
portcfgexport, fcrxlateconfig , fcrrouteshow, lsanzoneshow, and
fcrfabricshow.
Troubleshooting Command Summary

• Helpful router-centric commands: • Helpful switch-centric commands:
- fcrfabricshow - fabricshow
- fcrphydevshow - nsallshow
- fcrproxydevshow - nscamshow
- fcrrouteshow - swi tchshow
- fcrxlateconfig
- fcrproxyconfig
- fcrconfigure
- fcrresourceshow
- lsanzoneshow - s
- portshow
- portcfgexport
- swi tchshow

Summary
• In this module, we discussed:
FC Routing implementation from CU and DCFM
1. Enable FC routing
2. Configuring EX_Ports
3. Define LSAN zones
- Commands and DCFM tools used to verify routing and connectivity
• Edge fabric
• switchshow
• fabricshow
• cfgshow
• Backbone fabric
• lsanzoneshow -s
• fcrfabrics how
• fcrproxydevshow
• DCFM FC Router window
- Troubleshooting topics r,elated to routed fabrics

CFP380 Internal Use Only Adaptive Networking Fabric Profiling
BROCADE
EDUCATION SOLUTIONS
CFP380 Internal Use Only FCIP Theory
Revision 0110 4 - 1
Objectives
Upon completion of this module, you should be able to:
• Describe the Brocade 7800 Extension Switch and Brocade FX8-24
Extension Blade features
• Describe the components of the new Brocade extension products
- FCIP circuits
- FCIP Trunking
- FCIP tunnel
• Discuss new software features:
- Adaptive Rate Limiting
- FCIP QoS

Revision 0110 4-2

FCIP - Overview
• The Fibre Channel-over-IP (FCIP) protocol connects Fibre
Channel switches over an IP network
- IP packets generated by an FCIP-compliant port navigate the IP
network to reach the destination end point
- Implementation uses standards-based TCP, it interoperates with
regular network equipment
Server Storage
Brocade DCX-4S
with FXS-24 Blade Brocade 7800 ~
WAN

Revision 0110 4-3

FCIP - Usage Drivers

• Practical applications for FCIP include:
- Backup, consolidation, mirroring, business continuity solutions
- Environments where IP networking is preferred because of cost and
distance limitations of FC 1
- Existing IP-based networks connect two sites, but not dark fiber
- FICON connectivity over long distance
Remote Data Replication Centralized SAN Backup and Archiving Global Data M1grat1on and Sharing
Secondary Site

Footnote 1: Distances that utilize native FC can span 500km; these solutions
incorporate dark fiber, C/DWDM, and form a single fabric.
For additional FCIP details, reference RFC 3821 - Fibre Channel Over TCP/IP
(FCIP).
Brocade does not recommend FCIP for use in every distance extension scenario:
no technical solution can be all things to all people. FCIP has inherent performance,
reliability, data integrity, and manageability limitations when compared to native FC
solutions. Delay and packet loss may create bottlenecks in IP networks. FCIP can
support very long distances, as long as the carrier network is extremely high
performance and reliable. FCIP is typically deployed when long-haul applications
are not business critical, and do not need especially high performance. FCIP may
not be suitable for tape, since tape usage will often fail if packets are dropped. In
addition to its performance limitations, FCIP troubleshooting and performance
analysis requires evaluating all aspects of the IP LAN and WAN networks in addition
to all FC nodes, switches, and routers, which can make it more complex to manage
than other extension options.
Revision 01 10 4-4
FCIP - Overview (cont.)

• FCIP is a tunneling protocol that allows transparent interconnection
of remote locations through an IP-based network
• From the fabric view, an FCIP link is an ISL, transporting all needed
FC control and data frames between switches
• The IP network is invisible to the fabric
• The FC fabric and protocols are invisible to the TCP/IP network
Server Storage
Brocade DCX-4$
with FX8-24 Blade Brocade 7800 ~
WAN

FC traffic over an IP network

• Interconnection of islands of Fibre Channel storage area networks over IP-
based networks
• Distance Extension over IP LAN/WAN/MAN
Revision 0110 4-5

FCIP - Protocol Mapping

• FCIP transports FC frames across an IP network through an FCIP
tunnel, which is established between the two endpoints of the FCIP
connection
• FCIP is layered on TCP, so FC frames are encapsulated into TCP
packets
TCP J
J
LINK J r---
---P-HY
- --J r> WAN
To Fibre Channel To Fibre Channel

FCIP Tunnel
O 2009 Brocade Commun cal1ons Sys•ems Inc 6

Al R1gh's Reserved
After FC frames destined for devices at the remote side are encapsulated into TCP
packets, a standard IP header is added to each packet. The packet is then sent to
the next hop (usually an Ethernet router).
Revision 0110 4-6

FCIP Encapsulation
• Before an FC frame is sent out through FCI P over an Ethernet link,
the transmitting FCIP port encapsulates the FC frame in the
payload of each of the four protocols in the stack: FCIP, TCP, IP,
and Ethernet
• The receiving FCIP port de-encapsulates the Ethernet, IP, TCP,
and FCIP headers; reassembles the FC frame (if it was
fragmented); and forwards the FC frame into the FC fabric
FC

Revision 0110 4-7

BROCADE
EDUCATION SOLUTIONS
Revision 0110 4-8

Brocade FCIP Offerings

• There are five Brocade platforms that support FCI P:
- 7800 switch
- FX8-24 blade (DCX, DCX-48 director chassis)
- 7500 switch
- 7500E switch
- FR4-18i blade (DCX, DCX-48 and 48000 director chassis)
• Brocade FCIP products have been re-engineered in the latest release
- Frame packaging and flow have changed to provide better performance
• Previous generation and current generation devices are not compatible
• This course focuses on the new generation products. For details regard ~ ng
the 7500, 7500E and FR4-18i products, refer to the course appendix

Revision 0110 4-9

Brocade 7800 Extension Switch
• The Brocade 7800 Extension Switch has two configurations 1

- Brocade 7800 4/2 - 4 x 8 Gbps FC Ports and 2 x 1 GbE Ports2
- Brocade 7800 16/6 - 16 x 8 Gbps FC Ports and 6 x 1 GbE Ports
• Auto-sensing FC ports at 8, 4, 2 or 1 Gbps3
• One GoldenEye2 ASIC for FC ports
• One FCIP subsystem

Al 1R ghts Reserved
Footnote 1: An Upgrade License is required to enable all 16 FC ports and 6 GbE

ports.
Footnote 2: Two GbE ports can be configured for copp,e r (via built-in RJ45) or
optical (via SFP module) cable connectivity. The other four GbE ports offer optical
SFP module only. SFP ports will also accept 1 Gbps copper SFPs, Avago part
number ABCU-5700RZ-BR1.
Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP.
7800 Port Side

• Console Port (RJ-45) • 2 x 1 GbE RJ45 Ports1
• 10/1 OOMb Management Port • 6 x 1 GbE Ports SFP
(RJ-45) • 16 x 8 Gbps FC ports
• USB Port • Trunk groups: 0-7 and 8-15
• Switch ID Pull-out Tab • Supported port types are:
• LEDs: Status, Power, and Port - Fibre Channel - F, FL, E,
EX or Mirror
- GbE - VE and VEX
Port LEO Numbering
Console Port
2 GbE Ports - 1 Gbps - RJ45
D FC Ports
0 FCIPPorts
[:J Combo Ports
Switch 10 Pullout 16 FC Ports - 8f4/2/1 Gbps - SFP 6 GbE Po rts - 1 Gbps - SFP

Al 1R ghts Reserved
Footnote 1: The top two GbE ports (GEO and GE1) are configured for copper via
built-in RJ45. In standard configuration, users have the option of using either the top
two GbE ports, which are configured for copper SFPs, or the bottom two, left-most,
GbE ports (GEO and GE1) which are configured for optical SFPs. The remaining
four GbE ports can use either optical or copper via SFP module up to 1 Gbps. It is
possible to configure GEO as copper and GE1 as optical and vice versa.
GolclonEyo2
FCASIC
Cavium
Combo Ports
Ethernet
I\
nu
.a x 1G SFP
tt tt
2x 2 J;
1GSFP 1G
7800 Non-Port Side

• Two hot-swappable, redundant, 150W power supply/fan assemblies in an
Nl+1 configuration
- Two included as standard
- One required for operation
- Dual integrated fans push airflow from the non-port side to the port side
• Each power supply/fan has an ON/OFF switch, an AC plug, and a status
LED
Power Supply LED AC Plug
·.....
· · · · 1•
·········•
••••••••
·········
•••••••••
••••••••
__ .... ·-
•
•
·······"'··
g·········
••••••••
• , •••••••••
, ........
•••••••• .. (i,.~
l
1SOW Power Supply
with 2 Integrated Fans

Al 1R ghts Reserved
Footnote 1: Fabric OS identifies power supplies left to right as Power Supply #1

and Power Supply #2. The Brocade 7800 requires a minimum of one power supply
(connected and powered on) for switch operation.
Footnote 2: Fabric OS identifies the fan assemblies left to right as Fan #1 and Fan
#2.
Weig1ht: 9.3 kg (20.6 lb)1
Dimensions:
Width: 42.87 cm (16.88 in)
Height: 4.30 cm (1 .70 in) or 1U
Depth: 61 .0 cm (24.02 in)
Power Consumption: 84 Watts nominal, 91 Watts maximum
Both weight and power consumption numbers assume Brocade 7800 with two
power supply/fan FRUs and zero SFPs installed.
Revision 01 10 4 - 12
7800 Port Upgrade Layout

• 7800 Upgrade license adds:
- 12 FC ports for a total of 16 FC ports
- 4 x 1 GbE SFP ports for a total of 6 usable GbE ports
- Support for FICON CUP and Tape Pipelining
0 412 2 Copper Ports OR

D 16/6 Upgrade
Al 1R ghts Reserved
The base unit has 2 GbE ports available for use. They can be the two copper ports,
or the two fibre ports. The default is copper.
The upgrade license is named the 7800 Upgrade license.
FX8-24 Extension Blade (DCX & DCX-4S Only)

• The Brocade FX8-24 Extension Blade has:
- 2 x 10 GbE ports (license required)1
- 10 x 1 GbE ports
- 12 x 8 Gbps FC ports
- One Condor2 ASIC
• Max of 2 blades supported in both the DCX
a.nd DCX-48 as of Fabric OS v6.3.0
• Auto-senses FC link speed at 8, 4, 2 or 1 Gbps
• One Condor2 ASIC for FC ports
• Two FCIP subsystems

Please refer to the Release Notes for the most up-to-date information.
Enabling either of the 2 x10 GbE ports requires a 10 GbE license, which is a slot-
based license.
Footnote 1 : The supported operational modes are:
• 10 x 1 GbE port
• 10 x 1 GbE ports and 1 x 10 GbE port
• 2 x 10 GbE ports
Revision 01 10 4 - 14
Each Cavium processor has 4G of DOR memory.
12 x 8 Gbit/sec FC Ports
•c
a. "'
""IZu
Broadcom Ethernet Broadcom Ethernet
!
10 x 1G FCIP --~ 10G (xGE1)
!
10G (xGEO)
FX8-24 Extension Blade (cont.)

• LEDs: Status, Power, and Port • Supported port types as of
• 3 FC trunk groups1 Fabric OS v6.3.0 are:
- Ports 0 - 11 - Fibre Channel - F, FL, E, EX and
Mirror
- Ports 6 - 7
- GbE-VE
- Ports 2 - 5 and 8 - 11
Port Numbering Diagram
. GE Ports 10 GbE Ports @ 1 Gbps

. FCPorts 12 FC Ports
O XGEPorts 2 XGbE Ports @ 10 Gbps
3 FC Trunk Groups
C 2009 Brocade Co'Timun c a1 ans Syst ems nc 16

Al 1R ghts Reserved
Footnote 1: Typically, port groups for Condor2 AS ICs are 8-port trunk groups.
There are 40 ports on a Condor2 ASIC. As seen in the notes section of the previous
slide, there is a 5-port trunk to each of the four Blaster FPGAs. By definition, a trunk
must be created from the same octet. This means that each octet from a Blaster
trunk octet has 3 ports remaining in the octet. Thus, Brocade engineers used the
available ports to create 2 other 2-port trunks to use some of the remaining ports.
Weig1ht: 3.2 kg (7.0 lb)1
Dimensions:
Width: 3.60 cm (1.41 in)
Height: 42.06 cm (16.56 in)
Depth: 29.89 cm (11.77 in)
Power Consumption: 235 Watts nominal
Approximate weight with 0 SFPs installed. The minimum power consumption of the
Brocade FX8-24 Extension Blade is 235 watts with 0 optical SFPs installed running
at 8 Gbps.
FCIP Over 10 Gigabit Ethernet

• 10 Gigabit Ethernet (10 GbE) is a new feature available with the
FX8-24 platform
• 3 Modes of operation for the Ethernet ports on FX8-24 :
- 1G Only Mode: 10 x 1 Gbps ports are available for use (GEO through GE9)
- 10G Only: 2 x 10 Gbps ports are available for use (XGEO and XGE1) 1
- Dual Mode: 10 x 1 Gbps ports and 1 x 10 Gbps port available for use (GEO
through GE9, XGEO) 1
XGbE
lG Mode
10G Mode
Dual Mode
-
GEO GEl GE2 GE3 GE4 GES GE6 GE7 GE8 GE9 X<iEO XGEl

Footnote 1: Requires the 1OGbps license.
Revision 01 10 4 - 17
Available Licenses
• Fabric OS v6.3.0 available licenses for new extension products:
- 87800 Port Upgrade
• B7800 4/2 to B7800 16/6
- 10 GbE FCIP(FX8-24 only)
• Slot-based
- Integrated Routing
- Advanced Extension
• Slot-based
• FCIP Trunking
•Adaptive Rate Limiting
- Brocade Accelerator for FICON
- FICON Management Server (CUP)

87800 and FX8-24 Feature Matrix

Feature 87800 4/2 87800 16/6 FX8-24
Number of Fibre Channel ports 4 16 12
Number or Gigablt Ethernet (GbE) ports 2 6 10
Number or 10 Glgabit Ethernet (10 GbE) ports NIA NIA 2 (requires 10 GbE' licenS'8)
Fibre channel routing Requires Integrated Routi ng Requires Integrated Routing Requires Integrated Routin g
license license license
VEX_Port support Requires Integrated Routing Requires Integrated Routing Not lnduded'
hcense license
FCIPtunnel Included lnduded Included
Number or FCIP tunnels 2per syslem Up to sys1em hmlt 1 Up to system limit•
FCIP trunking Requ~es Advan ced Extension Requires Advanced Extension Requires Advanced Extensi on'
license license license
Adaptive Rate Umhlng Requ~es Advanced Extension Requires Advanced Extension Requires Advanced Extensi on'
license ricense Hcense
FCIP Qo:S Included lnduded lnduded
Hardware Compression Included lnduded Included

Software Compression Included tnduded Not lnduded
Fast Wr11e over FCIP Included lnduded Included
Fast Wrtte over FC N/A NIA N/A

Open Sy stems Tape Pipelining over FCIP N/A lnduded Included
FICON XRC emulation and N/A Requires FICON Accelerator R·equlres FICON Accelerator>
Tape Pipelining over FCIP license 6cense
FICON CUP N/A Requires FICON CUP license Requires FICON CUP licerlse
TCP performance graphing In Web Tools Included lnduded Included

Footnote 1:
FCIP Tunnels:
•87800 - up to 8 VE_Ports
•FX8-24 - up to 20 VE_Ports
FCIP Trunking:
•87800 - up to 4 circuits per trunk
•FX8-24 - up to 4 circuits per trunk for 1 GbE; up to 10 circuits per trunk for 10
GbE
Footnote 2: 10 GbE, Advanced Extension and FICON Accelerator licenses for FX8-
24 supported in DCX and DCX-48 are slot based licenses
Footnote 3: Not supported in initial Fabric OS v6.3.0 release.
Optics
Brocade Branded Brocade orderable PIN
1/2/4 Gbps and GbE SWL XBR-000139 (1-pack), XBR-000141 (8-pack), XBR-000158 (128-pack)
GbE copper XBR-000190 (1-pack)
1/2/4 Gbps LWL - 4km XBR-000142 (1-pack) and XBR-000143 (8-pack)
1/2/4 Gbps LWL - 1Okm XBR-000144 (1-pack) and XBR-000157 (8-pack)
1/2/4 Gbps ELWL - 30km XBR-000146 (1 -pack)
2/4/8 Gbps SFP+ SWL XBR-000147 (1 -pack), XBR-000148 (8-pack), XBR-000159 (128-pack)
2/4/8 Gbps SFP+ LWL- 1Okm XBR-000153 (1-pack), XBR-000172 (8-pack)
2/4/8 Gbps SFP+ ELWL - 25km XBR-000174 (1-pack)
10 GbE SFP+ SR XBR-000180 (1 -pack), XBR-000181 (8-pack)
10 GbE SFP+ LR XBR-000182 (1-pack), XBR-000183 (8-pack)
~:~;:.~eCA ~ ~
57·\000012.01 2:1C,,tC.J~
M•d• In Ch•n• ~
SN UAAo&076200000MH
1111 1 1111 1 1111111111111 I Ill
SFP+

Certified (Support is through the SFP vendor)

1/2/4 Gbps ELW L - 50km
1/2/4 Gbps ELWL- 100km
1/2/4 Gbps CWDM - 1470nm - 80km
1/2/4 Gbps CWDM - 1490nm - 80km
1/2/4 Gbps CWDM - 151 Onm - 80km
1/2/4 Gbps CWDM - 1530nm - 80km
1/2/4 Gbps CWDM - 1550nm - 80km
1/2/4 Gbps CWDM - 1570nm - 80km
1/2/4 Gbps CWDM - 1590nm - 80km
1/2/4 Gbps CWDM-1610nm - 80km
1/2/4 Gbps DWDM 80km (25 wavelengths from 1530nm-1560nm)
21418 Gbps CWDM 40km(8 wavelengths from 1290nm-1430nm)
For the most up-to-date support, refer to the following site:
Virtual Fabric Support

• FXB-24 support for Fabric OS v6.3.0
- 8 Gbps FC ports can be assigned to any logical switch
- VE_ Ports can be configured in any logical switch
- VE Ports cannot be used as XISL
- VEX_Ports are not supported in v6.3.0
• No Virtual Fabric support for 87800 in v6.3.0

- Cannot create logical switches on 87800
- 87800 can be connected to other switch in Virtual Fabric mode
• To learn more about Brocade Virtual Fabrics, please refer to the

Brocade AFS141 web-based training course

BROCADE
EDUCATION SOLUTIONS
CFP 380 Internal Use Only FCIP Theory
High-level Path Definition

Tunnel
. . .. ' FC Switc h ~--t'--------.:l~-M-ul-tip-le-Ci-rcu-its-O
1. FC ingress
2. VE Port
3. FCIP tunnel (contains trunked circuits 0 and 1)
4. Multiple circuits (circuits 0 and 1)
5. Ethernet interface, one physical GbE port for each circuit
• Multiple circuits can reside on one physical port

C2 and GE2 =Condor2 and GoldenEye2 ASICs
High-level Path Definition (cont.)

• Circuits and tunnels have a 1-to-1 relationship with matching
circuits and tunnels configured on partner device(s)
FCIP Circuits FCIP Circuits

FCIP Tunnel Concepts - VE_Port

• An FCIP tunnel is represented in a Brocade fabric as a Virtual
E_Port (VE_Port)
- Just like an E_Port, except underlying transport is IP not FC
• The VE Port emulates an E Port on either end of the IFCIP tunnel:
- The FCIP platforms at both ends of the links merge to form a single
fabric
- VE_Ports do not use FC flow control mechanisms (BB Credits); they
utilize TCP flow control mechanisms1
- VE_Ports do not support FC ISL trunking , but they do support
exchange-based routing (Dynamic Path Selection)
• Note that with FCIP Trunking, it is recommended to implement a multiple
circuit trlLlnk instead of having multiple VE ports to the same fabric

Footnote 1: FC flow control mechanisms include R RDYs and ACKs. FC

communications also utilize long distance modes, BB credits, and VC channels.
Valid VE Port Values

• V E_Ports do not have to be associated with a particular GbE port1
10G mode
10G mode

Footnote 1: There are 8 VE_ Ports for 6 physical ge ports on the 7800 16/6. An
FX8-24 blade can support 20 VE_Ports, and therefore 20 FCIP tunnels. Each FCIP
tunnel is associated with a specific VE_Port. On FX8-24 blades, and on the 7800
switch, VE_Ports do not have to be associated with a particular GbE port.
VE_Ports 12 through 21 may use GbE ports geO through ge9, or they may use XGE
port 1. VE_Ports 22 through 31 can only be used by XGE port 0. The total
bandwidth cannot exceed 20 Gbps.
FCIP Concepts - VEX_Port

• The Brocade 7800 supports FC-FC routing over an FCIP tunnel,
creating a Virtual EX_Port (VEX_Port) 1
- Allows long-distance FCIP connections with fabric-to-fabric isolation
- VEX_Ports are no different from EX_ Ports, except underlying transport
is IP rather than FC
• There are a few connectivity rules with VEX_Ports:
- A VEX_Port connects only to a VE_Port - it may not connect to
another VEX Port
- There can be multiple VEX-to-VE port connections between a
Backbone fabric and an Edge fabric
- EX-to-E and VEX-to-VE connections to the same Edge fabric can co-
exist in Fabric OS v5.2 and higher fabrics

Footnote 1: As of Fabric OS v6.3.0, VEX_Ports are not supported on the FX8-24.
FCIP Circuit
• An FCIP circuit is a logical connection created between two IP
address end points
~
X) WAN )(
IP Router IP Router
10.0.0.1 10.0.1.1
10.0.0.2 ............................................................................................... 10.0.1.2

FOP Circuit
• When created, a committed rate can be configured

- Each circuit supports a rate of 1.544 Mbps to 1 Gbps

The 7500E and 7500 switches and the FR4-18i blade support only one connection
per GbE port, so strictly speaking, the FCIP circuit concept does not apply
FCIP Tunnels
• Collection of one or more FCIP circuits that create one logical
connection between 2 FCIP devices1
• Each FCIP tunnel presents a VE_Port to the FC fabric
• Tunnel can span multiple physical ports
• Multiple FCIP circuits from different 1 GbE interfaces added to a
VE/FCIP tunnel increases the bandwidth available to an FCIP
tunnel
-·
87800
Tunnel Example: 87800
-·
- 2 Gbps tunnel created
- 4 circuit aggregated tunnel
- Each circuit 512 Mbps

VE ports and GbE ports are no longer 1 :1 associated as they were on the 7500 and
FR4-18i.
VE and GbE ports can have a 1:1 association, but they are not limited by the
design.
Footnote 1: Configuring a tunnel with more than one circuit requires an Advanced
Extension license. Without a license present, a second circuit will not be allowed to
be configured. The administrator will receive a message stating as such.
Revision 01 10 4 - 29
FCIP Trunking
• When more than one circuit is added to a tunnel , the bandwidth of
all active circuits within the tunnel will be aggregated together to
form a trunk (requires Advanced Extension license )1
• Purpose is two-fold:
- Increase tunnel bandwidth
- Provide failover paths in the event of a network failure
• The functionality of FCIP trunking will be transparent to the
applications using the FCIP tunnel
Tunnel 1

FCIP trunking requires multiple FCIP circuits, therefore FCIP trunking cannot be
implemented on the 7500E, 7500 switch, and the FR4-18i blade.
Footnote 1: Configuring a tunnel with more than one circuit requires an Advanced
Extension license. Without a license present, a second circuit will not be allowed to
be configured. The administrator will receive a message stating as such.
FCIP Trunking Concept
Circuit 1
+ + +
Circuit 2
(
• Link loss recovery is handled by a TCP supervisor that monitors the

trunk and encapsulates the circuit TCP sessions within a FCIP trunk
• Supervisor retransmits lost packets due to link loss
• Ensures FC frames are delivered in order (IOD)
• For operational links, the TCP sessions within the circuit handle the
floss

Time out values:

• Default FICON is 1 sec
• Default FC is 4 sec
• Depending on the solution being extended, the time out values may need to be
changed from default.
Circuit Metric
• Each circuit will be configured with a metric of 0 (active) or 1 (standby)
• The metric will be used by the tunnel supervisor to determine which circuit
or circuits will be used as active circuits
• Metric 0 circuits have the lowest metric and will be designated the active
circuits and will be used for all data transfers
• Metric 1 circuits are classified as standby circuits. It is in standby mode in
the event that all metric 0 circuits fail .
MetricO
MeuicO
Metric 1
87800

Example tunnel/circuit creation:

A tunnel using VE_Port 16 is created with an initial circuit 0 with a maximum rate of
1Gbps, metric of 0.
• IPPM_LINK_UP.bandwidth = 1Gbps
Circuit 1 is added to Tunnel 16 with a max rate of 500Mbs, metric 0.
• TUNNEL_UPDATE.bandwidth =1.5 Gbps
Circuit 2 is added to Tunnel 16 with a max rate of 1Gps, metric 1.
• Since this is a higher metric, it is considered a standby and no
TUNNEL_UPDATE is generated.
Circuit 3 is added to Tunnel 16 with a max rate of 1Gps, metric 1.
• Since this is a higher metric, it is considered a standby and no
TUNNEL_UPDATE is generated.
Example circuit failures/recovery (after initial creation state)

Circuit 1 fails.
• Since this was a low metric circuit (active) we will try to re-establish, no
TUNNEL_UPDATE is sent to CP (and no updated will be sent once it re-
establishes).
Circuit 0 fails, 1 is still failed .
• Since this is the last of the low metric circuits, the tunnel will failover to the next
lowest circuits, which are circuits 2 and 3. Since the combined bandwidth of
circuits 2 and 3 is different than circuits 0 and 1, a
TUNNEL_UPDATE.bandwidth = 2.0 Gbps is generated to the CP.
Circuit 0 and 1 come back on line.
• Since these are again the low metric circuits, we will resume traffic on these
circuits and 2 and 3 become the hot standbys. Since the bandwidth has
changed again, a TUNNEL_ UPDATE.bandwidth= 1.5 Gbps is generated to
the CP. Note that the bandwidth was reduced.
1
Example circuit deletions (after initial creation state);

Circuit 1 is deleted.
• Since this was a low metric circuit, all traffic will be directed to circuit 0. Again,
the bandwidth has changed so the tunnel will generate a
TUNNEL_UPDATE.bandwidth= 1.0 Gbps to the CP.
Deleting circuits 2 or 3 at this point would have no affect since they are standby
and the bandwidth is not included in the calculations.
Example circuit metric changes (after initial creation state);
Circuit 1's metric is changed to 1.

• Since this was a low metric circuit, all traffic will be directed to circuit 0. The
bandwidth has changed, so the tunnel will generate a
TUNNEL_ UPDATE.bandwidth= 1.0 Gbps to the CP. This would basically
have the same affect on bandwidth as if deleting the circuit instead.
Circuit 3's is changed to metric 0 (circuit 1 still has a metric of 1)
• Since this was a high metric circuit changed to a low. The bandwidth has
changed, so FTNL will generate a TUNNEL_UPDATE.bandwidth 2.0 Gbps to
the CP (circuit 0 and circuit 3 are now "active", circuits 1 and 2 are now in
"standby" mode).

• Configuration considerations:
- Max of 10 FCIP circuits can be configured on a 10 GbE port
- FCIP circuits are limited to a maximum committed rate of 1 Gbps
- No tunnel or trunking support across multiple 10 GbE ports1
- No 10 GbE to 1 GbE support as of Fabric OS v6.3.0

Footnote 1: Each 10 GbE port is assigned to a different FCIP subsystem. Because

of this, tunnels cannot be created traversing the two ports. All circuits that reside in
a tunnel must belong to the same FCIP subsystem.
Revision 01 10 4 - 34
CFP 380 Internal Use Only FCIP Theory
Circuit Scalability
Max. Circuits Max. Circuits Max. Circuits Max. Circuits

per per per per
1 GbE Port 10 GbE Port Tunnel System
87800 4/2 4 N/A 4 8 (2 tunnels)
8780016/6 4 N/A 4 32 (8 tunnels)
FX8-24 blade
1 GbE Only Mod'e 4 N/A 4 40 (10 tunnels)
Dual Mode 4 10 4 for 1 GbE ports 201

10 for 10 GbE ports
10 GbE Only Mode N/A 10 10 202
Footnote 1: 10 tunnels for all 1 GbE ports and 10 tunnels per 10 GbE port
Footnote 2: 10 tunnels per 10 GbE port

FCIP Tunnel Scalability
Max. Tunnels Max. Tunnels Max. Tunnels

per per per
1 GbE Port 10 GbE Port System
87800 4/2 1 N/A 2
87800 16/6 4 N/A 8
FX8-24 blade
1 GbE Only Mode 4 N/A 10
Dual Mode 4 10 20
10 GbE Only Mode NIA 10 20

3 Modes of operation for the Ethernet ports on FX8-24:

1G Only Mode: 10 x 1 Gbps ports are available for use (GEO through GE9)
10G Only: 2 x 10 Gbps ports are available for use (XGEO and XGE1 )1
Dual Mode: 10 x 1 Gbps ports and 1 x 10 Gbps port available for use (GEO
through GE9, XGEO)
Internal IP Routing Tables

• Subnet rules are used when configuring IP addresses1
- The 87800 contains one FCI P complex 2
• Each GbE port that participates in the same tunnel must be configured with
a different subnet3
- The FX8-24 blade contains two FCI P complexes 4
• On 10 GbE ports, IP addresses assigned to circuits in the same trunk can
contain addresses from the same subnet or different subnets
• Each 1 GbE port must be configured with a different subnet
. geO
0 gt1
• gel
0 gel

Footnote 1: IP address subnets are used so that the TCP supervisor can route
packets to the destination subnets. The FCIP subsystem contains an IP routing
table that directs packets with a tunnel destination to the correct circuits and
tunnels.
Footnote 2: The 87800 contains 6 x 1 Gbps connections between field-
programmable gate arrays (FPGAs) and the Ethernet ports.
Footnote 3: If multiple circuits are configured on the same physical port, they can
contain addresses from the same subnet or different subnets.
Footnote 4: For the FX8-24, each FCIP complex has ten 1 Gbps connections
between FPGAs and the Ethernet ports. This is the reason that the XGE ports
require 10 circuits to achieve 10 Gbps bandwidth.
FCIP Features - Selective Acknowledgement

• Packet loss significantly degrades FCIP performance, lost data
needs to be retransmitted
- Each lost packet requires a separate ACK response packet.
• To mitigate this, Fabric OS supports Selective Acknowledgement
(SACK) - (default ON)
- When SACK is disabled on a receiving VENEX_Port, each lost packet
requires a separate ACK response packet
- When SACK is enabled on a receiving VENEX_ Port, the
retransmission of multiple lost packets can be combined in a single
ACK packet1
- Result: Fewer ACK packets, faster recovery time, better performance

Al R1gh's Reserved
·Packet loss re-transmissions are compounded when errors are bursty. Selective
Acknowledgement (SACK) is an extension to a protocol which allows the
acknowledge reception of specific packets or messages.
The SACK option RFC 2883 [18] allows the receiver to acknowledge multiple lost
packets in a single ACK, enabling faster recovery. An FCIP Entity may negotiate
use of TCP SACK and use it for faster recovery from lost packets and holes in TCP
sequence number space.
Footnote 1: SACK improves loss detection, retransmission techniques, and
enables faster recovery.
Hardware Compression
• Both the Brocade 7800 and FX8-24 have hardware compression
capabilities
• Performed by hardware on an individual FC frame and is
configured on a tunnel basis
FCIP Subsystem
Compress? FCIP Encapsulation
DCJDIIl CJ CJ CJ CJ [!] CJ CJ ~ IllCJ I I I I I 11 I ii I I ic::f>

Hardware compression is performed at FC ingress and de-compression is

performed at FC egress.
Compression before encapsulation allows the TCP and FCIP headers to be visible
on the network.
Software Compression
• The 7800 provides an additional feature named Advanced
Compression , which is software compression
- Meant to provide higher compression ratios when dealing with lower
bandwidth
• Compression is configured by creating or modifying a tunnel using
the port cf g fcipt umnel command
• Compression options1 :
- 0: Off
- 1: Standard (hardware compression)
- 2: Moderate (7800 switch only)
- 3: Aggressive (7800 switch only)

Footnote 1: A value of 1 enables hardware compression. The 7800 switch provides

two additional levels of software compression. Settings 2 and 3 provide
incrementally higher compression ratios that can be used to improve performance
on slower links. A value of 0 disables compression.
BR0CADE
1
EDUCATION SOLUTIONS
Two Tunnel Example
2 Tunnel Example: D Tunnel 1 to Miami

- 2 x 2 GbiVsec tunnels created O Tunnel 2 to Dallas
- 4 circuits per tunnel
- Each circuit 512 MbiVsec
- 8 total circuits
- Provides load balancing and failover
VE16
VE17
geO
gel
ge2
Circuits
G
ge3
Circuits
B7800

Al 1R ghts Reserved
Four Tunnel Example

4 Tunnel Example: 0 Tunnel 1 to Denver
- 4 x 1 Gbit/sec tunnels created D Tunnel 2 to Miami
- 4 circuits per tunnel 0 Tunnel 3 to Dallas
- Each circ\Jil 256 Mbit/sec • Tunnel 4 to Los Angeles
- 16 total circuits
- Provides load balancing and failover
geO
gel
ge2
ge3
87800
C 2009 Brocade Co'Timun ca1 ans Systems nc .t:3

Al 1R ghts Reserved
Tunnel Example Using Metrics

• 3 circuits:
- 2 x metric 0
- 1 x metric 1
• Provides load balancing and failover
• Traffic will not traverse T3 until no metric 0 paths are available
• OC3 = 155 MbiUsec
• T3 =45 MbiUsec
...
B7800

1OG Mode Example
....----''-----~
.
1ox 1~sec circuits "' 10
Gbit/secTunnel ( \ _
- \...,,
Giami
xgeO
xgel
FX8-24
D l 0 Gbit/sec Tunnel
D 5 Gbit/sec Tunnel
• 5 Gbit/sec Tunnel

Al R1g~·s Reserved
Dual Mode Example
~-~-~ 10 x 1Gbit/sec circuits= 10 Gbit/sec Tunnel

xgeO
xge 1 ·Disabled
geO
gel
ge2
ge3
geS
ge6
gel
ge8
O 10 Gbit/sec Tunnel
Note: The m aximum circuits that can be configured for a
D 4 Gbit/secTunnel
• 4 Gbit/secTunnel
tunnel using 1 GbE ports is 4 circuits.

Ports ge4 and ge9 are available to be used, but are not in the example above.
BR0CADE
1
EDUCATION SOLUTIONS
FCIP ARL Support

• Adaptive Rate Limiting (ARL) provides for an adaptive committed rate
configuration on an FCIP circuit
- This is implemented by configuring a minimum and a maximum committed rate
• ARL allows the traffic rate on a circuit to float between the minimum and
maximum
- If there is traffic demand from FCIP and the network connection is clean (no
retransm its) then the rate will grow
- If TCP reports retransmits, the rate will retreat back to the minimum
- If traffic demand subsides, the rate will shrink backwards
- Growth is accomplished by testing the ceiling and stepping the available
bandwidth up gradually
• This provides more efficient bandwidth sharing between applications using
the same network infrastructure
• Requires the Advanced Extension license

Rate Limit Problem - Oversubscribed

Bad Solution
•Aggregate Egress Bandwidth= 2 x OC-3 (155 Mbit/sec)= 310 Mbit/sec
• Oversubscribed during normal operation
• Results in a massive number of dropped frames
Port Rate Limit= OC-3
Port Rate limit = OC-3

Best practice is to use a commit rate that uses 90% of the bandwidth allocated to
the FCIP traffic. In this example, both devices are configured for 100% of the total
bandwidth of the WAN gateway, which means that the router is oversubscribed 2:1.
This configuration can lead to many dropped frames and errors.
Rate Limit Problem - Underutilized During Outage

Better Solution - Not Perfect
• Aggregate Egress Bandwidth =2 x %(0C-3) = 1S5 Mbps
• No oversubscription during normal operation
• Less than 155 Mbps at times of failure of equipment or when application is
offline
Port Rat~ Limit = 1 /2 OC-3

The configuration above is a better solution than the previous slide, but still has
shortcomings. While the router is not oversubscribed, the bandwidth available for
the WAN is not fully utilized when there is either a failure of one of the devices, or a
simple case of a device needing less bandwidth than what is configured. This can
leave the link underutilized during certain times.
ARL Solution - Fully Utilized Connections

Best Solution
•Aggregate Egress Bandwidth= 1 x OC-3 = 155 Mbps 1
• Not oversubscribed or underutilized
• Rate Limiting automatically changes with network conditions
ARL
Minimum 1/2 OC-3
Maximum OC-3
ARL
Minimum 1/2 OC-3
Maximum OC-3

Footnote 1: Best practice dictates setting maximum commit rates to 90% of

physical connection.
The best solution would to use Adaptive Rate Limiting (ARL) to better utilize the
available WAN bandwidth. By configuring each device with a minimum commit rate
of 50% of the allocated bandwidth, each device is then guaranteed a minimum
amount of bandwidth at all times. If an outage occurs in one of the devices, or a
device is not using all of its minimum committed rate bandwidth, the other active
device can utilize the extra bandwidth up to its configured maximum commit rate.
Revision 01 10 4 - 51
ARL - Application Bandwidth Optimization

Traffic for Tape Tape ramps up using
the spare bandwidth
100% made available by
reduced Replication
90% demand
80%
70%
60% ~Tunnel 1
50%
--0-Tunnel 2
40%
- - Min Commit
30%
- Max Commit
20%
10%
0%
0 0 I.{) 0 I.{) 0 I.{) 0 I.{)
0 N N C'0 C'0 ~ ~ U"> LO
..-- .,_. ~ .,_. .,_. .,_. .,- .,_. N
~
N
During night time, this

Traffic for Replication application is using
less bandwidth

ARL - Optimization After a Path Failure

With 1 interface When a 2M interface is e nabled the network About 20 seconds to
online, all available experiences a congestion event and the first find ceiling
bandwidth is interface drops its rate dlown to the minimum
configured rate. From here it will seek the
ceiling again
0
90° ~Tunnel 1
80o/o 'ltl--~-\-~~~~-,,.£~~~~----:::-J!l'!-::..._~~~~-
-a-Tunnel 2
70%
60% -Max Commit
50% - Min Commit
40%
30% This interface steps up
20% to fully utilize the
10% ~-~...-~~~~.r---t.~~~~~~~~~~--
- available WAN BW. It
oo stops seeking a ceiling
at the maximum
............. ....- ......
The 2nd interface ..................... ...- N N N N N
starts by c laiming
its minimum Offline equipment/link results Blips are TCP
configured rate in available BW testing the ceiling

BR0CADE
1
EDUCATION SOLUTIONS
QoS - SID/DID Based Traffic Prioritization

• QoS SID/DID traffic prioritization is a licensed Fabric OS feature
(Adaptive Networking)
• Allows you to categorize the traffic flow between a given host and
target as having a high or low priority
- For example, you could assign online transaction processing (OLTP) to
high priority and backup traffic to low priority
• This feature is based on an extension of original Brocade virtual
channel model:
- Divides an ISL into 8 Virtual Channels (VCs) to ensure that traffic of
various priorities can travel across the link without disruption
- Data flows use 4 channels (2 - 5) and share the same priority in this
model
C: 7009 Br cac Co mun ca1 c ns Sy• ems Inc 55

, 11 >, .... , ~es1 '\ .. ,
. ...T!'El.'£1m:i1i
- 1
0 Class F
1 Class 2 Ack/Link Control

......
2 Data
Data All
3
Data
4 Data
Traffic
5
6
Data
Class 3 Multicast
--
7 Broadcast/Multicast
Brocade's original virtual channel model (pre-Condor2/GoldenEye2) divides an ISL

into 8 virtual channels to insure that traffic of multiple priorities can travel across the
link at the same time, without being disrupted, or disrupting other traffic.
SID/DID Based Traffic Prioritization

• Illustration of QoS data flow
VCO DDDl}Tx Queue {

High DDDDDDDI
Medium DDDDDDI ' - - - - • ====:::::
Low I i•l• }lit• J• I !' . ' . ' . ':
. : : : : : ::
RxQueue

The QoS feature only comes into play if there is contention on the link. If there is no
congestion on the link QoS will not engage.
The order of operations during congestion is as follows and repeats as necessary:
1. VCO then,
2. 6 frames of High priority traffic then,
3. 3 frames of Medium priority traffic then,
4. 1 frame of Low priority traffic
Results of QoS on a congested link:
000~~~1
vco } { O DDI
:1::tm===o;; ;O
; ;;: ; ;g; ;: 1 TUl
x •o u.eue
_ _. ,
Low l• T• I• l• ll I;::.
._= _=_=_=_=_=_==._..!•= I
Rx Queue
Revision 01 10 4 - 56
FCIP QoS Support

• Provides for appropriate bandwidth usage when multiple FC QoS
priorities are running over the same FCIP circuit
- Allows QoS traffic to be distributed as follows:
• F-Class: Gets what it needs. This is the highest priority.
• QoS High: Gets at least 50% of the bandwidth
•QoS Medium: Gets at least 30% of the bandwidth
• QoS Low: Gets at least 20% of the bandwidth
- QoS allows each priority to consume the full bandwidth of a circuit if the
bandwidth is available (i.e. when only one QoS priority is in use)

FCI P Circuit (cont.)

• Each circuit includes 4 TCP connections1 :
- F-Class
- QoS High
- QoS Medium
- QoS Low
Tunnel
0 F-Class /
• High
/
0 Medium
O Low
/

Footnote 1: TCP port used is 3225.
Per Priority TCP QoS

• Extends QoS zoning that exists in fabric
- Identifies incoming virtual channels on a FC ISL
• QoS is not enforced unless there is congestion
• Lower priority flows are never limited if higher priority flows are not
currently utilizing the BW
• Different TCP sessions can be treated independently
• High Prio<lty
LI ~lurn Priority
• Low Prlo<ily

Summary
• Brocade 7800 Extension Switch and Brocade FX8-24 Extension
Blade features include ... ..
• Brocade extension products components include
- FCIP circuits
- FCIP Trunking
- FCIP tunnel
• Software features include:
- FCIP QoS
- Adaptive Rate Limiting

BROCADE
EDUCATION SOLUTIONS
FastWrite and OSTP

• FastWrite arnd OSTP (Open Systems Tape Pipelining ) provide a
means to miitigate latency thereby increasing performance of
certain storage applications such as:
- Remote Data Recovery (RDR)
- Backup/Recovery/Archive (BURA)
• FastW rite accelerates SCSI write I/O's over FCIP
• OSTP accellerates SCSI tape read and write I/O's over FCIP
• FW and OSTP reduce the effects of latency by
- Initiating data transfer sooner
- Reducing the number of round trips across th,e WAN
• F astWrite arnd OSTP are supported over VE and VEX_Ports

Al R1gh's Reserved
SCSI Writes Without FastWrite

• Write command latency:
- SCSI write commands require at least 2 x RTT
- SCSI Read commands require only one RTT
• - Write Command R
equest
Transfer Ready
-
2xRTI
• - Data Transfer
Status
-
Al R1gh's Reserved
SCSI Writes With FastWrite

• Entire data sequence is transported without waiting for the transfer ready
• FastWrite reduces write command latency to one RTT or roughly 50% reduction in
response time
• FICON traffic is not affected by the FastWrite engine
lnitator 87800 87800 Target
:_;
FCIPover -......
- Write Command R
...
-
equest
Transfer Ready
...
Data Transfer
~ransfer Ready
lxRTI Status
-
Al R1gh's Reserved
OSTP Write
Backup Server Location • Remote router acts as server to tape

controller from a SCSI perspective
Write 1/0 Local • When 1/0 commands are complete, Write
Server Commands MP-75008 Multiple Filemark is sent.
Write I/Os • When all data is successfully written to tape,
~ Write Filemark response will indicate success
•-"~
LocalACKs
Write Filemark CMD Tape Device Location

Write Filemark RSP
Write 1/0
Requests
~ _.._ _..____.llill
~ "'lo!.il~...
Remote Acks
MP-75008
• Possible because tape is a sequential device

• Locally Acknowledged 1/0 requests
• 110 operations and data are sent into· the
pipeline
0 2009 Brocade Commun cal1ons Sys•ems Inc 66

Al R1gh's Reserved
Revision 011 O 4 - 66
Open Systems Tape Write

No OSTP
Host Initiator Local Channel Extender Remote Channel Extender Tape Target
(Proxy Target) (Proxy Initiator)
~
-· ,. ~
~~ ~ ..:ii
~
FC
•=
c......__,.
II
FCIPWAN ·=
·=
~ ,.
II
II
FC
ii
WRT CM01 ~
WRT_O.t01 ["""""'" WRT CM01
XFER ROY 1A XFER ROY 1A XFE.R ROY 1A
DATA OUT 1A
DATA OUT 1A
'
- Ii·
DATA OUT 1A
DATA OUT 1A :
DATA OUT 1A
OATA_OUT 1A
"i'
XFR ROY 18
XFER_ROY 18
DATA OUT 18 1- XFR_ROY 18
DATA OUT 18 I~ F;
DATA OUT 18
:
~
OATA_OUT 18
RSP1
WRT_CMD2
I DATA~OUT 18
RSP 1
,,
1•1,:
l'• '
DATA OUT 18
RSP1
II
WRT C /11102 WRT_CMD2
XFER_ROY2 XFER_RDY2 XFER ROY 2
DATA OUT 2
·I
OATA_OUT2
: DATA OUT2
DATA_OUT 2
f DATA OUT2 :? :
DATA OUT 2
RSP2
WRITE FILE.MARK
... RSP2
WRITE FILEMARK
...
RSP2
WRITE FILE.MARK
FCP_RSP
,, ..,
RSP
FCP_RSP
O 2009 Brocade Commun ca! ons Sys•ems Inc 67

Al R1gh's Reserved
OSTP Write
FC FCIPWAN FC
WRT_CMD 1
WRT_CMDl
XFER ROY 1 WRT_CM0 1
DATA OUT 1 XFER_RDY 1A
DATA OUT 1
DATA_OUT 1A
OATA_OUT 1
DATA_OUT 1
DATA_OUT1A
RSP 1
~
XFER_ROY 18
WRT_CMD2
WRT_CMD2 DATA_OUT1B
XFER ROY 2
DATA_OUT 2
OATA_OUT 2
DATA_OUT1B
RSP 1
OATA_OUT2 WRT_CM02
DATA_OUT2
RSP2 XFER_RDY2
WRITE_FILEMARK DATA_OUT2
WRITE_FILEMARK
DATA_OUT2
RSP2
WRITE_FILEMARK
RSP RSP
RSP

Al R1gh's Reserved
OSTP Read
• When the R part of DR is needed read throughput would be poor,

RTO would be long, and that is bad
- OSTP Read solves this problem
• Allows tape read transfers at greater performance at e xtended
distances
• New feature in FOS 6.2
- OSTP Read/Write requires the "High-Performance Extension over FCIP" license
• Both Read/Write are enabled
- Read or W rite alone cannot be enabled
• Takes advantage of the sequential nature of tape
• Requires a single path , same as OSTP Write & FCIP-FastWriter M 1

Al R1gh's Reserved
OSTP Read
Backup Server Location

Read 110 local
Host command MP-75008
Tape Device Location

Read 110
• After receiving the first read command, the remote router will
automatically respond locally with another read command after
each read 1/0 it receives from the tape controller
• The remote router pre-issues read commands for multiple data
blocks to maintain full utilization of bandwidth with links that have
significant latency
• Immediately after the host requests the next read 1/0, the local
router can respond with the data from its buffer
• This is possible because of the sequential nature of tape

Al R1gh's Reserved
Open Systems Tape Read

No OSTP
(Proxy Target} (Proxy Initiator}
FC FCIPWAN FC
READ CMD1
READ_CMD1
READ_CMD1
DATA_OUT 1 DATA_OUT 1 DATA OUT 1
DATA_OUT 1 DATA_OlJT1 DATA_OUT 1

RSP1 RSP 1 RSP 1
READ CMD2
READ_CMD2
READ_CMD2
DATA_OUT 2 OATA_OUT2 DATA OUT2
DATA_OUT2 DATA_OlJT2 DATA_OUT2

RSP2 RSP2 RSP2

Al R1gh's Reserved
OSTP Read
Host Initiator Local Router Remote Router Tape Target

FC FCIP WAN FC
READ CMD1
READ_CMD1
READ_CMD1
DATA_OUT 1 OATA_OUT 1 DATA OUT 1
DATA_OUT 1 DATA_OlJT1 OATA_OUT 1

RSP1 RSP 1 RSP1
READ CMD2 READ CMD2

DATA_OUT 2 OATA_OUT 2 DATA OUT 2
DATA_OUT2 DATA_OllT 2 OATA_OUT2

RSP2 RSP2 RSP 2

Al R1gh's Reserved
OSTP
What is Not Accelerated
The following sequences are not pipelined:

• Tape label read/write processing
• Tape open/close processing
• Tape load/unload processing
•Tape exception processing
- Terminates emulation processing1(gracefully) allowing the host and
tape controller to communicate/recover directly
•The pipelining process escapes if the tape needs to be

repositioned for any reason
- OSTP will reposition the tape to the correct location
C 2009 Brocade Commun ca1 ans Systems Inc i3

CFP380 Internal Use Only FCIP Administration
Revision 0110 5- 1
Objectives
- Configure and verify a VE_Port-to-VE_Port connection between the
Brocade 7800 and the Brocade FX8-24 Extension Blade
- Review CLI and DCFM configuration procedures

Recall that a VE_Port to VE_Port configuration merges both ends of the

connections, whereas a VEX_Port to VE_Port configuration isolates both ends of
the connection.
Revision 0110 5-2

Tunnel Configuration - Overview

• All steps must be performed on both ends of the link
• Follow these basic steps to create a VE-to-VE_ Port tunnel:
1. Determine the required parameters for the VE_Ports and FCIP tunnel
2. Configure hardware ports if necessary for appropriate media type and mode
3. Persistently disable the virtual FC E_Ports (VE) associated with the tunnel
4. Create an IP interface for each physical Ethernet port to be used
5. Configure an IP route for each port to specify an IP gateway (optional)
6. Verify the IP network between the two IP interfaces that will form the FCIP
tunnel
7. Create am FCIP tunnel (circuit 0 will automatically be created)
8. As needed, configure FCIP features (SACK, compression, etc.)
9. Verify configuration, enable the associated VE_Port, and validate functionality

Footnote 1: We will go through the steps to create a VE_Port-to-VE_Port tunnel

and then go through them again and add steps to create a VEX_Port-to-VE_Port
tunnel. We will focus on the different aspects of the configuration each time.
Revision 0110 5- 3
Determine the VE Port and FCIP Parameters

• Before creating a VE_Port and an FCIP tunnel, make sure that you
have identified the basic parameters on each physical port being
used by the tunnel:
- IP Interface: GbE port, IP address, Netmask, MTU size1
- IP Route (optional): Gateway IP address, Netmask
- FCIP tunnel basics: Tunnel ID (determines the virtual FC port), source
and destination IP addresses, committed rate (minimum and
maximum), and FCIP tunnel parameters

Remember: an FCIP tunnel requires two endpoints.

Footnote 1: As of Fabric OS v6.3.0, the only supported value for the MTU is 1500.
Revision 0110 5-4

CLI - portcfggernediatype
• Can be used to configure the media type (on 87800 only)
• Allowed on ports GEO .and GE1 only
• Two media types allowed - Copper (default) and Optical
• The media type is configured using the command:
portcfggemediatype ge<Oll> <copper l optical>
B7800 : admin> portcfggemediatype geO optical

B7800 : admin> switchshow lgrep ge
geO id lG Online FCIP
ge l cu lG No_Sync FC IP Copper
ge2 lG No Module FCIP
ge3 lG No Module FC IP

The default media type is copper. The copper ports are RJ45 ports. The optical
ports are SFP ports.
Revision 0110 5- 5
CFP380 Internal Use Only FC IP Administration

• 10 Gigabit Ethernet (10 GbE) is a new feature available with the
FX8-24 platform
• 3 Modes of operation for the Ethernet ports on FX8-24 :
- 1G Only Mode: 10 x 1 Gbps ports are available for use (GEO through GE9)
- 10G Only: 2 x 10 Gbps ports are available for use (XGEO and XGE1) 1
- Dual Mode: 10 x 1 Gbps ports and 1 x 10 Gbps port available for use (GEO
through GE9, XGEO) 1
XGbE
1G Mode
10G Mode
Dual Mode
GEO GE1 GU GE3 GE4 GE5 GE6 GE7 GE8 GE9 XGEO XGE1

Footnote 1: Requires the 1OGbE FCIP license.
Revision 01 10 5-6
1G, 1 OG and Dual Mode swi tchshow outputs.

204 8 12 01cc00 Off line UE
205 8 13 01cd00 Off line UE
206 8 14 01ee 00 Off line UE
207 8 15 01cf 00 Off line UE
208 8 16 01d000 Off line UE Valid VE- Ports
209 8 17 01d100 Off line UE
210 8 18 01d200 Off line UE
211 8 19 01d300 Off line UE
212 8 20 01d400 Off line UE
111;
?1 '.l II ?1 lil1 ;llC;:litlill ni-:&1.;..,, ..
8 22 01d600 Off line UE
8 23 01d700 Offline UE
8 26 01da00 Off line UE
8 27 01db00 Off line UE
8 28 01dc00 Off line UE
8 29 01dd00 Off line UE
8
0 .,.
30 01de00
filllof ..::l.£Qf.lll
Offline
n1:1:1.:--
UE
llC'
8 ge0 id 1G MoJ,ight FCI P

8 ge1 id 1G MoJ,ight FCIP
8 ge2 1G MoJlodule FCIP
8 ge3 1G MoJ1odule FCIP
8 ge4 1G MoJ1odule FCIP Valid ge Ports
8 ge7 1G MoJ1odule FCI P
8 gc8 1G MoJ1odulc PCIP
8 ge? 1G M~Jl~d~l~ FCIP
"'~""'-
~ ~: Disabled <1G Mode>
8 xge1 10G ti~J;~d~i; FCIP Dis abled <1G Mode)
204 8 12 01cc00 Off line
205 8 13 01cd00 Off line
206 8 14 01ce00 Offline
20? 8 15 01cf 00 Off line
208 8 16 01d000 Offline
209 8 1? 01d100 Off line
210 8 18 01d200 Off line
211 8 19 01d300 Off line
212 8 20 01d400 Off line
213 8 21 01d500 Off line
214 8 22 01d600 Off line
215 8 23 01d?00 Off line
216 8 24 01d800 Off line
21? 8 25 01d900 Off line
218 8 26 01da00 Offline
219 8 2? 01db00 Off line
220 8 28 01dc00 Off line
221 8 29 01dd00 Off line
222
........ 8
!: . ..
30 01de00
Q4 ~rt>D
Off line
_,..r,....:i~-
8 ge0 id 1G Ho:::Li911t FCIP Disabled <10G tlode>

8 9e1 id 1G NoJ,i9ht FCI P Disabled ( 10G tlode)
8 9e2 1G NoJ1odu le FCIP Disabled ( 10G tlode)
8 9e3 1G NoJlodule FCIP Disabled ( 10G tlode)
8 9e4 1G NoJ1odule FCIP Disabled <10G tlode>
8 9e5 1G NoJ1odul e FCIP Disabled <10G tlode>
8 ge6 1G NoJ1odule FCI P Di s abled <10G tlode>
8 ge? 1G NoJ1odule FCIP Dis abled <10G tlode>
8 9e! 1G ~o-!'!o'!u!e FCIP Disabled <10G tlode>
&~ t 'IU' __.--• ..., ......"'" ...,... :'> is ab led <10G tlode)
8 xge0 10G NoJ1odul e FCIP
8 xge1 10G No_l!odule FCI P
204 8 12 01cc00 Offline UE
205 8 13 01cd00 Off line UE
206 8 14 01ce00 Off line UE
20? 8 15 01cf00 Off line UE
208 8 16 01d000 Off line UE
209 8 1? 01d100 Off line UE
210 8 18 01d200 Off line UE
211 8 19 01d300 Off line UE
212 8 20 01d400 Off line UE
213 8 21 01d500 Off line UE
214 8 22 01d600 Off line UE
215 8 23 01d?00 Off line UE
216 8 24 01d800 Off line UE
21? 8 25 01d900 Off line UE
218 8 26 01da00 Off line UE
219 8 2? 01db00 Off line UE
220 8 28 01dc00 Off line UE
221 8 29 01dd00 Off line UE
222 8 30 01de00 Off line UE
223 ~ 31 01df 00 Off line UE
0 !jCU .LU .&:\J nui__,ul..lfJI'- -rw;
8 9e1 id 1G H<>J,i9ht FCIP
8 ge2 1G H<>J1odule FCIP
8 9e3 1G H<>J1odule FCIP
8 9e4 1G H<>J1odule FC I P
8 ge5 1G H<>Jlodule FCIP
8 9e6 1G H<>J1odule FCIP
8 9e? 1G N<>Jtodule FCIP
8 9e8 1G H<>Jtodule FCIP
8 9e9 1G H<>J1odule FCI P
8n x9e~ 10G
. n n
MoJ1odule FCIP
,..~ ......... ~~ _.~~~~!~ ~~ ! ~!°;.-lsabled <Dual ltode)
Revision 0110 5-7

CLI - bladecf ggemode

• Can be used to configure the GE mode on FXB-24
• Requires 1OGbE FCIP License (FTR_1OG)
• Switching modes is disruptive for FCIP traffic
• The GE mode can be set using the command :
DCX : admi n> bladecfggemode --set <mode> -slot slot number

mode: lg I lOg I dual
DCX : admi n> bladecfggemode --set dual -slot 8

Revision 0110 5-8

Slot-based Licenses for FX8-24
• FXB-24 blades can be configured with a license using:

licenseslotcfg --add <license> <slot#>
• Can be configured before/after installing the license
• Advanced Extension and 1OGbE FCIP licenses are both slot-
based
DCX : admin> licenseadd tXBfmAMWWmS3RmBaaHZGCTtA9MW7r399RSXQFrGAfZtB
adding license-key [tXBfmAMWWmS3RmBaaHZGCTtA9MW7r399RSXQFrGAfZtB]
DCX : admin> licen seslotcfg --add FTR_lOG 8
Blade slot-8 added to FTR_lOG slot-based license configuration
Remaining capacity for FTR_lOG slot-based license = 1
DCX : admin> licen sesh ow
tXBfmAMWWmS3RmBaaHZGCTtA9MW7r39 9RSXQFrGAfZtB :
10 Gigabit Ethernet ( FTR_lOG) license
Capacity 2
Consume d 1
Configured Blade Slots 8

Use licenseadd to install a license on a switch. The command

licenseslotcfg --remove is used when needing to reallocate a license to a
different slot.
10 GbE license: Enables the two 10 GbE ports and opt ional 10 GbE port
configurations
Advanced Extension license: Enables FCIP Trunking and Adaptive Rate Limiting
Revision 01 10 5-9
Persistently Disable the Virtual FC Ports

• Before beginning VE_Port configuration, persistently disable the virtual FC port
with the portcfgpersistentdisable command 1
DCX : admin> portcfgpersistentdisable 8/12
B7800 : admin> portcfgpersistentdisable 16
Server Storage
Brocade DCX-4S
with FXS-24 Blade
Brocade 7800
IP Router
Port 8/geO
(FC port 8/12)
VE_Port
. -. .. Port geO
(FC port 16)
VE_Port

Al 1R ghts Reserved
Footnote 1: While disabling the port is supported, it is recommended that the port
be persistently disabled during the tunnel configuration.
Create IP Interface
• Create an IP interface using the portcf g command with the ipif
operand:
• portcfg ipif [slot/] ge port create ip_addr netmask MTU
- Required args for ipif include:
• ipaddr- Unicast 1Pv4 address
• netmask - Contiguous 1Pv4 bitmask
• mt u _ s i z e - 1500 Bytes
- Maximum MTU size support is 1500

• Jumbo frames are not supported in Fabric OS v6.3.0 for the 87800 and
FX8-24 FCIP devices
•An MTU size greater than 1500 enables jumbo packet support

Create an IP interface on the tunnel

Each IP interface requires:
•A static IP address
• MTU size specification
•TCP port (3225)- Note: this port is automatically assigned (not configurable)
IP Subnet Rules
• Each circuit that is included in a tunnel must use a different subnet
for each GbE interface it spans
- There cannot be multiple IP addresses on the same subnet spread
across multiple GE ports
• An GbE port can host multiple circuits that participate in multiple
tunnels. These circuits do not need to be in the same subnet, but
can be
• Circuits that make up a tunnel on a 10 GbE interface can reside in
the same subnet

Create IP Interfaces Example

• On the Brocade DCX-4S, create an IP interface on port 8/geO with an MTU size of
1500:
DCX : admin> portcfg ipif 8/geO create 192. 1 68.1.24 255.255.255.0 1500
Operation Succeeded
• On the Brocade 7800, create an IP interface on port geO with an MT U size of
1500:
B7800 : admin> portcfg ipif geO create 192. 1 68.11.78 255.255.255.0 1500
Operation Succeeded
Server Storage
Brocade DCX-4S
with FXS-24 Blade
Brocade 7800
VE_Port
192.168.1.24
--- VE_Port
192.168.11 .78
C 2009 Brocade Commun ca1 ans Systems Inc

13
Create IP Interfaces Example (cont.)

Verify the IP interface settings with the port show ipi f [ s l ot I J port
command:
DCX : adrnin> portshow ipif 8/geO
Slot : 8 Port : geO
Interface IP Address NetMask MTU
0 192 . 168 . 1. 2 4 255 . 255 . 255 . 0 1500

B7800 : adrnin> portshow ipi f geO
Port : geO
Int erface IP Address NetMask MTU
0 192 . 168 . 11 . 78 255 . 255 . 255 . 0 1500
BrOCilldo OCX-4S
with FX8·24 Blade
VE_Pon VE_Pon
1112168124 • - - - 192.168 11 78

The port s h ow ipi f [slot / ] port command displays the interface ID, IP
address, netmask, and MTU slide for each IP interface.
The command po r tshow ipif a ll displays all interfaces.
Define an IP Route
• After defining the IP interface of the remote switch, define
destination routes on an interface by configuring an IP route
- Add IP routes when crossing subnets
• IP routes are not necessary if both endpoints reside in the same subnet
- A maximum of 32 routes can be added per GbE port1
- Use the portcf g iproute command:
portcfg iproute [slot/] [ge] create <destination ip_address>
<netmask> <local gateway> <metric> (weight)
- Specify the ip_address and netmas k of the destination tunnel
• Specify the IP address of the local gateway responsible for forwarding
frames to destination IP address (must be on same subnet as local device)
• Use the default metric of 0 (if configuring more than one route you can make
the second one less preferred by setting a higher metric)

Footnote 1: IP interface must be configured before adding a destination route on an

interface. A maximum of 32 routes can be added on one GbE port. Route additions
do not require tearing down tunnels. The IP address and gateway parameters use
Unicast 1Pv4 addresses in dotted decimal format (1.0.0.1 through 223.255.255.254);
netmask uses contiguous bitmask in 1Pv4 dotted decimal format. The associated
metric (weight) value range is 0 through 255.
The specified IP address needs to be an actual IP address at the other end of the
link, not a subnet address.
When creating multiple routes:
• Specify a metric of 0 to use a preferred gateway
• Specify a higher metric value to configure alternate, secondary gateways
• The higher the metric, the less preferred the route
Define an IP Route Example
• On the Brocade DCX-4S, add a route on port 8/geO to the remote IP interface
192 . 168 . 11 . 78 through local gateway 192 . 168 . 1 . 1 with a default metric of 0:
DCX : admin> portcfg iproute 8/geO create 192.168.11.0
255.255.255.0 192 . 168 . 1 . l 0
• On the Brocade 7800, add a route on port geO to the remote IP interface
192 . 168 . 1. 2 4 through local gateway 192 . 168 . 11 . 1 with a default metric of O:
B7800 : admin> portcfg iproute geO create 192.168.1.0
255.255.255.0 192 . 168.11.l 0
Server Gateway 192.168.1.1 Storage

Brocade DCX-45
with FX8-24 Blade
VE_Po11
192.168.1.24
--- VE_Port
192.168.11 .78
C 2009 Brocade Co'Timun ca1 ans Systems nc

Al 1R ghts Reserved
16
The portcfg iproute [slot) /port c r eate command configures an IP

route from an local IP address to a gateway IP address over an GbE port. The
command has the following required arguments:
• [slot] /port : The port on which the command is to operate.
• ipaddr: The IP address of the route.
• netmask: The IP netmask.
• gateway_ router: The IP address of the gateway router.
• metric: The gateway metric; if not specified the default metric of 0 will be used.
In the example above, IP routes are configured at each end of the link because the
two IP interfaces that were configured are in different subnets.
Define an IP Route Example (cont.)

Verify the IP route settings with the ports how iproute [slot/] port command:
DCX : admin> portshow iproute 8/geO
Slot : 8 Port : geO
IP Address Mask Gateway Metric Flags
192 . 168 . 11.0 255 . 255 . 255 . 0 192 . 168 . 1 . 1 0 Interface

B7800 : admin> portshow iproute geO
Port : geO
192 . 168 . 1.0 255 . 255 . 255 . 0 192 . 168 . 11.1 0 Interface
Gateway 192.168 1 1
VE_P011
192.168.1 2•
. --. VE_POl1
192.168.11.78

Validate the IP Network

• Verify the IP connectivity between the two IP interfaces with the
portcmd - -ping [slot J /port command
• A l ways specify a local GbE port, the source IP (- s), and destination IP (- d):
B7800 : admin> portcmd --p ing geO -s 192.168.11.78 -d 192.168.1.24
Ping ing 192 . 168 . 1 . 24 from ip interface 192 . 168 . 11 . 78 on 0/0 with 40 bytes of data
Reply from 192 . 168 . 1 . 24 : bytes-40 rtt-Oms
Reply from 192 . 168 . 1 . 24 : bytes•40 rtt•Oms
Ping Statistics for 192 . 168 . 1 . 24 :
Packets : Sent - 4 , Received - 4 , Loss - 0 ( 0 percent loss)
Min RTT • Oms, Max RTT • Oms Average • Oms
Server Storage
Brocade DCX-CS
with FXtl-24 Blado
Brocade 7800
VE_Poo VE_Poo
192. 168.1.24 •• - • 192.168.11.78

The portcmd --ping [s l ot] /port command validates end-to-end IP connectivity

over an GbE port. The command has the following required arguments:
• [s lot] /port : The port on which the command is to operate (here, geO on 7800 ).
• -s: The source IP address for an IP interface on a local GbE port.
-d: The destination IP address for an IP interface on a remote GbE port.
The command also has several optional parameters:
-n num_ requests: Specifies the number of ping requests. The default is 4.
- q service_ type: Specifies the type of service in the ping request. The default is 0
and service_type must be an integer from 0 to 255.
-t ttl: Specifies the time to live. The default is 100.
• - w wait_ time: Specifies the time to wait for the response of each ping request. The
default is 5000 milliseconds and the maximum wait time is 9000.
• -z: Specifies the default packet size to a fixed size in bytes. The default is 64 bytes. The
total size. including ICMP/IP headers (28 bytes without IP options) cannot be greater
than the IP MTU configured on the interface.
• If no optional parameters are specified, the command displays the currently configured
values for the specified port.
In the example above, a ping command is issued from the new IP interfaces on the
Brocade 7800, to the new IP interface on the Brocade DCX-4S. The command output
shows that the ping messages are received and returned by the Brocade DCX-4S, verifying
IP connectivity between the IP interfaces.
Validate the IP Network (cont.)

• Verify the IP router hops between the two IP interfaces with the following
command: portcmd - -traceroute [slot ] /port
• Allways specify local GbE port, the source IP (- s), and destination IP (- d):
B7800 : admin> portcmd --traceroute geO -s 192.168.11.78 -d 192.168.1.24
Traceroute to 192 . 168 . 1 . 24 from IP interface 192 . 168 . 11 . 78 on 0/0 , 64 hops
max
1 192 . 168 . 11 . 1 16 ms 0 ms 0 ms
2 192 . 168 . 1 . 24 16 ms 0 ms 0 ms
Traceroute complete .
DCX : admin> portcmd - -traceroute 8/geO -s 192.168.1.24 - d 192.168.11.78

Tr<:1cerouLe to 192 . 168 . 11 . 78 from IP i n terf<:1ce 192 . 168 . 1 . 24 011 8/0 , 64 hop~
max
1 192 . 168 . 1 . 1 16 ms 0 ms 0 ms
2 192 . 168 . 11 . 78 16 ms 0 ms 0 ms

portcmd --traceroute [slot/ ] geport -s src_ ip -d dst_ ip [-h ma x_ hops ]

[ -f £irst_ttl ] [ -q type_of_service ] [ -w t imeout ] [ -z size ]
Traces the IP router hops used to reach the host ds t_ ip from one of the source IP interfaces on the
GbE port. Valid arguments include:
-s src_ip: Specifies the local IP address to use for sourcing the probe packets.
-d dst_ip : Specifies the destination IP address to which to probe the IP router path .
-h max_ hops: Specifies the maximum hop limited used in the outgoing probe packets.
The default of probing a maximum of 30 IP router hops. This operand is optional.
-£ first_ ttl: Specifies the starting time to live value to first_ttl. The default is 1. --
traceroute skips processing for those intermediate gateways that are less than the first_ttl hops.
This operand is optional.
-q service_ type: Specifies the type of service in the ping request. The default is 0 and
service_ type must be an integer from 0 to 255. This operand is optional.
-w timeout: Sets the time, in seconds, to wait for a response to a probe. The default is 5 seconds.
-z size: Specifies the size, in bytes, of the trace route packet to use. The default is 64 bytes. The
total size, including ICMP/IP headers (28 bytes without IP options) cannot be greater than the IP
MTU configured on the interface. This operand is optional.
Create an FCIP Tunnel

• With the IP interface created and IP routes defined, optionally),
create an FC IP tunnel on the configured local IP interfaces
• Compression, Fastwrite, and Tape Pipelining settings must match
the opposite endpoint
• Use the command: po r t c fg f c ipt unnel [s l ot/ ] [ge]
create a rgs [opt ional args ]
Server Storage
Brocade DCX-45
with FXS-24 Blade
Brocade 7800
VE_Port
192.168.1.24
•
Tunnel 0 (Port 8112)

Circuit()
-- .. VE_Port
192.168.11.78
Tunnel 0 (Port 16)
Circui10

Optional tunnel_arguments for fciptunnel create and modify include:

-f 1--fastwri te o 11 J Enables (1) or disables (0) FastWrite on the specified FCIP tunnel.
-t !--tape-pipelining o 11 Enables (1) or disables (0) Tape Pipelining on the specified FCIP
tunnel. If Tape Pipelining is enabled, FastWrite should also be enabled.
-c I --compression compression_ level Configures compression on the specified FCIP
tunnel. By default, com pression is disabled (0). Specify one of the following values:
• O Compression disabled
• 1 Standard compression
• 2 Moderate compression (Brocade 7800 only)
• 3 Aggressive (Brocade 7800 only)
-T 1--tperf o 11 Enables (1) or disables (0) TPerf test mode.
- n 1--remote-wwn remote- wwn Specifies the WWN of the remote FC entity.
-d I --description string Specifies a description for the specified tunnel.
-F 1--ficon o 11 Enables (1) or disables (0) FICON emulation on the specified FCIP tunnel.
Create an FCIP Tunnel Example

• On the Brocade DCX-45: Port 8/geO, Remote IP = 192.168.11.78; local IP = 192.168.1.24,
min commit rate= 155000 Kbits/sec (OC-3), max commit rate= 1000000 (1 Gbps)
DCX : admin> portcfg fciptunnel 8/geO create 192.168.11 . 78 192 . 168.1.24
-b 155000 -B 1000000
• On the Brocade 7800: Port geO, Remote IP= 192.168.1.24; local IP= 192.168.11.78,
min commit rate= 155000 Kbits/sec (OC-3), max commit rate= 1000000 (1Gbps)
B7800 : admi n > portcfg fciptunnel geO create 192.168.1.24 192 . 168.11.78
-b 155000 - B 1000000
Server Storage
Brocade DCX-4S
with FXS-24 B lade
Brocade 7800
VE_ Port
VE_Port
192.168.1.24 • - - - 192.168.11.78
Tunnel O (!Port 8/12) Tunnel O (Port 16)
CircuitO Circuit O

Optionally, a value can be set for a minimum and a maximum committed rate to configure the tunnel
for Adaptive Rate Limiting (ARL), which allows for a more effective sharing of bandwidth between
applications. The valid range is 1544. Kbps - 1000000 Kbps. Both sides of the tunnel must have
matching configurations.
-b I --min-comm-rate minimum
Modifies the minimum committed traffic rate on the FCIP circuit O in Kbps.
-B I --max-comm-rate maximum
Modifies the maximum committed traffic rate on the FCIP circuit 0 in Kbps.
Add Circuits
• Once a tunnel is created, additional circuits can be configured to
provide redundancy and bandwidth
portcfg fcipc i rcui t [s l ot/]ve_ port option circu it ID option s
[arguments ] [optional_arguments ]

portcfg fcipcircuit <ve- port> <createJmodifyJdelete> <circuitid> [<parameters>)

create parameters :
<remoteip> <localip> <comrnited.Rate> [<optional args>J
or
<remote!p> <localip> --min- comm- rate <kbps> --max- comm- rate <kbps> [<optargs>J
optional circuit args :

-a , --admin-status <Oil> - enable/disable the circuit
- s , --sack - turn sack off
- k , --keepalive- timeout <ms> - set the keepalive timeout in ms
-x, --metric <metric> - set the circuit metric
-b, --min-comm-rate <kbps> - set min comm rate value in kbps
-8, --max- comm- rate <kbps> - set max comm rate value in kbps
- m, --min- retrans- time <ms> - set min retrasmit time in ms
-r, --max-retransmits <rtx> - set maximum number of retransmits
-v, --vlan-tagging <vlan-id> - set the vlan-id for the circuit
--12cos- f - class <12cos> - set the L2CoS value for F- Class Traffic
--12cos- h igh <12cos> - set the L2CoS value for High Priority
--12cos-medium <12cos> - set the L2CoS value for Medium Priority
--12cos-low <12cos> - set the L2CoS value for Low Priority
--dscp- f - class <dscp> - set the DSCP value for F- Class Traffic
--dscp- high <dscp> - set the DSCP value for High Priority
--dscp- medium <dscp> - set the DSCP value for Medium Priority
--dscp- low <dscp> - set the DSCP value for Low Priority
Creating IP Interfaces for Additional Circuit

• On the DCX-4S, create an IP interface on port 8/ge1 with an MTU size of 1500:
DCX : admin> portcfg ipif 8/gel create 192.168.2.24 255.255.255.0 1500
Operation Succeeded
• On the 7800, create an IP interface on port geO with an MTU size of 1500:
87800 : adrnin> portcfg ipif l gel lcreate (192 .168 .12=::±!) 255. 255. 255. 0 1500
Operation Succeeded
• On the DCX-4S, add a route on port 8/ge1 to the remote IP interface
192 . 168 . 12 . 78 through !local gateway 192 . 168 . 2 . 1 with a default metric of O:
DCX : admin> portcfg iproute 8/gel create 192 .168.12 . 0 255.255.255.0
192.168.2.l 0
• On the 7800, add a route on port ge1 to the remote IP interface 192 . 168 . 2 . 2 4
through local gateway 192 . 168 . 12 . 1 with a default metric of o:
B7800 : adrni n> portcfg iproute gel create 192.168 .2.0 255.255.255.0
192.168.12.1 0

Adding a Circuit Example

• Notice:
Using different GbE port
Using same VE_Port (tunnel)
Using different subnet
• On the Brocade DCX-4S: Port 8/12, Circuit ID =1 , Remote IP = 192.168.12.78; local IP =
192.168.2.2, min commit rate= 155000 Kbit/sec (OC-3), max commit rate= 1000000, using a
metric of 1 to use the circuit as a standby circuit
DCX : admin> portcfg fcipcircuit 8/12 create 1 192.168.12.78 192.168.2.24
-b 155000 -B 1000000 -x 1
• On the Brocade 7800: Port 16, Circuit ID =1 , Remote IP= 192.168.2.2; local IP=
192.168.12.78, min commit rate= 155000 Kbit/sec (OC-3), max commit rate= 1000000 ,
using a metric of 1 to use the circuit as a standby circuit
B7800 : admin> portcfg fcipcircuit 16 create 1 192 . 168.2.24 192 . 168.12.78
-b 155000 -B 1000000 -x 1

Ve rify FCI P Tunnel Settings

• Verify the FCIP tunnel settings with the port show fciptunnel [slot/] port
command:
DCX : admin> portshow fciptunnel 8/12
Tunnel ID : 8/12
Tunnel Description :
Admin Status : Enabled
Oper Status : Up
Compression : Off
Fastwrite : Off
Tape Acceleration : Off
TPerf Option : Off
IPSec : Disabled
Remote WWN : 10 : 00 : 00 : 05 : 1e : SS : a1 : 80
Local WWN : 10 : 00 : 00 : 05 : le : 92 : db : OO
Peer WWN : 10 : 00 : 00 : 05 : le : SS : a1 : 80
Circuit Count : 2
Flags : OxOOOOOOOO
FICON : Off

Verify FCIP Circuits

• Display FCIP circuit configuration and status using the port show fcipcircui t
command
DCX : admin> portshow fcipcircuit 8/12
Tunnel Circuit OpStatus Flags Uptime TxMBps RxMBps ConnCnt CommRt Met
8/12 0 8/geO Up ----s lh10m32s 0 . 00 0 . 00 2 155/1000 0

8/12 1 8/gel Up ----s lh13m41s 0 . 00 0 . 00 2 155/1000 1
Flags : circuit : s=sack

ConnCnt =Connection count. Increments the times the circuit has been initialized.
Verify, Enable, and Validate the VE_Port

• Before enabling the VE_Port, ensure that fabric merge-related parameters
(domain ID, zoning , etc.) are set so that the fabrics will merge
successfu Ily1
• When the fabric merge-related parameters are set correctly, enable the
VE Port
- Review the swi tchshow command output to determine whether
portenable or portcfgpersistentenable should be used 2
B7800 : admin> switchshow
. .. <truncated output> . ..
Area Port Media Speed Sta te
15 15 No Module
16 16 Offline Disabled (Persistent)
17 17 No Module
<Truncated Output>
B78 00 : admin> portcfgpersistentenable 16

Footnote 1: VE Ports are virtual E Ports established over a FCIP tunnel. Some of
the parameters that cause VE_Ports to segment include domain overlap, zoning,
incompatible fabric parameters. Note that these are the same parameters that will
cause E_Ports to segment (see fabstatss h ow help information).
DCX : admin> fabstatsshow
Description Count
Domain ID forcibly changed : O

E Port offline transitions : 0
Reconfigurations : 0
Segmentations due to :
Loopback : 0
Incompatibility : 0
Overlap : 0
Zoning : 0
E_Port Segment : 0
Licensing : 0
Disabled E Port : 0
Platform DB : 0
Sec Incompatibility : O
Sec Violation : 0
ECP Error : 0
Duplicate WWN :
Eport IsolatedAD header conflict : 0
Footnote 2: In the swi tchshow output above, VE_Port 16 is persistently disabled.
Revision 01 10 5 - 27
Verify, Enable, and Validate the VE_Port (cont.)

• Validate VE_ Port functionality with the same commands used to
validate E_Ports: swi tchshow , fabricshow , and
topologyshow 1
B7800 : admin> swi tchsho w

<Truncated Outpu t >
Area Port Media Speed State
15 15 No_Modu le
16 16 Online VE - Port 10 : 00 : 00 : 05 : le : 36 : 04 : 06 " DCX- 4S " ( downstream)
17 17 No Modu le
<Truncated Outpu t >
geO id lG Onli ne

Footnote 1: The fabricshow output associated with establishing VE_Port

connections is no different than a fabrics h ow output established over E_Ports.
DCX : admin> topol ogyshow

2 doma i n (s) i n the fabr i c ; Local Domain I D: 1
~:~:i: ; isool
Name : Rl - STOl - 8780 0
Path Cou nt : 1
Hops : 1
Out Por t : 8/1 2
I n.Eorts : 119
[ Total Bandwi dth : 0 . 2 5 6 Gbp s (adj usted)
Bandwi dth Demand : 390 %
F lags ; D
Rl - ST 01 - DCX- 4S : FID128 : admi n> fabricshow

Swi tch I D Wor ldwi de Name Enet IP Addr FC IP Addr Name
1 : fffcO l 1 0 : 00 : 00 : 0 5 : 1e : 92 : db : OO 10 . 2 55 . 24 8 . 1 5 0 .0 .0 .0 " DCX"

3 : fffc03 1 0 : 00 : 00 : 05 : 1e : 55 : a1 : 80 10 . 255 . 248 . 19 0.0 .0.0 " 8 7 800"
The Fabric has 2 switc h e s
The metric is derived from two paths at 100 Mbps, subtracted from 2000 1800. =
Total Bandwidth is derived from .128 Gbps per 100 Mbps. 200 x .128 0.256 Gbps. =
Revision 01 10 5 - 28
BROCADE
EDUCATION SOLUTIONS
FCIP Management Using DCFM

• The current release of DCFM supports:
- FCIP tunnel configuration
- Tunnel property views
- FCIP data collection
- Topology tunnel links
- Connection properties, etc. for legacy FCIP platforms
• The new platforms bring in some conceptual changes including
circuits, multiple TCP connections per tunnel etc.
• The DCFM FCIP features have been enhanced to support legacy
as well as the new platforms

FCIP Tunnels Dialog

• The Configure> FCIP Tunnel menu launches the FCIP Tunnel view
• One point interface for all management operations for tunnels and
circuits
• Shows all tunnels discovered in DCFM across all fabrics with facility to
filter on fabrics/switches
-
Upper table
displays
..··="'~
I ,_
,... .........
. .._ ·--
_,~
~-.-.... ~
!l
- Fabric,
·-·
IB ~ll!-"91. . . .
..~..-.es
...4191. . . .
,
....
.....oo.- '
~
Switch,
Tunnel
J
--·-
...,,_..,,(an ..,"'
~-
-- "' Lower table
·--...........
W2tU • t t
displays
-- 'llJlll)llH
details of
-
_,...c--,._,....~
i::~•1 )••1fOfXX11
selection
in upper table
-- ... -
C 2009 Brocade Commun cat ens Syst ems Inc 31
Add FCIP Tunnel Dialog

• Open using Add Tunnel button
• User has option to configure one or both sides of the tunnel. When the tunnel is
created , DCFM will create the tunnel by configuring both devices
;~ Add f[JP Tunnel EJ
Conflgl.we the settng. for the 1...-.el lo be eclcled on the - l ! d swta:h end a second swtc:h tt selecll!d
Al least one ci'c:uil (I' W'le.1ace) shoUd be set"" ror ac1c1ng al\nlel,
Switch One Settings Sw Mch Two Seltlngs Seled Swtc:h Two
SwCc:h R1 .STOl.OCX-4S Swtc:h

FelJric FCI' Fal>rlc
Port Type • VE Port Pct1Type
VEXPort

Al 1R ghts Reserved
Select Switch Dialog

Launched from Add FCIP Tunnel dialog when the Select Switch Two button is
clicked (optional)
;~ Select Sw1tcill Ef
Select lho second awtcn f« lho tlrfl<I from the:e - awlchcs.

Alle\/Ols
13 O ~ Fa> FCF
Q R1 .STOl -87800 R1 -STOl -87 ... Swtcn 0011
8 Rl-STOl-OCX-CS Rl-STOl-OC ... Swlcn 0011
3 Chassis Group

Add FCIP Tunnel Dialog (2 Switches)

Two switches are selected fo r proposed tunnel.
;<; Add FCIP Tunnel £!

Config<re the sett'1g$ for the tlnlel to be added on lhe -od swl<:h and a sec<>nd swtch H selected.
N. leost one clrCUll (P lnl..-foce) shoUll be ut '-"tor odcll>g a tlnlel.
Swil•h One Settings ( wMch Two Settings
Swtch R1 -STOl-OCX--4S ~ch Rl-STOl-87800

Febrlc FCI' Fabric FCI'
__J De=~ion
Por1 Type • VEPor1 Por1 Type le VE Por1
(., VEXPor1 (' VEXPor1
OK

Must Define a Circuit

• If the user clicks OK in the Add FCIP Tunnel dialog without first cr·eating a Circuit,
a dialog is launched, as one circuit is req uired to configure the Tunnel
In otdef to ~e the tl.llnel conf9,r.Clon, P lnlef1oces on bolh ends of lhe tl.llnel need to
be defhd by ~ on the Add Cl-cult bttlon.

Add FCIP Circuit Dialog

• The Add FCIP Circuit
dialog is the main a....-.
~CGITICdll:rll:ldW'eenhtwoMY'd&'Wkhulareq.rod!a~M1\lm.,.,.~~tnd~P~
·-........-
$wtl.c hOf\oSettino9' SWil:thTWO~
configuration page
·-........
R1.$T01.0CX-4S 111.$1('1~
"""" •a> ra>
for an FCIP circuit ..... ....

• This dialog is explained . "" _
.. ... . ""
--"""-- --
p Addlest Type PAOca-eul~
PAlbe:u t1tl.lll81 I 119211181 2
in the next several

- -
Diet.,. rcMew9Ql!lcreud"*19h ~Padctea Delll&.C roi.ce wl gill crOllled u9"9 lhe abcW8 P ed:hl•
Otllt Ncn-OtlU: RcUt
slides wru Stl'9 c1 • •ZJ.M) NTU Stz9 (1290. 23481
--)
'II.ANO
(Bit* er 1 ·4'&',f"OSVer - 8.0.D)
• ~(15'&.1000Mbll)
-..
1!.~
...-...
~
--)
'II.ANO ~., Swtc:hOrw
SMll: • Swid'I One
o.n... Maxll!lulll BlndwtCln

Add FCIP Circuit Dialog (cont.)

• The circuit number is already populated
• Select the g e ports to be used on both switches
• Enter the IP address and subnet mask
• If IP routers exist between the two switches, enter a gateway
address used to create an FCIP IP Route
....... .._. Circuit ID#

Select which GbE ._,.........,.,.
........
port to use
·-..,...., ·-
-
fCP
IP Route
IPAddress fl'Mcte:n~ • Pv4
Subnet i----t- • f'AMeu 11121fllt 12
•Nb--
...__ uo11111
...__
""'**.....
-
Defd~ew9 . . ~U1W'9h~Padll-HI Oe1'U etlllltd ll!l'IO . .mow p ad!SMS
.J o.-.~-..-.c. o ... ~,..,.._..
°""'"'

Al R1gh's Reserved

• Enter a MTU size for both switches
- 1500 is the maximum supported MTU value for the 87800 and FX8-24
as of Fabric OS v6.3.0
• Enter a VLAN ID
- VLAN IDs are not supported for the 87800 and FX8-24 as of Fabric OS
v6.3.0
• Enter a committed rate for the circuit1
- Minimum= 1.544 Mbps, Maximum= 1 Gbps
MTUSt1•(12'0·2)4f)
(l!liri:Or t . 4094.fOSVw -100)

MTU (1500
Set bandwidth of
•
--
COll'!f!ICed(1 ,St&..1COO...,.)
'lit.c
in this
release),
VLANID
--<->
15'44
committed rates
""'""""""""-

Al R1gh's Reserved
Footnote 1: Entering the same value in both fields is effectively setting a committed
rate w ithout using Adaptive Rate Limiting (ARL).

• To test the IP settings for the circuit, select the Verify IP
Connectivity button
Configures IP Veri1y IP Comec:ttvly

CD Advanced Settngs
and pings, then
rolls back
settings
® t:mml·@§MbijiiiiiiiiQ!.!, -
t:.....
.......
..
.,_...
0 Conf9">g P lnterteco on the loc:<ll swlch H does not exist

_ .._
,.,,._,71..
----
p ..........
Configl.rflll f> to.ie on ll'le 1oc:<11 swtch ~ ooes no1 emt. ,._
Conf9.lrtng P ni..tace on the remcle swlch if does not e>dst
~
....
Conf9JM!1 P Ro.le on the remote swlch if does not exist.
ExOQ.Cflg p Ang
1t2 1•21•
112Ull271
,.,,._,71 ....

Al 1R ghts Reserved
This is a temporary test.

1. Select the Verify IP Connectivity button.
2. DCFM will configure the IP interface of both devices, execute a ping test and
then remove tine IP configuration from both devices.
3. The results of the test are displayed.
Tunnel and Circuit Advanced Settings

• You can configure advanced settings for:
- Circuit - Metric, SACK, Keep Alive Time Out, Retransmission Time
- Tunnel - Compression, Fast Write , Security, FICON Emulation
-(Mbls)
• ~ed(1.544-1000Mbls)
Mmun MalCilun
!!SL I ~ =mJ
Circuit - from c..rert-.....-
Add Circuit ;:::rsmmmiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii~
page ~•NM!CM"91t0tlht111V'1111atlt . . . . anlNMllclecl&'Wlcf'lr.d• aeconcla....wt•utlcMd
MlltUll o--. '*°o.tl<f'.-w1eet)st'CUib9 Mt 't>lor eck;tlnlJe .........
_ -
5""".d'IOMS4mlr!Q9 S..mtwoSett.-. Sillld$wtch fVll'O
s-.....
,,,
fh-STOt.OOC-'$
,_ "1.$101-
"'"'
""""'_ '"'
-- _
Yenty P ComecWty
....... ....... '"""
....._.
... . .,........
,,.. ...,,.. . .....
"""""
Tunnel
M4o...A
---Y
O 2009 Brocade Commun cal1ons Sys•ems Inc
OK c....t
40
-
Al R1gh's Reserved
Tunnel I Circuit Configuration Report

• Clicking OK in the Add/Edit Tunnel dialog will launch the report displaying status, tasks and
messages
. ; f(IP TuOMUCircu1I ConhlJJrntloo fWport ~
,_. g..,. l tnor Mnui;e

~P....,tkeonhtixolllll""IWtd'lifdo-~ ---------
~Mc:llhtthPrurfaeedllllil ~
~.... ••GUI• on IN • • S'fll.ICh" dcln n SUccMOIG
~IDCllS¥r"lchP 10,u_... S4.cc.eotdlicl
ConftiJ.IWIOWUllP«tonlDCll~ 5uc'cffdtcl
~..,.,,..C°""lju'...,cnloe;lla'W:ld'I~
P.,11W11JtlC4ilP."'W't..,.....lnl~~ ~"°""
~..,...~ontic.i ..... -un ~
~htulPQrl $4.lc(:""'91;1
~P...."<tanNt..-C•ll1111-1ct'lcfd t'IPt09'tu
.....,..,......""'unPn• UC•..,...
C°""9'"1Pft1Macnh10ct11. . . . 111 dCet
fW..-.g IMCll:e rNtm fl' IM• dlllll
Canf'9.l"'wWpartanrMd•awtcft
eonni.,rTigUndc.ontigll11ionanremctesv.1ctt
Per¥111r'9t.....Xes-.vut\ ........ intor!MCIOndttall
~""'*-....onon 1 ...olt!s~
~ ... l ...:C.part

Al R1g~·s Reserved
Rollback
• If any errors are encountered when configuring the tunnel I circuits, the option is given to
rollback the configuration
...
~·""f«t.,h!KltP'tCfl«•
,__.,,loc.lls:Y1oldlP..-,_.....,
~·IOU•cn. . lrK#f""Nf<fttl ..... 1'1 ~
FwMSn11Ktf,..olitlC:ftP101M.... Si.i«..-.
~..,..,., .., • • •we(ft ~
~1'1Qll.n'llll~an_.1W'ldl.~
........ ~&'oNC:cnt.mlllnf~,... ~
~UWll~M~l..Cd'I $'&.CC..-,
~ . . . k1Uf'f(lr1 Sitt.....
,.....,.. JWtd\.............
~~·~tAt•onhr..Us-<dCtl lll G
~•"°'-"'on1NllX.al1""-Ulli il ooes
1-.,toeclltP"*'l.u
Mlo1tll
~
~1COIT*1toh.-.-ldl .-r-cr~Gtlj
,_~,..-.11.,._,.P•Ol..t•_. ~
~.,,..to1Cll"t r..ar.P'tdl MliontcJ
~.,,.~~ .... ,-..~.AllonM
""Mll"'il'-*lwtd't.lU'ftlll~--~
~..,.,......,..Ol'l,_. .....(11 ...,..
~ . . , . . . . port "'**'
0o'llQ,IWW"C-tof<llOIK*-.,M,_~~""""~'-.lld10tcll:llP.,...,_,.,
ICOJDi,oomedtohc'WllCtl 9mlt' ~otifltt.,..,, . .

Al 1R ghts Reserved
Rollback Status
• Displayed after clicking the Rollback button
•.; roP Tunlll'l/Circun Confl&0raUon Rl!port [g)
COf'l(9s1"19 Prcttt.e.onh:ioeSS'A'CCftdOo SUCCMded

P.~toc•S'#teh•n•tac.•Clellll ~
Conf91nDP,o.uonewlOicfilP'CUIWO:..n SUcceeded
P9~lclcailsWkhProi.tedlla ~
Con11gt.trig.,.po11ontoct1s:wtd'I ~
Contq.rngi ....,,.. CO"ll9T.., on loell swtd'I SUccMded

,.~loail&WlthlU'lnelnforlMllOndetllfl ~
Con~.....,.. .......... onbt.e~"fllo-Ci:h ~
&'llt*'IG .,..1o<el l)Ol1 SUcc.tdtd
~IPh•taceon . .1.ncleswtchrld '-.OloeddPr1ert«• c~com.c:tto1'11~'1ch WTcrlltequlcedCll
l ~1..X•'f'NCCfllPl't•-ftie•OtUtt. Ab<wtt4
> Conf91ng PRM9on lhe IKtl ~Ifdoff Aorttd
PefM11"19lell!OfeS>t1otO'IP•Medie&lll ~
' ~f'lgwtullPQrtonr.,..eswtch Abcn:H ~
'
--
F.....,
===-~~~~~~~~~~~~'...
.;
I+
.,;
=--
P#Mb'19t1C11t1•'td'l1U"ne1nrorwm;oncti11...
51-.n lfn"oMnug.
Sutc.dld
====.:i=s-..,..Ch =~CllletePl'Wleu«U:t'llcomtctlON~Ch
Ptlt11!Ar"9toc.al!S~PrMt.... sueutdld
-
errcrt
eil' Conf9.tt'OIPr011UonhlOc.alsw1cnffdOesn 5'.cceeded

~ ID9i"~*"'""'tct'!Pn•'faud9Clft SUcCMdild
.,; Conf9.l.-.OPf'C•'l1it•on1'!il'toet1a""cn1100 ~
C 2009 Brocade Commun ca• ons Sys•ems l'lC 43

Al R1g~·s Reserved
BROCADE
EDUCATION SOLUTIONS
Tperf
• Tperf (tunnel performance) is a tunnel test tool that was added in
Fabric OS v6.3.0 to provide an alternative to IPPERF
- Tperf is only supported on the 7800 and FX8-24 platform s
- IPPERF is only supported on the 7500 and FR4-18i platforms
• The intent is to provide a network bandwidth reliability test tool that
utilizes test data between a pair of switches to determine the
network characteristics between the switches
• With the advent of FCIP Trunking, a new tool is required with the
ability to generate and report on the sending of test data over an
FCIP Tunnel
- IPPERF was GE source and destination IP Address specific, therefore
it was not suited to the newly implemented FCIP Tunnels on the 7800
and FX8-24 platforms

Tperf (cont.)
• There are two main modes of operation for FCIP Tunnels:
- Primary mode of operation is as a VE port
- Tperf tunnel
• An FCIP tunnel cannot be in both modes at the same time
- When the FCIP tunnel is modified or created to be a tperf tunnel, there
is no associated online VE port with that FCIP tunnel
• The intent was to 100°/o separate a test tunnel from an on line
VE_ Port path
• A tperf tunnel will consume bandwidth, so care must be taken
when creating a tperf tunnel on the same device that hosts a
production tunnel

Tperf (cont.)
• TPerf option requires two separate FCIP devices to function
- One device plays the role of a sink (destination) and the other device plays the
role of the source
• User must specify that the tunnel is a TPerf tunnel by:
- Creating a new tunnel
- Or modifying an existing tunnel using the TPerf flag - T <O 11>
• The -sink command must be run before the - source . Once the
- source command runs, traffic will be generated.
• Commands must be run on both devices. This example is
modifying an existing tunnel:
B7800 : admi n > portcfg fciptunnel 16 modify -T 1
DCX-4S : admi n> portcfg fciptunnel 8/12 modify - T 1
DCX-4S : admin> portcmd --tperf 8/12 -si nk
B7800 : admin> portcmd --tperf 16 -source - high - low - random

portcmd -- t perf - Determines the path characteristics to a remote host or

tunnel destination. The --tperf option requires two separate FCIP end devices to
function. One device plays the role of a data sink and the other device plays the role
of the data source. Tperf also requires that you define a tunnel as a tperf tunnel.
- sink I - source Designates the switch to function either as a data sink or a
data source. This operand is required . When -sink is specified, tperf begins to
respond to traffic sent by the switch acting as the data source. The process
continues to run until it is either terminated by user intervention (Ctrl +C) or, if a
duration is specified with the -t option, until the process completes the set time
frame.
The following optional arguments are ignored on the data sink, because it services
all requests from the data source: --high J --medium, --low, -
u nidirectional , - random , - pattern , and - size.
The following arguments are optional:

-high Generates high priority traffic.
-medium Generates medium priority traffic.
-low Generates low priority traffic. If no traffic priority is specified, high, medium, and
low priority traffic is generated.
-time duration Specifies the duration of the TPerf traffic flow in seconds. If a duration is not
specified, the process continues to run until it is terminated with Ctrl + C.
-unidirectional Generates traffic in one direction only. The default is round-trip.
-loop Re-issues a send request as fast as possible after completion of the previous
send request.
-random Specifies a random protocol data unit (POU) size between 1 and the size of
the send request. Refer to -size below.
-ere Specifies cyclic redundancy check (CRC) to be performed on the payload.
-pattern pattern Specifies the test data pattern for the payload as one of the
following values:
0 No pattern is specified. TPerf applies whatever is
already set or in memory. This is the default
value.
1 All zeros
2 All ones
3 lnicrementing byte
4 Random
5 Jitter
-size pdu_size Specifies the POU size to use (not including headers). The valid range is
between 1k and 16k. The default is equivalent to the maximum segment size
(MSS). This is the maximum size if the -random option is specified.
-interval interval
Specifies the interval at which the statistics display is refreshed , in seconds. The
default is 30 seconds.
Tperf Example Output

B7800 : admin> portcmd --tperf 16 -source -high -low -random
TPerf has been configured successfully for 16
TPerf is generating traffic on 16 priority: high
TPerf is generating traffic on 16 priority: low
Tunnel ID: 16
High Priority Medium Priority Low Priority
bytes tx 241064 0 312292
bytes rx 80 0 40
PDUs tx 52 0 51
PDUs rx 2 0 1
bad CRC headers rx 0 0 0
bad CRC payloads rx 0 0 0
out of seq PDUs rx 0 0 0
flow control count 0
last rtt 0 2
Tunnel ID: 16
High Priority Medium Priority Low rriority
bytes tx 241064 0 312292
bytes rx 80 0 40
PDUs tx 52 0 51
PDUs rx 0 1
flow control count 0 1
last rtt 0 2

When - source is specified, tperf generates traffic until it is interrupted by user

intervention (Ctrl + C) or, if a duration is specified with the -t option, until the process
completes the set time frame. The tperf module on the remote host will immediately
begin generating traffic; it is therefore imperative that the data sink has been
started on the opposing switch before the data source is started on the local
switch.
Tperf generates statistics every 30 seconds by default unless you specify a different
value for -interval. The output displays the following information:
Tunnel ID Numeric identifier for the tperf tunnel.
Traffic Priority High, Medium, or Low.
bytes tx Number of bytes transmitted.
bytes rx Number of bytes received.
PDUs tx Number of protocol data units transmitted.
PDUs rx Number of protocol data units received.
bad CRC headers rx Number of bad CRC headers received.
bad CRC payloads rx Number of bad CRC payloads received.
out of seq PDUs rx Number of out-of-sequence PDUs received.
flow control count Flow control count.
last rtt Last Round trip in milliseconds (RT traffic only).
7800 :admin> portcmd --tper! 16 -source -high -low random

TPerf has been con figured successfully for 16 • Example output from tperf
TPerf is generating traffic on 16 priority: high
TPerf is generating tra!fic on 16 priority: low • Output updated every 30 seconds
• Issue CTRL+C to exit
Tunnel ID : 16
High Priority Medium Priority LOw Priority
bytes tx 263860632 0 104565600
bytes rx 1031640 0 410000
PDUs tx 25991 0 10300
PDUs rx 25941 0 10250
bad CRC headers rx 0 0
bad CRC payloads rx 0 0
out of seq PDUs rx 0 0
last rt t 58 0 146
Tunnel ID : 16
High Priority Medium Priority LOw Priority
bytes tx 521914320 0 201160 680
bytes rx 2054400 0 816600
PDUs tx 51410 0 20465
PDUs rx 51360 0 20415
out of seq PDUs r x 0 0 0
flow c~ntrol count 0 0 0
last rt t 58 0 146
Tunnel ID : 16
bytes t x 179957856 0 310976064
bytes rx 3011120 0 1223280
PDUs tx 76828 0 30632
PDUs rx 76118 0 30582
out oC seq PDUs rx 0 0 0
flow control count 0 0 0
last rt t 58 0 146
Tunnel ID : 16
bytes t x 1038072456 0 414150840
bytes rx 4088120 0 1629800
POUs tx 102253 0 40795
PDUs rx 102203 0 40745
last rt t 58 0 146
Tunnel ID : 16
H.igh Priority Medium Priority Low Priority
bytes tx 1296ll5992 0 517345920
bytes rx 5104840 0 2036400
PDUs tx 127671 0 50960
PDUs rx 127621 0 50910
lase rt t 58 0 146
Tunnel ID : 16
bytes tx 1554169680 0 620541000
bytes rx 6121600 0 2443000
PDUs tx 153090 0 61125
POUS rx 153040 0 61075
bad CRC headers r.x 0 0 0
out of seq POUs rx 0 0 0
last rt t 58 0 146
Tunnel ID : 16
bytes tx 1812182760 0 723725 928
bytes n 7138200 0 284956-0
PDUs tx 178505 0 71289
POUs rx 178455 0 71239
last rt t 58 0 146
Tunnel IO : 16
bytes tx 2070216144 0 826921008
bytes rx 8154880 0 3256160
POUs tx 203922 0 81454
PDUs rx 203872 0 81404
bad CRC headers rx 0 0
out of seq POUs rx 0 0
flow control count O 0 0
last rtt 58 0 146
Tunnel ID : 16
High Priority Medium Priority Low P["iority
bytes tx 2328269832 0 930095784
bytes rx 9171640 0 3662680
PDUs tx 229341 0 91611
POUS rx 229291 0 91567
last rt t 58 0 146
TUnnel ID : 16
bytes tx 2586313368 0 1033290864
bytes rx 10188360 0 4069280
POUs tx 254759 0 101782
PDUs rx 254709 0 101732
out of seq POUs rx 0 0 0
last rt t 58 0 146
BROCADE
EDUCATION SOLUTIONS
FCIP Configuration Parameters

• Virtual ports associated with the Brocade 7800 and the Brocade
FX8-24 have configuration (conf igshow) parameters
- Blade configurations are stored on a slot basis; if blades are swapped,
the configuration stays; the new blades take the old , corresponding slot
configuration
• Similar to prior Fabric OS releases, a Fabric OS configupload
command will not cause 1/0 disruption
- A configuration upload does not have any dependency on Brocade
FX8-24 or Brocade 7800 state
- All FCIP tunnel related configuration is saved 1

Footnote 1: If the Brocade FX8-24 or a Brocade 7800 is disabled, a configupload

would still succeed in pushing the FCIP configuration to the config file.
Revision 01 10 5 - 53
FCIP Configuration Parameters (cont.)

• A conf igdownload that includes FCIP settings requires the switch to be
disabled first
- I/Os in FCIP tunnels are disrupted
• To apply additive downloaded FCIP configuration, you will additionally
need to:
- Reboot the Brocade 7800
- Use s l otpoweroff /slotpoweron for the Brocade FX8-24 or
reboot the Brocade DCX-4S
• Delete all IP interface and FCIP tunnel configurations by:
- Disabling the GbE port
- Invoking portcfgdefau l t
- Re-enabling the port
54
Footnote 1: Invoke the following commands to delete the tunnel created between
the Brocade 7800 port 17 Brocade DCX-4S 10/17:
From Brocade 7800: portcfgdefaul t geO; portcfgdefaul t 1 7
From Brocade DCX-4S: portcfgdefault 10/geO. Slot 10 port 17 acts as a
Virtual E_Port, it does not have and VEX_Port parameters to delete.
Note that because geO was defaulted, the FCIP parameters associated with the
connection between the Brocade 7800 port 16 and Brocade DCX-48 10/16 created
earlier would also be deleted. If the portcfgdefaul t command were invoked on
the Brocade 7800 port 16 the VEX_Port parameters would also be deleted.
Revision 01 10 5 - 54
High Availability
• When Active CP fails over, there is no impact to distance traffic
• Using slotpoweroff for the FX8-24 blade would gracefully
shutdown both FC and extension traffic
• An upgrade of Fabric OS would result in res.et of the FCIP
subsystem
- Layer 2 FC traffic is non-disruptive
- Distance (FCIP) traffic would become temporarily unavailable

Summary
• Configuring and verifying a VE_Port-to-VE_Port connection
between a Brocade 7800 and a Brocade FX8-24 extension
blade can be done from the CLI or by us.ing DCFM
• Tunnel performance can be tested using tperf

Al R1gh's Reserved
BROCADE
EDUCATION SOLUTIONS
CFP380 Internal Use Only Adaptive Networking Traffic Management
Revision 0110 6- 1
Objectives
- Define Brocade FC Adaptive Networking
- Differentiate the features of Brocade FC Adaptive Networking related to
traffic management
- Implement Quality of Service (QoS) Source ID/Destination ID
(SID/DID) traffic prioritization in the fabric
- Describe QoS traffic prioritization for Brocade HBAs
- Implement Ingress Rate Limiting (IRL) in the fabric
- Describe Target Rate Limiting (TRL) for Brocade HBAs
- Implement Traffic Isolation (Tl) zoning in the fabric

Revision 0110 6-2

BROCADE
EDUCATION SOLUTIONS
Revision 0110 6-3

Adaptive Networking
• Adaptive Networking is a framework concept encompassing:
- Traffic management
• Quality of Service (QoS)
• Allocates priority if congestion occurs
•Assigning data flows to a dedicated physical link (Tl zones)
• Setting bandwidth limits for a data flow (Ingress/Target Rate Limiting)
- Fabric profiling : reports information that can be utilized to best
implement QoS, Tl , IRL and TRL
•Top Talkers
• Uses internal monitoring to measure bandwidth and queue utilization
• Bottleneck detection
• Identifies devices attached to the fabric that are causing the slowing down of
traffic

While a Fabric OS license does exist with the same name, the concept of adaptive
networking goes beyond the single license.
Adaptive Networking is a suite of tools and capabilities that enable you to ensure
optimized behavior in the SAN. Even under the worst congestion conditions, the
Adaptive Networking features can maximize the fabric behavior and provide
necessary bandwidth for high-priority, mission-critical applications and connections.
The Adaptive Networking framework includes the following features:

• Top Talkers
• Traffic Isolation Routing
• QoS Ingress Rate Limiting
• HBA QoS Target Rate Limiting
• QoS SID/DID Traffic Prioritization
• QoS HBA Traffic Prioritization
• Bottleneck detection
The Adaptive Networking license only activates these features:

• QoS Ingress Rate Limiting
• QoS SID/DID Traffic Prioritization
Revision 0110 6-4

Licensing
• Adaptive Networking does have a license, but is not required for

all the features, where some features require additional licenses
Services Service Name Fabric OS License Required
Traffic Fabric QoS Adaptive Networking
Management QoS on an HBA/ Server Application Optimization
Covered in Target Rate Limiting and Adaptive Networking
Module 6- Ingress Rate Adaptive Networking
this module Limiting
Traffic Isolation None
Fabric Top Talkers Advanced Performance Monito r
Profiling - Bottleneck None
Module 7 detection
Revision 0110 6-5

BROCADE
EDUCATION SOLUTIONS
Revision 0110 6-6

QoS - SID/DID Based Traffic Prioritization

• SID/DID traffic prioritization is a licensed feature (Adaptive
Networking)
• Categorize the traffic flow between initiators and targets
• Focus is on latency, not bandwidth

- Traffic that requires lower latency is placed in front of the queue
• For example, you could assign online transaction processing (OLTP) as high
priority and backup traffic as low priority

Al R1gh's Reserved
Revision 0110 6-7

QoS - SID/DID Based Traffic Prioritization (cont.)

• This feature is based on an extension of original Brocade virtual
channel model:
- Divides an ISL into 8 virtual channels (VCs) to ensure that traffic of
various priorities can travel across the link without disruption
- Data flows use 4 virtual channels (2 - 5) and share the same priority
• All 8 Gbps platforms/blades support 16 VCs., which are divided up
based on traffic type and priority (low, medium, high)1
- SID/DID Based Traffic Prioritization enables the setting of priorities
between specific hosts and targets
• Allows for the control of frame flow in the fabric when contention occurs
- FSPF route selection is not affected by QoS priorities

Footnote 1 : VC bandwidth is assigned to priority types. When all priorities are being
used , High priority gets approximately 60% of total bandwidth, Medium priority gets
approximately 30%, and Low priority gets approximately 10%. If not all bandwidth is
utilized in a particular priority level, unused bandwidth can be used by other
priorities.
QoS is not supported on mirror ports or 10 Gbps ISLs.

The QoS for ISLs is a per-port configurable attribute applicable to E-Ports in both
Condor and Condor2-based switches running Fabric OS v6.0. A QoS enabled E-
Port will form a QoS capable ISL with a neighboring switch only if the connecting E-
Port on the neighboring switch is also QoS capable, with QoS enabled. Otherwise,
the fabric module will negotiate down to non-QoS mode (medium priority, VCs 2 - 5
only). The side that negotiated to a lower flow control value will log the information
in the RASLOG.
Revision 0110 6-8

Brocade's original virtual channel model (pre-Condor2/GoldenEye2) divides an ISL

into 8 virtual channels to insure that traffic of multiple priorities can travel across the
link at the same time, without being disrupted, or disrupting other traffic.
. /!~1111f:J1 - ilftit
I l ......___._ ......- . -...........
•
.-. -
0 Class F
1 Reserved
~
2 Data
Data
All
3
~ Data
4 Data
Traffic
5
6
Data
Class 3 Multicast
--
Revision 0110 6-9

8 Gbps - 16 Virtual Channel Model

\!l!l'!J .. . ··ifl!!l
0 Class F
1 Reserved
.l;i .......,.....,.:.1 .
..
----c;'~;:
.''.'__
·;-
•·.
~~·Q08 ·
-~-
~-
-._,
~- ~>J. -~ --~
ct< · -;-;
. ;,:-;:- -;;,;:-
":
-·
1'1 ::· ...
6 Class 3 Mlulticast
8 Low Priority QoS
9 Low Priority QoS
10 High Priority QoS

15 Reserved

The example above illustrates the breakdown of the 8 Gbps ISL into individual VCs
and their priority assignments. Both Condor2 and Goldeneye2 AS ICs support this
model.

• The QoS features are engaged when there is contention on the link
- Contention occurs if:
• Multiple frames arrive at the link at the same time, or
• Bandwidth is congest,ed on the link
- If there is no contention on the link QoS will not engage
• The order of operations during congestion is to send frames in the
order listed below1 :
1. VCO Class F frames then,
2. High priority traffic then,
3. Medium priority traffic then,
4. Low priority traffic

Footnote 1: These operations will only occur if there are enough frames in each
queue to process. Results may vary depending on the status of the fabric profile at
that moment, known as a time slice.
Contention vs Congestion
• Contention is when multiple frames arrive at the link at the same
time Host
4 Gbps Point of Contention
8Gbps
4 Gbps
• Congestion is when there is more traffic than the link is capable of

carrying Point of Congestion
4Gbps
Host
Tape
8 Gbps

Illustration of QoS Data Flow

• Frames of each type of priority are contending for their turn to be
sent, higher priority frames are moved to the front of the queue
• This is a snapshot of the Tx queue with the frames classified into
their respective priorities
VCO DDDl}TxQueue {
High D ODDDDDI
Medium DDDDDDI ====
;;::::::=:===
low l l• l• i• J• l• I I,___ _ __.
RxQueue

Illustration of QoS Data Flow (cont.)

• Any frames in VCO are sent, followed by six high priority
frames, three medium priority frames, and 1 low priority frame,
respectively
~::
M•<lla'Tl==:::;;;
o;;;;o;;;;o;;;;:::i .__ _.,
D l}Tx Queue { DOD~~~:
ODDI
;::::::::::::::;:::::::::::;:::::::::;::::::::
Low i• l• ,• l• I I •I
Rx Queue


• While bandwidth is not part of the calculation for QoS traffic flow,
these generally stated percentages of bandwidth can be used to
make prioritization decisions1
- High priority gets approximately 60% of the total bandwidth available
for data traffic
- Medium priority gets assigned approximately 30%
- Low priority gets remaining bandwidth available less what F-Class VCO
traffic takes, approximately 10%

Footnote 1: These numbers are generic and assume full payload frame size with a
complete variety of queued traffic. If payloads are not full, then the percentages
could vary greatly. If there are not enough frames of a particular priority in the
queue, the frames that are present will be sent and process will skip the remaining
allotment and move to the next priority, thus varying the percentage ..
For, example if there are only 4 frames of high priority traffic, but VCO has 4 frames
and medium and low priority queues each have 20 frames, then the proportion of
high priority bandwidth to the other traffic will be much less than 60% at
approximately 33% (4 high/(4 VC0+4 high+3 med+1 low))
The percentages will vary depending on the fabric profile at the moment of QoS
processing.
8 Gbps - 16 Virtual Channel Model Bandwidth 1

~
-
t!:T"-:r-.ll•lllrj,t•{•] l!.1• • • Ill':~
..:.rr.-:t:111u...·• .:u11
0 Class F
1 Reserved
I .?i .:. :~~'-

r ·tl i. ---~~~
!
~
L -~ ..~. . . . .-~,QO'B\
r=·r•I= )~;~-
6 Class 3 Multicast
7 Broadcast/Mult.icast
8 Low Priority QoS
9 Low Priority QoS -10%1
10 Hlgh Priority QoS

11 H lgh Priority QoS
-60%
12 H lgh Priority QoS
15 Reserved

Footnote 1: Bandwidth that is actually observed in the fabric during contention for
the line is coincidental and not a set, nor directly configurable, rate. The number of
frames and their payloads of the fabric profile determine the actual bandwidth. QoS
does not use any of these metrics to determine frame priority.
QoS Zones
• Prioritization is accomplished by the use of QoS zones
- These will appear as rnormal zones
- All regular zoning rules apply
- Can be created using WWN notation or D,I (domain,index) notation
• D,I notation requires Fabric OS v6. 3.0 or later
• To distinguish QoS zones from normal zones, special prefixes are
used in the zone names:
- QOSH_ to set high priority
- QOSL_ to set low priority
- Prefixes are not case sensitive
• Default setting is medium priority, and is used when no QoS zones
a re specified or when QoS is not enforced

QoS Zones (cont.)

• Prioritization can be managed to use a specific VC for a given zone
• When creating the zone using the QOSH or QOSL prefix add a flow
ID value
- QOSHid_
- QOSLid_
• id' - is an optional flow identifier to designate a specific virtual channel
• If no id is supplied VCs are assigned using a round-robin method
• Refer to the next slide: 8 Gbps - 16 Virtual Channel Model Flow Ids for id to
VC pairings
• Example:
- QOSH2 zone1
• When QoS is enforced, traffic for this zone will use VC 11

Footnote 1: Requires Fabric OS v6.3.0 or later
If a QoS zone name prefix is specified in an LSAN zone (a zone beginning with
prefix "LSAN_"), the QoS tag is ignored. Only the first prefix in a zone name is
recognized. For example, a zone with the name "LSAN_QOSH_zone1" is
recognized as an LSAN zone and not a QoS zone.
8 Gbps - 16 Virtual Channel Model Flow IDs

\BJ .. . .. .... ~ ~1•••~·.•111•11
0 Class F
1 Reserved
....-~:.QQI:
-~
~.. .
:llidliiit.~. i
·- - ~
·•·
~~
J! ··~.~-· ._, !';>'
•;
6 Class 3 Multicast
8 Low Priority QoS 1
9 Low Priority QoS 2 - 10% 1
10 High Priority QoS 1

-600.4
15 Reserved

The high and low priority sections have flow ids for each VC that can be used during
the creation of the QoS zone to designate specific VCs for use. Medium VCs do not
have a flow id.
QoS Zones (cont.)

• In the illustration below, the QoS zone could look like:
- QOSH2_zone1 Members: HostB, TargetB
• Traffic from HostB to Targets would have high priority on VC11,
whereas all other patterns of traffic would be at the default
(medium) priority Target A
Host A
Domain 1
Target B
~
~~F-O-S-v6-.3-~---1~
Oomain2
QoSH2_zone1

QoS Zones (cont.)

• To assign a host or tar,g et to the high or low priority, it must be
attached to a 8 Gbps capable switch
- Condor/GoldenEye-based 4 Gbps switches are supported in the data
path and QoS is preserved
• If host and target are on different switches connected by ISLs, VC

priority is carried over each hop1 if:
- All intermediate switches are 8 Gbps capable
- 4 Gbps switches in the path are running Fabric OS v6.0 or later
• QoS is enabled by default, but can be disabled and enabled on

individual E_ Ports that carry prioritized traffic

Footnote 1: QoS queue handling is executed at each link along the path that is
experiencing contention. If there is no contention at that link, all traffic is treated as
medium priority. This does not affect latency, as bandwidth is available to handle
requests at line rate.
Revision 01 10 6 - 21
CLI Commands for QoS

• To enable QoS on a given port:
portcfgqos --enabl e [slot/]<port>
• To disable QoS on a given port:
portcfgqos --disable [slot/ ] <port>
• To set the configuration to default 1 :
portcfgqos --default [slot/ ] <port>
R8 - st02 - dcx- l : adrnin> portcfgshow 10 /0
Area Number : 80
Speed Level : Auto
<<< Truncated Output >>>
Persistent Disable OFF
NPIV cap abilit y ON
QOS E Port
Rate Limit 0 . 2G

QoS is enabled by default.

Footnote 1: QoS is a -ts: effort" facility, meaning it will work if there are enough
buffers to support it. In the default setting, QoS will work if there are enough buffers
available to sustaiin the requested links. If tlhere are not enough buffers to sustain
the requested links, Buffer Limited Mode will be enforced, until such time that
buffers are not available. When this happens, VCs will start to collapse within their
priorities to conserve buffers.
If the number of Buffer Usage is less than the number of Needed Buffers, the port is
operating in the buffer limited mode.
Footnote 2: By default, the value is AE, Auto Enable. =ON'or =OFFwill only appear
if the feature has been explicitly enabled or disabled, respectively.
QoS - Traffic Prioritization Considerations

• If a host and target are included in two or more QoS zones with
different priorities, the zone with the lowest priority takes
precedence
- For example1, if an effective zone configuration has QOSH_z1 (H,T)
and QOSL_z2 (H,T), the traffic flow between Hand Twill be of Low
QoS priority
• If QoS is enabled, additional buffer credits are allocated per port for
8 Gbps ports in LE mode
- Condor2 ASIC allocates 8 additional credits
- GoldenEye2 ASIC allocated 2 additional credits

Footnote 1: Additionally, if QOSH_z1 (H,T) overlaps with a regular -dmain,port"

zone at the H port, the traffic flow between H and T is dropped to medium priority
and the H port is marked as a session-based zoning port.
QoS - Traffic Prioritization Considerations (cont.)

• Traffic prioritization is enforced on the egress ports only, not on the
ingress ports
• QoS zones using D,I notation should not be used for loop or NPIV
ports
• QoS zones using D, I notation are not supported for QoS over
FC - FC routing
• Traffic prioritization is not supported in the following configurations:
- 10 Gbps ISLs
- Mirrored ports
- CryptoTarget container (redirection zone)
- McDATA Fabric Mode (interopmode 2) or McDATA Open Fabric Mode
(interopmode 3)

Firmware Upgrade/Downgrade with QoS
If upgrading from Fabric OS v6.x, QoS is not disrupted.

If upgrading from Fabric OS v5.x, QoS will not become effective until the E_ Ports
are disabled/enabled following the firmware upgrade.
If QoS enabled E_Ports are active when a firmware downgrade is activated, the
firmware downgrade will not be allowed until QoS is disabled.
After a firmware downgrade to a version of Fabric OS prior to v6.0, any existing
QoS configurations will no longer be effective.
• Any existing zoning configurations will be intact, but any existing QoS zones
will be treated as regular zones.
After a firmware downgrade, the zoning configuration would be intact, however, any
zones that had a QOSX_ in the name would now be treated as regular zones. Since
earlier versions of Fabric OS do not understand the concept of QoS zoning, these
zones would be treated as normal zones.
Revision 01 10 6 - 25
BROCADE
EDUCATION SOLUTIONS
QoS Traffic Prioritization on Brocade HBAs

• Can be enabled on Brocade 8 Gbps capable HBAs only
- Not supported on Brocade 4 Gbps HBAs or Brocade CNAs
• Works with the QoS feature on Brocade switch F_ports
• Requires Server Application Optimization(SAO) license
• Managed only through the Brocade Configuration Utility (BCU)
- Host based CLI
- Not configured through the switch
• BCU Commands:
bcu qos --enable <port_ id>
bcu qos - - disable <port_ id>
bcu qos --query <port_ id>
bcu qos --stats <port_ id>
bcu qos --s t atscl r <port_ id>

Command operands:
--enab l e Enables Quality of Service (QoS)
port_ id Specifies the ID of the port on which QoS is enabled
- - disabl e Disables Quality of Service (QoS)
port_ id Specifies the ID of the port on which QoS is disabled
--q u ery Queries the QoS details
port_ i d Specifies the ID of the port for which you want to display information.
- - s t at s Displays the QoS statistics
port_ id Specifies the ID of the port for which you want to display statistical
information.
--s t a t sclr Clears the QoS statistics
port_ id Specifies the ID of the port for which you want to clear statistical
information
Revision 01 10 6 - 27
BROCADE
EDUCATION SOLUTIONS
Ingress Rate Limiting

• A licensed feature available only on 8 Gbps platforms/blades1 •2
- Allows the Condor2/Goldeneye2 ASIC to delay the return of R_ RDY or
VC _RDY primitives3 to the external device by throttling back the
ingress port speed , thereby limiting the throughput on the ingress side
of the port
- Ingress Rate Limiting is only supported for F/FL_Ports
• It is not supported for E/EX_Ports
• Ingress Rate Limiting is designed to help alleviate choke points in
the fabric caused by slow drain devices, congested ISLs, etc. 4
• Example use cases:
- To reduce existing congestion in the network or proactively avoid
congestion
- To enable more important devices to use the network bandwidth during
specific services, such as network backup
Footnote 1: The Brocade 48000 with 8 Gbps blades will support Ingress Rate
Limiting on all installed 8 Gbps ports, but QoS is limited to pass-through support in
the same way all other 4 Gbps switches operate. In other words, all 4 Gbps and 8
Gbps FC ports support QoS through the switch, but do not support QoS to/from
devices attached to the Brocade 48000.
Footnote 2: The portdisable/portenable commands are required after the

addition of the license for the features to be effective.
Footnote 3: This is accomplished by throttling back the flow of R_ RDYs.
Footnote 4: Slow drain and congestion issues will be covered in more detail in the
bottleneck detection section of this module.
Ingress Rate Limiting (cont. )
SPEED
LIMIT
200
••
Mbps
e Ta pe
~
••
• • •••
• ••
Hosts
Disk
portcfgqos -- setratelimit [slot/)<port> rate (Mbps)

The settings for Ingress Rate Limiting are unidirectional. In the example above,
traffic returning from a target to a host would travel at full line speed, unless the
ingress side of the target's port is also throttled back. In which case, traffic would be
rate limited in both directions.
The switch ports in the example above are assumed to be capable of 8 Gbps.
Note: If Virtual Fabrics is enabled, the rate limit configuration on a port is on a per-
logical switch basis. That is, if a port is configured to have a certain rate limit value,
and the port is then moved to a different logical switch, it would have no rate limit
applied to it in the new logical switch. If that same port is moved back to the original
logical switch, it would have the original rate limit take effect again.
Revision 01 10 6 - 30
Ingress Rate Limiting Commands

• Use the port cf gqos CLI command to implement Ingress Rate
Limiting:
- To enable an Ingress Rate Limit on a given port: 1
portcfgqos --setra telimi t [ sl ot/ ] <port> rate (in Mbps)
- To disable an Ingress Rate Limit from a given port:

portcfgqos --resetrate l imi t [s l ot/]<port>

Footnote 1: Rate Limit is set in Mbits/sec. For example, to set a rate limit on slot 3,
port 4 to 2 Gbps, the command syntax would be:
portcfgqos --setratelirni t 3/4 2000
SetRateLimi t allows the following rates in Mbits/sec:
• 200
• 400
• 600
• 800
• 1000
• 1500
• 2000
• 2500
• 3000
• 3500
• 4000
• 5000
• 6000
• 7000
• 8000
Configure Ingress Rate Limiting with Web Tools

'-# R l l -5T02-851 - Port Admm1strnt1on
FCl'orb I
R!J · ST02· B51 · Web Tools Is0 1H;11p11g,gn.,
FC Ports E>cplot« : l l - Odells Ii ~ C<lntiglnljon
.-"'.........__........__
. . m·•·•"~. .._..--•mj--- x
Swli 4. Report. 4. RRpon
"' Sw
AIE
Oes>edOosUnce(lm)
Tine
Fri Dee 19 2008 00:3'1:06 GMT
~ue Feb 03 2009 20:13:11 GM"
T elnet!SSH Client
Tue Feb 03 2009 20:13:22 GM"

-·
Al R1gh's Reserved
From Web Tools:

1. Select Port Admin
2. Select Edit Configuration with the desired port selected
3. Specify the Port Parameters then click next
4. Select the Ingress Rate Liimit(Mb/s)
Ingress Rate Limiting (cont.)

• The following command shows a port with Ingress Rate Limiting :
RB-st02-dcx : admin> portcfgqos --setratelimit 10/0 200
RB-st02-dcx : admin> portcfgshow 10/0
Area Number : 80
Speed Level : Auto
AL PA Offset 13 : OFF
Trunk Port OFF
Long Distance O FF
VC Link Ini t OFF
Locked L Port OFF
Locked G Port OFF
Disabled E Port O FF
ISL R_RDY Mode OFF
RSCN Suppressed OFF
Persistent Disable OFF
NPIV capability ON
QOS E_Port ON
I Rate Limit 0 . 2G 200 Mbps]
Mirror Port OFF
RB - st02 - dcx : admin>

Running portcfgsh ow command without specifying a port will show ON for any
ports that have Rate Limiting enabled. The configured speed will not be displayed .
BROCADE
EDUCATION SOLUTIONS
HBA Target Rate Limiting

• Can be enabled on Brocade 8 Gbps capable HBAs
- Not supported on CNAs
- Licensed feature
• Managed only through the Brocade Configuration Utility (BCU)
- Host based CLI
- Not configured through the switch
• Used to minimize congestion at the HBA port due to a slow
drain device operating in the fabric at a slower speed
• Traffic destined to the remote port is limited to its current
operating speed
• The default rate limit is 1 Gbps

Command operands:
--enable Enables Quality of Service (QoS).
port_ id Specifies the ID of the port on which QoS is enabled.
--disabl e Disables Quality of Service (QoS).
port_ id Specifies the ID of the port on which QoS is disabled.
--query Queries the QoS details.
port_ id Specifies the ID of the port for which you want to display information.
--stats Displays the QoS statistics.
port_ id Specifies the ID of the port for which you want to display statistical
information.
--statsclr Clears the QoS statistics .
port_ id Specifies the ID of the port for which you want to clear statistical
information.
HBA Target Rate Limiting (cont. )

• Target rate limiting (TRL) is supported only when the HBA
port is connected to the fabric
- T RL is not supported w hen the port is directly connected with another
device
• BCU Commands:
bcu ratelim --enable <port_ id>
bcu ratelim --disable <port_ id>
bcu ratelim --query <port - id>
bcu ratelim --defspeed <port_id> [<11214>]

Command operands:
--enable Enables target rate limiting, if currently disabled
port_ id Specifies the ID of the port you want to enable
--disabl e Disables target rate limiting on the HBA, if currently enabled
port_ id Specifies the ID of the port you want to disable
--query Queries the target rate limiting details
port_ id Specifies the ID of the port for which you want to display
information.
--def speed Sets the default target rate limiting speed. The default TRL
speed must be supported and less than the maximum speed
at which the card can operate
port_id Specifies the ID of the port on which you want to set the
speed
speed 1 1214 Sets the default target rate limiting speed on the HBA. Options
are 1 Gbps, 2 Gbps, and 4 Gbps.
HBA Target Rate Limiting (cont.)
8Gbps Storage
2Gbps Tape

Al R1g~·s Reserved
For all discovered remote ports, the HBA management tool and corresponding
driver will use RPSC to find their port speed capabilities and then use this
information to throttle the transmitted traffic rate to that remote port. This will
provide protection only for FCP write traffic.
At each port level, target rate limiting can be turned on/off. When on , there are 2
scenarios:
• Target supports RPSC (Report Port Speed Capabilities) ELS
• Target does not support RPSC ELS. In this case, the HBA management tool
will assume a default target speed of 1G. A hidden configuration parameter
(not exposed to user) to change this default speed setting will be available for
troubleshooting purposes.
Revision 01 10 6 - 37
BROCADE
EDUCATION SOLUTIONS
Traffic Isolation (Tl)

• The Traffic Isolation feature allows you to control the flow of inter-
switch traffic by creating a dedicated path for traffic flowing from a
specific set of source ports (N_ Ports)
• Traffic Isolation is supported on Condor, Condor2, GoldenEye and
GoldenEye2-based 1 switches running Fabric OS 6.0 or later
• Does not require a license
• Examples for isolating traffic are:
- Dedicate an ISL for high priority host to target traffic
- Force high volume, low priority traffic onto a given ISL to limit the effect
of that traffic on the overall fabric
- Separate FICON traffic from Opern Systems traffic2
- A storage system needs to have its data restored from a tape device,
and the two systems should have a dedicated connection

This feature can aid in the control of frame flow through a fabric. Please see the
Fabric OS admin guide for a more detailed discussion of the Traffic Isolation feature.
Footnote 1: With Fabric OS v6.3.0, Traffic Isolation Routing is supported only on

Brocade200E, 300, 4100, 4900,5000, 5100, 5300, 5410, 5424,5450,5480, 7500,
7500E, 7600 switches, the Brocade 48000 and Brocade DCX platforms, all
configured in Brocade Native Mode (interopmode 0).
Footnote 2: Traffic Isolation Routing has limited support for FICON FCIP in
McDATA Fabric Mode (interopmode 2), in the following configuration only:
• Brocade 7500 with E- Port connections to an M-switch and VE- Port
connections to another Brocade 7500
• Devices attached to M-switch only
Following is a sample configuration:
Devices - M-switch - Brocade 7500 - Brocade 7500 - M-switch - Devices
Revision 01 10 6 - 39
Traffic Isolation Zones

• Traffic Isolation zones:
- Can create a dedicated route
- Do not modify the routing table1
- Are implemented across the entire data path from a single location
• Traffic Isolation zones use a special zone command that indicates
the set of N_Ports and E_Ports to be used for a specific traffic flow
• They are intended to control the routing of frames through the
fabric between zone members, not to control access to devices

Footnote 1: Routes are not being changed, but one or more are being dedicated for
use by a specific set of devices. The urouteshow command will show available· Tl-
zoned routes.
Prior to Fabric OS v6.0, traffic isolation in a SAN environment was somewhat

difficult to implement
The process involved setting up static routes through the SAN with the
urouteconf ig command, or the linkcost command.
• These would modify the routing table, and had to be done on each switch in
the data path
• The linkcost command is unidirectional, and would involve settings for both
directions along the route
• If the need to isolate this traffic was temporary, the routing table would have to
be modified again when traffic isolation was no longer necessary- on each
switch in the data path
Tl Zoning is a routing related feature, but, unlike using urouteconf ig or
l inkcost , Tl Zoning is bi-directional.
Revision 01 10 6 - 40
Traffic Flow without Tl Zones

• W ithout Tl zones, traffic is free to use any ISL, subject to the ru les of
FSPF, and DPS
Host A Target A
HostB
---- + 2>
3
h.
~
Host e · · ~TargetC
4

Fabric Shortest Path First (FSPF) is the protocol by which routes are selected in a
fabric . Dynamic Path Selection (DPS) is also called =Ex;hange-Based Routing'.
Revision 01 10 6 - 41
Traffic Flow with Tl Zones

• With Tl zones, traffic is rerouted so that only Tl zoned traffic uses the
isolated route, and all non-Tl zoned traffic must use the remaining ISLs
Tl Zone
..
• ~ TargetC

The ISL in the Tl zone depicted above will be exclusively reserved to Tl zoned traffic
as long as there is another equivalent cost route available.
Tl Zones Analyzed
• They are called "zones", but they are really about FSPF routing
• A standard zoning configuration must be in effect for this feature to
work
• Tl zones will only appear in the defined zoning configuration , not in
the effective zoning configuration
• Tl zones can be used with McDATA Fabric mode1
- Cannot be used with McDATA Open Fabric mode
• A maximum of 255 Tl zones can be created in a single fabric 2
• Ports in a Tl zone must belong to switches that run Fabric OS
v6.0+
- For Tl zones over FC-FC routing , ports must belong to switches that
run Fabric OS v6.1 or later

Possible uses for Tl zones might be:

• Dedicate an ISL for high priority host to target traffic
• Force high volume, low priority traffic onto a given ISL to limit the effect of that
traffic on the overall fabric
• Whatever the reason , a Tl zone can be created that contains the set of
N_Ports and E_ Ports to be used for a specific traffic flow.
Footnote 1: However, the Tl zones will not be enforced on the M-Series switch.
Traffic Isolation Routing has limited support for FICON FCIP in McDATA Fabric
Mode (interopmode 2), in the following configuration only:
• Brocade 7500 with E- Port connections to an M-switch and VE- Port
connections to another Brocade 7500.
• Devices attached to M-switch only.
Footnote 2: Fabric OS v6.1 has the limitation of 255 Tl zones. Fabric OS 6.0 has a
limit of 239 Tl zones. A fabric merge resulting in more than the supported number of
Tl zones will cause a merge failure, and segmentation.
Tl zones cannot be used in heterogeneous fabrics (Fabric OS and M-EOS) if
interopmode 3 is being used. However, regular zoning can be used, and can be
activated from any M-series switch in the fabric.
Revision 01 10 6 - 43
Tl Zones Analyzed (cont.)

• Tl zones can only be created using D,I (Domain, Index) notation
• In routed fabrics, Tl zones must be created in each of the edge
fabrics with devices requiring communication, as well as in the
backbone fabric, to create a dedicated end-to-end route
• Tl zones must include E Ports and F and FL Ports in order to
create a complete, dedicated, end-to-end route from initiator to
target
- Both VE_Ports and VEX_Ports are supported in Tl zones
• If multiple E_Ports are configured that are on the lowest cost route
to a domain, the various source ports are load-balanced across
those E Ports
• If a Tl zoned E_Port is a member of a trunk, all ports in that trunk
must be included in the Tl zone11, as Tl zones are enforced on the
trunk master only

Footnote 1: If all ports in a trunk group are not included in a Tl zone, the behavior
of the Tl zone will be uncertain, as Tl zones are enforced on the trunk master only.
For example, if 3 out of 4 ISLs in a trunk group are included in a Tl zone, and the
trunk master is part of the Tl zone, behavior is normal. However, if the trunk master
fails , and the new trunk master is the ISL which is not included in the Tl zone,
behavior will be dependent on the failover setting. If failover is disabled, the Tl zone,
and thereby the dedicated route between host and target, will be broken, and no
data will flow. More information on failover on the next few slides.
Tl Zones Analyzed (cont.)

• Traffic isolation to a Tl zone route is supported only if other, non-
dedicated, ISLs exist, and only if the Tl zoned route is an
equivalent, "lowest cost path" to the destination1
• Each Tl zone is interpreted by each switch , and each switch only
considers the routing required for its local ports
- There is no end-to-end checking to insure the integrity of the route
• Ports can only be members of a single Tl zone. Two N_Ports
having the same shared area must be configured in the same Tl
zone2 •
- For example: 02XXOO and 02XX80 both need to be in the Tl zone
- This limitation does not apply to E_Ports that use the same shared
area

Footnote 1: Traffic Isolation is a -l:S effort" facility that will work as long as it
doesn't violate the FSPF -Q\.vest cost route" rules. This means that traffic from one
Tl zone may have to share an ISL with other Tl zones and devices if no equal-cost
routes are available and failover is enabled.
If a Tl zoned E_Port fails, traffic will failover to a non-Tl zoned E_Port, if no other
equal-cost Tl zoned E_Ports exist (this behavior is dependent on the -ailover"
setting, which is covered on another slide). Also, a non-Tl zoned device will use a Tl
zoned E_Port if no equal cost .alternatives exist and failover is enabled.
If used within an AD, the E_Ports specified in a Tl zone must be in that AD's device
list (enforced during zone creation/modification).
• Since Tl zones must use D,I notation, the AD's device list must be declared
using D,I for ports that are to be used in such zones (enforced during zone
creation/modification).
• Care must be taken if using Tl zones for shared ports (E_Port or N_Port)
because of the limitation that a given port can only appear in a single Tl zone.
Conflicting members across ADs can be detected by use of zone --validate,
and -ts practices" would demand that such situations not be .allowed to
persist.
Footnote 2: If you only want one N_Port (on that switch) in the zone put the port on
a non-shared port (0-15).
Failover Enabled
• If paths within a Tl zone go offline, the Tl zone Failover setting
determines the resulting behavior
• If a "failover enabled" Tl zoned route fails:
- Traffic will be moved to another E_Port in the same Tl zone, if one is
available
- If there are no other E Ports available in that Tl zone, traffic will be
moved to an E Port outside the Tl zone, if available
- When a failed Tl zoned route is restored, traffic will be automatically
failed back to the original route1
• Using the default settings when creating a Tl zone will activate the
zone, with failover enabled

Footnote 1: Failback is not a configurable feature.
Failover Enabled (cont.)

• With Tl zones having failover enabled , if the dedicated ISL fails, traffic will
be rerouted onto the non-dedicated ISLs, since they are the only available
routes
Host A Target A
Tl Zone
~
Host B / Ta<getB
~
10
9 - - - - ..
e
C 2009 Brocade Co'Timun ca1 ans Systems nc ~7

Al 1R ghts Reserved
Traffic Isolation is subject to the rules of FSPF.

Note: This diagram represents only half of a redundant fabric.
Revision 01 10 6 - 47
Failover Enabled (cont.)

• With Tl zones enabled, if the non-dedicated ISLs fails, traffic will be
rerouted onto the Tl zoned ISL, since it is the only available route
Tl Zone
Domain 1
1
- - - ... 2~
3
Target C

When a zone is marked as a ----fBffic Isolation" zone (Tl zone) and failover is
enabled, the fabric will attempt to isolate all inter-switch traffic entering a switch from
a member of that zone to only those E_Ports that have been included in the zone.
In other words, the domain routes for any of the members (N_Port or E_ Port) to the
domains of other N- Port members of the zone will be set to use an E- Port included
in the zone, if it exists. Such domain routes will be used only if they are on a 4.ivest
cost" path to the target domain (i.e. the FSPF routing rules will continue to be
obeyed). The fabric will also attempt to exclude traffic from other Tl zones from
using E_Ports within a different traffic isolation zone. This Traffic Isolation is a -ts
effort" facility that will do its work only as long as doing so does not violate the FSPF
-ffiwest cost route" rules. This means that traffic from one Traffic Isolation zone may
have to share E_Ports with other Tl zones and devices when no equal-cost routes
can be found using a -prerred" E_Port. And if a -prterred" E_Port fails, traffic will
failover to a -ro-preferred" E_ Port, if no preferred E_ Ports offer a -atvvest routing
cost" route to the target domain. Similarly, a non-Tl device's traffic will use an
E_Port from a Tl zone if no equal cost alternatives exist.
Revision 01 10 6 - 48
Failover Disabled
• If a "failover disabled" Tl zoned E Port fails:
- Traffic can still be moved to another E Port in the same Tl zone, if orne
is available
- Traffic cannot be moved to an E_ Port outside the Tl zone, even if one
is available
• An RSCN will be generated noting the failure of a path
• Non-Tl zoned traffic wi II not be moved onto a Tl zoned route, even
if there are no non-dedicated routes available
- If an E_port that is not in the Tl zone fails the non-Tl zoned traffic will
not move onto a Tl route
- When the Tl zoned route is restored , traffic w ill automatically be
rerouted back onto the Tl zoned route
• An RSCN will be generated noting the restoration of a path

Whether failover is enabled or disabled can be determined at the time the Tl zone is
created. The default is failover enabled.
Disabling failover is intended for use in simple linear fabric configurations.
Failover Disabled (cont.)

• With Tl zones having failover disabled, if the dedicated ISL fails, traffic will
not be rerouted onto the non-dedicated ISLs, and the Tl zoned route will be
considered broken
Host A Target A
Tl Zone
~
Host B / Ta<getB
~
10
9 - - - - ..
e

Al 1R ghts Reserved
Failover Disabled (cont.)

• With Tl zones having failover disabled, if the non-dedicated ISLs fails,
traffic will not be rerouted onto the dedicated (Tl zoned) ISL, and the non-
Tl zoned routes will be considered broken
Host A
Tl Zone
Domain 2
-------
Al 1R ghts Reserved
Requirements and Compatibility
• Tl zones are supported with Condor, Condor2, GoldenEye, GoldenEye2-based

Brocade switches running Fabric OS v6.x1
- This feature is supported in Brocade Native Mode (lnterop Mode 0) and in
McDATA Fabric Mode (lnterop Mode 2). It also is supported for EX_Ports,
VEX Ports and VE Ports.
• Tl zones are not supported by versions of Fabric OS prior to v6.x
Target A
Host A
~ain3
Note: While Host Band Target B can form a Tl zone, Host A and Target A cannot,
because of the Fabric OS v5.3 switch in the data path. However, the presence of
the Tl zone in the fabric above will not disrupt traffic flowing from Host A to Target A.
Revision 01 10 6 - 52
Requirements and Compatibility (cont.)

• Tl zones cannot be created on switches running pre-v6.0 Fabriic OS versions,
and must obey FSPF rules.
• In fabrics consisting of mixed Fabric OS versions, Tl zones will not disrupt
traffic in switches running older versions, however, Tl zones cannot be
enforced on a pre-Fabric OS v6.x switch.
Target A
Host A
In the example above, Tl zones cannot be enforced in default configurations,

because Domain 2 is running a version of Fabric OS prior to v6.0, on which the Tl
Zoning feature is not supported. Also, the T l zones would not be enforced using
Domains 4 and 5., since they are not lowest cost paths.
However, it could be possible to change the link costs to make the path through
domain 4 and domain 5 the FSPF path of choice and thereby allowing a Tl zone in
that direction to function.

FSPF routing rules and traffic isolation
All traffic must use the lowest cost path. FSPF routing rules take precedence over
the Tl zones.
• Example 1:
- If the dedicated ISL is not the lowest cost path ISL, then the following ru les
apply:
• If failover is enabled, the traffic path for the Tl zone is broken,
and Tl zone traffic uses the lowest cost path instead
• If failover is disabled, the Tl zone traffic is blocked
Host 1 traffic blocked Host 1 traffic will flow

here if failover disabled here if failover enabled
Domain 1 Domain 3
Host 1
1 9
~14
,
. ,,
Host 2 15 ,,
Domain 2
:r' Domain 4
6 JI ____ _
----
#
Dedicated Path
Ports in the Tl zone
Revision 01 10 6 - 54

Example 2:
• If the dedicated ISL is the only lowest cost path ISL, then the following rules
apply:
- If failover is enabled, non-Tl zone traffic as well as Tl zone traffic uses the
dedicated ISL
- If failover is disabled, non-Tl zone traffic is blocked because it cannot use
the dedicated ISL, which is the lowest cost path
Host 2 traffic merged

here if failover enabled
Domain 1 Domain 3
Host 1
__.. __
Host 2 traffic blocked 6 ~ -----

here if failover disabled 5
Domain 4
I - - - - Dedicated Path
# Ports in the Tl zone
Tl Failover
If failover is disabled:
• Intended for use in simple linear fabric configurations
- Ficon is the driving force behind implementing Tl zone; the Mainframe
wants to see all traffic from the source come from one and only one path,
not from two or more paths.
• Ensure that there are multiple paths between switches
• Ensure that there are non-dedicated paths through the fabric for all devices
that are not in a Tl zone
For administrative reasons, it is recommended that Tl zone definitions and regular
zone definitions match
It is recommended that the insistent Domain ID feature be enabled
• If a switch changes its active domain ID, the route is broken
Revision 01 10 6 - 55
Note the following configuration rules for Tl zones:

Traffic Isolation Routing is supported only on Brocade 200E, 300, 4100, 4900,
5000, 5100, 5300, 5410, 5424, 5450, 5480, 7500, 7500E, 7600 switches, the
Brocade 48000 and Brocade DCX platforms, all configured in Brocade Native Mode
(interopmode 0).
Ports in a Tl zone must belong to switches that run Fabric OS v6.0.0 or later. For Tl
over FCR zones, ports must belong to switches that run Fabric OS v6.1.0 or later.
Traffic Isolation Routing has limited support for FICON FCIP in McDATA Fabric
Mode (interopmode 2), in the following configuration only:
- Brocade 7500 with E Port connections to an M-switch and VE Port connections
to another Brocade 7500.
- Devices attached to M-switch only.
Fabric OS 6.1.0 or later supports Traffic Isolation Routing in a mixed fabric (that is,
a fabric with Fabric OS and M-EOS switch es) operating in interopmode 2. Traffic
1
Isolation Routing is not supported in fabrics configured in Open Fabric Mode

(interopmode 3).
In interopmode 2, a zone member for a Tl zone is limited to a port index of 255 or
less.
VE_Ports are supported in Tl zones.
Traffic Isolation Routing is not supported in fabrics with switches running firmware
versions earlier than Fabric OS v6.0.0. However, the existence of a Tl zone in such
a fabric is backward-compatible and does not disrupt fabric operation in switches
running earlier firmware versions.
Tl over FCR is not backward compatible with Fabric OS v6.0.x or earlier. The -1 in
the domain, index entries causes issues to legacy switches in a zone merge.
Firmware downgrade is prevented if Tl over FCR zones exist.
BROCADE
EDUCATION SOLUTIONS
Tl Zone Create - CLI Commands

• The following command will allow creation of a Tl zone, with
specified options and portlist:
zone --create - t ti [ - o opt lis t] name -p portlist
• The default settings will enable failover and activate the Tl zone
• There are settings to disable failover and deactivate the zone
• To enable a new Tl zone, or commit any changes made, including
deactivation, a cf genable command must be issued
- Activating alone wi ll not enable a Tl zone

zone --operation [- t obj type ] [ - o optionli st ] name - p

[po rt lis t ]
operation: create, add, remove, delete, activate, deactivate and show
-t obj type : ti (traffic isolation zone)
-o optionlis t : a (activate), d (deactivate),n (no-failover), f (enable-failover)
-p portlist : -i:D; ...; port" (In Domain, Index notation)
Tl Zone Create - Examples
• Example 1: create a T l zone with failover enabled and activate the

zone
zone --create - t ti " yellowzonen -p "1, 36; 1 , 216 ; 5 , 10 ; 5 , 4n
• Example 2: create a T l zone with the zone deactivated (d) and

failover disabled (n)
zone --create -t ti -o dn "yellowzonen -p " 1 , 36 ; 1 , 216 ; 5 , 10; 5 , 4n
Host A Target A
Do ma in 1 Do main 5
··· ~
1 , 36 1. 215 - - - - 5. 10

Example 1 shows how to create a Tl zone with default settings -failover enabled,
and activate the Tl zone upon creation. In Example 2, the ~,, argument is required
because the Tl zone is being created with failover disabled, and not activating upon
creation.
To verify that the Tl zone has been enabled on the fabric, issue a cfgshow, or
zone -- show command on each switch in the data path, and verify the Tl zone
shows up in the Defined Configuration.
CLI Commands
• The zone --add command allows the addition/change of portlist
members and change status options
R8 - st02 - DCX : admin> zone --add -o n yellowzone -p
" 1 , 217 ; 5 , 11"
R8 - st02 - DCX : admi n > zone --show
Defined TI zone configuration :
T I zone Name : yel l owzone

Port List : 1 , 2 1 7 ; 5 , 11 ; 1 , 36 ; 1 , 216 ; 5 , 10 ; 5 , 4
Configured Status : Activated I Failover -Di sabled
Enabled Status : ['Deactivated]
Note: the cfgshow CLI command does not include Tl zone

information
Examples:
• Add port member as a portlist to an existing Tl zone
zone - - add " ye ll owzone" - p " 1 , 2 1 7 ; 5 , 11 "
• Add option to disable/enable failover for a Tl zone

zone - - add - o n " yellowzone " -p " 1 , 217 ; 5 , 11 "
z one - -add -o f " ye l low z one " -p " 1 , 217 ; 5 , 1 1"
• Remove portlist member from an existing Tl zone

zone - - remove " yellowzone " - p " 1, 217 ; 5 ,1 1 "
CLI Commands (cont.)

• The zone --remove command allows the removal of portlis t
members
• If the last member of an active Tl zone is removed, the zone will be
deactivated and removed from the defined Tl zone list
R8 - st02 - DCX : admin> zone --remove ye11owzone -p "1 , 217 ; 5 , 11 "
R8 - st02 - DCX : admi n> zone --show
Defined TI zone configurat i on :
T I zone Name : yel l owzone
Port List : 1 , 36 ; 1 , 216 ; 5 , 10 ; 5 , 4
Conf i gured Status : Activated /Fai l ove r - Disab l ed
Enab l ed Statu s : Deact i vated
• The c fg e nable command is required to commit all Tl zoning

changes

• The zone --activate command activates Tl zones
R8 - st02 - DCX : admin> zone --activate yellowzone
Defi ned TI zone configurat i on :
TI zone Name : yel l owzone
Port List : 1 , 36 ; 1 , 216 ; 5 , 10 ; 5 , 4
Conf i gured Status : Activated /Failover - Disabled
! Enabled Status : Act i vated


• The zone --deactivate command deactivates Tl zones
R8 - st02 - DCX : admin> zone --deactivate yellowzone
Defi ned TI zone configur at i on :
TI zone Name : yel l owzone
Port List : 1 , 36 ; 1 , 216 ; 5 , 10 ; 5 , 4
Conf i gured Status : Activated /Failover - Disabled
( Enabled Status : Deactivated


• The zone --de l ete command deletes Tl zones from the defined
configuration
R8 - st02 - DCX : admin> zone --delete yellowzone
R8-st02-DCX : admin> z one -- show
I no TI zone configuration defined

Modified Pre-existing CLI Commands

• These commands will generate an error message1 if used with Tl
zones
- c f gadd/ cfgcre ate
- zonecreate/zoneadd/zonedelete/zoneremove
• The existing cf gs how and zone show commands can be used to

display Tl zones and their members
- Failover attributes and status will not be displayed
• The existing cfgtransshow and cfgtransabort commands

have been modified to support Tl zones

Footnote 1: Erro r: Operation is not a l lowed on TI zone .
Configuring Tl Zones with DCFM

;~ zoning
• DCFM Zoning Tool ZonlngSccpe r$111_st2 Zone oe Fabric Zone oe •
- Zone Type
(
• Zcnes
• Domain;Port Index Type Domain
Fl
Allevels ~ A ~ El • NewZone
- New Zone 13 .@ Swilch Gr0<4>

13 0R11-ST02-830
•New Tl Zone 13 ~ E POO Tnnc 17
lilis E-Por1
Iii 17 E-Por1
El .lj E POO Tnnc 9
Iii a E-Por1
Iii 9 E-Por1
Iii 1 F-Por1
Do lJ.-Port
D2 lJ.-Port
£33 lJ.-Port
ti• lJ.-Port
tis U-Port
Ch U-Port
D1 lJ.-Port
--·-
D10 lJ.-Port
....
ti11
~
NowZone
Br..,_ NetiYe - Now TI Zone

Al R1gh's Reserved
To define Tl Zones in DCFM using the Zoning tool

1. Select the Zone DB tab
2. Choose Domain;Port Index for the Type
3_ Choose New Tl Zone
Configuring Tl Zones with DCFM (cont.)

• Select the E Ports of
all switches in the path
and the device ports
• Icon for Tl Zone
• Configure Failover and

Enable
• Tl Zones do not get
added to zone configs
OUM Messag• Ei
0 TI xone Is nol e v e l d - to add too zone con19n1Jon

Al 1R ghts Reserved
To define Tl Zones in DCFM using the Zoning tool (cont)

4. Select the E_Ports and Devices
5. Add them to the Tl Zone
6. Right-click to edit the failover and enable configuration
7. Activate the Zone Config
Configuring Tl Zones with DCFM (cont.)

• Review the Activate Zone Configuration summary
f'9t'.§IMW¥NH!!.!iil.i!i .;,
: ctw>ges to :<onlng lhot - be In ollect one< ectlvation ore - -. _.._
New~ Any~ CllQNehon eon disrupt tobrtc crottic. Re-Aow your ¢hOnOea COir"eftJJtv bolOl'O cllcldng on Oto<.
F~- ;~ zoning
CuTenl Active Zone Conflgwellor
- e d Zone Con•-otlon Fabric Zone oe

~
Active zone corub,o'•IOn c Zone oe Al:INt> Zone Con~allon
00 12.zcno __ _
Zone Added
>
TIZono~
Zone-Oloplioy Al Membe<s
0 1"TIZoneAdded Active Zone ConfWaliln
0 6 .Zone Members Added
rsai-z......eonflg
Ad.Ne Zone Conflg..a'-CllliOn ~ I 13 !i!D-Zone
El 0 .e.o (S'SI
NowZoneeoniog iJ (F00:0106e4J SEAGATE TECINll.OOY (WMI 2Z00:00:20:37.CA.36:EO)
EJ NowZone ~C0:020100JBr~~
___§t.!!_
ems Inc ~10:00:00:05:1E:S7:7C:8A
0 - IFCI0-020100 13 AdlYe n Zones
> El o ):(
0 ijl IFCD:01°""
Nowzone_;i
0 ~ IFCD 0106001
0 Q IF00-0108001
0 <::> IF00-0109001
J
0 - IFCIO 020100) B<c

e.J
a.J
13 j(] NeWzone_2
13Pr~
f - . Enallled
~ (FCD.010600! SEAGATE TECH-IOl.OOY ( - 1 . port 6). R11-ST02-BS1
Q (f00-010800! Br~~. Inc. ( - 1 . port 8) • R11-ST02-BS1
0 Q JFCI0:021000) ...J
_..ft~r..:r.n..11VHA..nn.1 ::J 0 (FCD.010900! Br~ Cotntta.rbllons, Inc. ( - 1 . port 9) - R11-ST02-BS1
Zone OB ~son Compert ~(FCD.020100JBr~~Systems,lnc_ (_,2,port 1 )-R11-ST02-B30
0(FCD.021000J Br~~.lnc. (-2,port16)-R11 -ST02-830
....J Oonenwe e report wfh 1he ec
0 C0.021100 BrocedeCommlri:eltons Inc. - 2 17 -R1 1-ST02-830
J
Al 1R ghts Reserved
Summary
• Adaptive networking is a framework concept including traffic
management and fabric profiling
• Fa bric QoS allows prioritization of high, medium and low priority
frames (requires Adaptive Networking License)
• Bandwidth can be limited and the ingress port using Ingress Rate
limiting (requires Adaptive Networking License)
• Traffic can be prioritized and rate limited starting at the HBA using
Brocade HBA QoS and Target Rate Limiting (requires Server
Application Optimization License)
• Traffic Isolation zones can control the flow of inter-switch traffic by
creating a dedicated path for traffic flowing from a specific set of
source ports (no license required)

BROCADE
EDUCATION SOLUTIONS
QoS over FC-FC Routers

• Traffic prioritization between devices in edge fabrics over an
FC router supports:
- Only edge-to-edge fabric configuration
• Not supported in a backbone-to-edge fabric configuration
- Brocade native mode only
• Not supported in interopmode 2 or interopmode· 3
- EX- Ports and VEX- Ports
•The EX_Ports (or VEX_Ports) in the path between the QoS devices must be
on switches running Fabric OS v6.3 or later
• QoS zones must use WWN notation only; D,I notation is not
supported for QoS over FC-FC routing

QoS over FC routers is supported only if Virtual Fabrics is disabled in the backbone
fabric. QoS over FC routers cannot be enabled if Virtual Fabrics is also enabled in
the backbone fabric.
QoS over FC-FC Routers (cont.)

• To configure QoS in routed fabrics, you must do the following :
- Define QoS zones in each edge fabric
- Define LSAN zones in each edge fabric
- Enable QoS on the E_ Ports (or VE_ Ports) in each edge fabric
- Enable QoS on the EX_ Ports (or VEX_ Ports) in the backbone fabric
Edge Fabric 1 Backbone Fabric Edge Fabric 2

The port WWN of the host or target and the port WWN of the proxy device must be
in both an LSAN zone and a QoS zone.
QoS over FC routers is supported on both EX_Ports and VEX_Ports. QoS over FC
routers is not supported on the FR4-18i blade.
Zones for Edge Fabric 1:

LSAN_zone1: Host1 , Target1 , Target2
QOSH_zone1: Host1 , Target1 , Target2
Zones for Edge Fabric 2:

LSAN_zone2: Host1 , Target1, Target2
QOSH_zone2: Host1 , Target1 , Target2
Revision 01 10 6 - 73
BROCADE
EDUCATION SOLUTIONS
Tl and FC-FC Routing

• Tl can be used in FC routed environments, ensuring isolation
throughout the routed SAN
• All switches in the path through the backbone fabric and in the
edge fabrics must be running Fabric OS v6.1 .0 or later1
• Set up a Tl zone in the edge fabrics to guarantee that traffic from a
specific device in that edge fabric is routed through a particular
EX Port or VEX Port
• Set up a Tl zone in the backbone fabric to guarantee that traffic
between two devices in different fabrics is routed through a
particular ISL (VE_Ports or E_ Ports) in the backbone
• LSAN_ zones should align with the Tl zones to ensure proxy device
translation2

Footnote 1: Support for Tl zones across FC-FC routing was not implemented until
Fabric OS v6.1.0.
Footnote 2: It is a best practice to have these align, however, it is not a requirement
that the LSAN and Tl zones be exactly the same. At the least, the devices that are
using the Tl paths must be in the LSANs for proper functionality.
Tl and FC-FC Routing Example
--~--
.. .. .... .._.}
~
-
Edge Fabric 1 Backbone Fabric Edge Fabric 2
j- - - Dedicated path set up by Tl zone in edge fabric 1~

- - Dedicated path set up by Tl zone in edge fabric 2
L - Dedicated path set up by Tl zone in backbone fabric

Al 1R ghts Reserved
Tl and FC-FC Routing Configuration

• In the edge fabric Tl zone you must include:
- Ports of the initiators
- Ports of the targets
- E_ Port and EX_ Ports, including the front and translate domains
• To zone the phantom ports on the front and translate domains, use
the value '-11' for the index in D,I notation
• Tl over FC-FC Routing is not backward compatible with Fabric OS
v6.0.x or earlier1
- Firmware downgrade is prevented if Tl over FCR zones exist
• Failover must be enabled in the Tl zones in the edge fabrics
• The failover mode for Tl zones in the backbone fabric can be
enabled or disabled

Footnote 1 :The -1 in the D,I entries causes issues to legacy switches in a zone
merg1e
Tl Zone Within An Edge Fabric

• The Tl zone would have the following members:
- 1,8
- 1,1
- 3,-1 (E_Port for the front phantom domain)
- 4,-1 (E_Port for the xlate phantom domain)
Domain 1 Front Domain 3
9
Host 1
2 10
/
EX_Ports
1-1,
-1
Host2
Xlate Domain 4 Proxy Target

I-- Dedicated Path I
j # Ports in the Tl zone I
Example for creating the Tl zone in the edge fabric
Elswitch:admin> zone --create -t t~ TI Zonel -p " 1 ,8; 1 , 1 ,

3,-1; 4 , -1 "
Elswitch : admin> zone - -show

TI Zone Name: TI Zonel
Port Lis t : 9, 2; 9 , 3 ; 9 ,6; 1 ,- 1 ; 4,-1
Configured Status : Activated I Failover- Enabled

Enabled Status : Deactivated
Tl Zone Within A Backbone Fabric

Host Target Target
• In the edge fabric Tl zone 1
you must include:
- PWWN of the initiator(s) I
~ ~~
- PWWN of the Target
'• '
- EX Port and VE Ports
• The Tl zone would
( Edge
Fabric 1
Edge
Fabric 2
I I
•••
have the following •
I
members: I
I
•
1 2 3 1 2
- 1, 1 (EX_Port for FC router 1)
- 1,4 (VE_Port for FC router 1) VE Ports
- 2,7 (VE_Port for FC router 2) 4 - - - 7'
- 2, 1 (EX_Port for FC router 2) 5 8

6 9
- Host PWWN
FC Router 1 FC Router2
- Target1 PWWN
- Target2 PWWN BackbOne Fabric

Al 1R ghts Reserved
Footnote 1: Non-Tl data traffic is not restricted from going through the Tl path in the
backbone fabric.
Example configuration:
• Host PWWN - 10:00:00:00:00:08:00:00
• Target1 PWWN - 10:00:00:00:00:02:00:00
• Target2 PWWN - 10:00:00:00:00:03:00:00
BB DCX l: a dmin > zone --create -t ti TI_Zonel -p "1,1; 1 , 4 ;

2,1 ; 2,7; 10:00:00:00:00:08 :00: 00 ; 10:00 : 00:00:00:02:00:00 ;
10 :00:00:00:00:03:00:00"
BB DCX l: a dmin > zone --show

Def ined TI zone configur ation :
T I Zone Name : TI Zonel
Port Lis t : 1 , 9 ; 1 ,1; 2 ,4 ; 2 , 7 ; 1 0 : 00 : 0 0 : 00 : 00 : 0 8 : 00 : 00 ;
10 : 00 : 0 0: 00 : 00 : 02 : 00 : 00 ; 1 0 : 00 : 0 0 : 0 0 : 0 0 : 03 : 00 : 0 0
<Ou tpu t trunca ted>
Revision 0110 7- 1
Objectives
• After completing this module, attendees will be able to:
- Interpret Top Talkers (TT) output
- Utilize bottleneck detection to proactively monitor Brocade FC SANs

Revision 0110 7-2

BROCADE
EDUCATION SOLUTIONS
Revision 0110 7-3

Licensing
• Adaptive Networking does have a license, but is not required for

all the features, where some features require additional licenses
Services Service Name Fabric OS License Required
Traffic Fabric QoS Adaptive Networking
Management QoS on an HBA/ Server Application Optimization
Target Rate Limiting and Adaptive Networking
Ingress Rate Adaptive Networking
Limiting
Traffic Isolation None
r
Fabric Top Talkers Advanced Performance Monito r "
Profiling Bottleneck None
detection
Revision 0110 7-4

BROCADE
EDUCATION SOLUTIONS
Revision 0110 7-5

The Top Talkers Feature

• Top Talkers (TT) is an enhancement to Advanced Performance
Monitor (APM) end-to-end monitors
- When enabled , these monitors determine which SID-DID pairs are the
major users of switch F_ Port bandwidth
- Can be enabled on specific switch E_ Ports oir F_ Ports in the fabric
• Determines the flows (SID/DID pairs) that are the major users of
bandwidth
• Measures b.andwidth usage data in real-time and relative to the
port on which the monitor is installed
• Requires APM license

The Top Talker feature is based on the Advanced Performance Monitor (APM)
feature. The Top Talker feature determines the largest users of F_Port bandwidth by
monitoring all flows (SID-DID pairs) through one or more switch F_Ports on any
switch in the fabric.
This feature does not work on F_Port on cascaded Access Gateways.
Top Talker Monitors discards bandwidth information collected during the initial
stabilization. Initial stabilization is the time taken by a flow to reach the maximum
bandwidth. This time varies depending on the number of flows in the fabric and
other factors. The incubation period can be up to 14 seconds in the Brocade DCX
and DCX-4S, and up to 82 seconds in the Brocade 4100, 4900, 5000, 5100, 5300,
7500, 7500E, 7600, 7800, 8000, 48000, and Brocade Encryption Switch.
Revision 0110 7-6

Advanced Performance Monitor (APM)

• This licensed feature provides comprehensive tools for monitoring
the performance of networked storage resources
- Supports direct-attach, loop, and switched-fabric Fibre Channel SAN
topologies
- Monitors transaction performance from source to destination (end-to-
end monitoring)
- Reports Cyclic Redundancy Check (CRC) error measurement statistics
- Measures ISL performance and resource usage

Revision 0110 7-7

APM (cont.)
• In earlier versions of Fabric OS, APM could capture end-to-end
performance information, but not perform any further analysis
- Which SID-DID pairs are driving the most traffic?
- Which switch ports are experiencing the heaviest traffic?
Fabric Top Talker:
1. SID1 - 010 1
2. SID2 - DID1
3. SID3 - 0104

Al 1R ghts Reserved
In the example above, the busiest SID/DID pairs are shown. Advanced Performance
Monitor can measure performance quantitatively, but cannot determine the "busiest"
SID/DID pairs. Knowing the busiest devices can be a key factor in optimizing the
performance of a SAN design.
Revision 0110 7-8

End-to-End vs. Top Talker Monitors
End-to-End Top Talker
Identify all possible SID-DID flow

Focus on a single SID-DID pair
combinations on a given port
May fail to get real-time data if the

number of flows exceeds the
hardware resources Can monitor up to 10,000 flows
• Condor-based: Up to 256 flows
• Condor2-based: Up to 2048 flows
Can display a sorted output of the

Can display performance data for
largest flows for a given E_ Port or
only one SID-DID pair
F Port

Note: A Flow is a stream of data traffic.
Revision 011 O 7-9

Top Talker Modes

• Top Talker can be configured in two modes:
- Port Mode: enabled on an F Port to measure the traffic between the
F Port and all other devices that it can communicate with
- Fabric Mode: enabled on all E Ports in the fabric to measure the data
rate of all the possible flows in the fabric (ingress E_Port traffic only)
• In Fabric Mode, Top Talker monitors can determine the top n bandwidth
users on a given switch
• Can be configured in Port Mode or Fabric Mode only, not both

simultaneously

Top Talker Applications

• Traffic monitoring: Determine the largest flows so traffic can be
re-routed through a switch or fabric to balance bandwidth utilization
• Virtual servers: Determine the largest flow virtual servers through

a given HBA
• Service tracking: Identify when the largest flows exceed a

maximum standard set by a service agreement
• Adaptive Networking: Identify flows that would gain the greatest

benefits from Adaptive Networking1 features used to optimize fabric
behaviors

There are a number of ways to balance bandwidth utilization, including trunking,

adding more ISLs, and Tl zoning, to name a few.
Footnote 1: The Adaptive Networking licensed feature introduces QoS and Ingress
Rate Limiting functions.
Top Talker Details

• Top Talker monitors cannot be installed on a mirrored port
• Top Talker data is available after an initial stabilization period 1
- Condor2-based ASIC: Up to 14 seconds
- GoldenEye2-based ASIC: Up to 14 seconds
- Condor-based ASIC: Up to 82 seconds
• Top Talker monitors can track up to 10,000 flows in the fabric2
• The maximum number of F_Port Top Talker monitors on an ASIC
is eight3

Footnote 1 : Initial stabilization period is the time taken by a flow to reach the
maximum bandwidth. This time varies depending on the number of flows in the
fabric and other factors. The stabilization period can be up to 14 seconds in the
Brocade 300, 5100, 5300, and DCX; up to 82 seconds in the Brocade 4100, 4900,
5000, and 48000. The Brocade 200E is not supported (GoldenEye ASIC).
Footnote 2: Because Top Talker identifies all possible flows on a given switch port
or switch, Top Talker may exceed the ASIC hardware resources (up to 2048 flows
per Condor2; up to 256 flows per Condor). If there are more flows than the H/W
resources can support, the Top Talker algorithm samples traffic (by looking at a new
set of 256/2048 flows every second) and extrapolates the measurement (estimating
the actual performance from the sampled data).
Interaction with other Fabric OS features:
• Administrative Domains: Top Talker monitors are placed in AD255
• FCIP and FC Routing: Not supported on VE_Ports, EX_Ports, or VEX_ Ports
• Virtual Fabrics: All logical switches in the same chassis can use either fabric
mode Top Talker monitors or port mode Top Talker and end-to-end monitors.
You cannot use fabric mode Top Talker monitors and end-to-end monitors
together on the same logical switch.
Footnote 3: If Virtual Fabrics is enabled, the maximum number of F_ Port Top Talker
monitors on an ASIC is 4.
Revision 01 10 7 - 12
Top Talker Details (cont.)

• Requires the APM license to be installed on all switches
• Not available a Brocade 200E switch or earlier platforms
• VE, EX, VEX and M_Ports (mirror ports) are not supported 1
• Supports NPIV-attached devices
• Not supported on the CEE ports of the Brocade 8000 switch

Footnote 1: FC-FC routing and TopTalkers Fabric Mode cannot coexist on the
same switch. Enabling one while the other is already enabled will be prevented.
Top Talker F_Port

• Add an F_ Port Top Talker monitor to a switch port:
- In-coming traffic: perfttmon - - add ingress [slot I] <port>
perfttmon --add ingress 7
- Out-going traffic: perfttmon - - add egress [slot/ ] <po r t>
perfttmon --add egress 2/4
• Delete an F_ Port Top Talker monitor from a switch port with the
perfttrnon --de lete [slot/] <port> command

• The perfttmon - -add ingress command adds a F_ Port Top Talker monitor
for traffic entering a switch F_ Port (receive, or Rx). The command has one
argument: the [slot]/port identifier for the port.
• The perfttmon - -add egress command adds a F_Port Top Talker monitor
for traffic exiting a switch F_Port (transmit, or Tx). The command has one
argument: the [slot]/port identifier for the port.
• The perfttmon --delete command deletes an existing F_ Port Top Talker
monitor from a port. The command has one argument: the [slot]/port identifier
for the port.
Rll - ST11 - B30 : admin> perfttmon --show 1 5
Src WWN Dst WWN MB/sec
10 : 00 : 00:05 : 1e :57 : 7c : a4 22 : 00 : 00 : 04 : cf:bd:89 : 5f 60 . 290

10 : 00 : 00 : 05 : 1e : 57 : 7c : a4 22 : 00 : 00 : 04 : cf : 92 : 5c : al 60 . 233
;;;rop Talkers - F Port Mode for 20:01:00:05:1E:OB:EA:CA
~ Display Top 5 ..- 1Refresh Interval 10 Seconds ..- 1Flow Tx ..- \ Pause I~
Top Talker Summary
Tx Ave( ... ..., Last Occured Occurances SID Source Name Source Swlt ... DID Destination Name Destination S... Port Speed % utilization
50.615 Thu Nov 19 22:24:5... 3 030100 10:00:00:05: ... R11-ST11-B... 6206EF 22:00:00:04:CF:92 ... R11-ST11-B... 2 19.53125
Thu Nov 19 22:24:5... 3 030100 10:00:00:05: ... R11-ST11-B... 6206E8 22:00:00:04:CF:BD... R11.ST11-B... 2 17.96875
Revision 01 10 7 - 14
Top Talker Implementation (cont.)

• Display the top flows on an F_ Port with the perfttrnon --show
[slot/ J <port> [wwnlpid ] [#of TT f l ows ] command
- Default: Top 10 flows displayed in WWN format
- Can display up to 32 of the largest flows, depending on platform 1
DCX : adrnin> perfttmon --show 1/12 pid 5

--------------------------------------------------
--------------------------------------------------
Src PIO Ost PIO MB/sec
Ox030c00 Ox060600 150 . 123

Ox030c00 Ox110800 10 . 001
Ox030c00 Ox021800 9 . 999
Ox030c00 Ox060c00 0 . 000
Ox030c00 Ox060400 0 . 000

The perfttmon - - show command displays the largest flows measured by the
F_Port Top Talker monitor on a port. The command has the following arguments:
• [slot]/port: The identifier for the port. A mandatory argument.
• [wwnlpid]: The format of the SID and DID identifiers - WWN (wwn) or PIO
(pid). The default is WWN format; an optional argument.
• [#of TT flows]: The number of largest-bandwidth flows to be displayed. The
default is 10 flows; an optional argument.
In the example above, the 5 largest flows through port 1/12 are displayed in PIO
format.
Footnote 1: The number of flows displayed is dependent on the hardware platform;
32 flows for Brocade 300, 5100, 5300, and FC8-xx port blades; 16 flows for
Brocade 4100, 4900, 5000, 7600, and FC4-xx port blades; 4 flows for Brocade
7500.
Top Talker Fabric Mode Implementation

• Add E_Port Top Talker monitors to a fabric:
perfttmon --add fabricmode
•This will enable TT monitors on all E Ports in the fabric
• Delete all E_ Port Top Talker monitors from a fabric:

perfttmon --delete fabricmode

If end-to-end monitors are present on the local switch, the command fails with the
message:
Cannot install Fabric Mode Top Talker because EE monitor is
al r eady present
If end-to-end monitors are present on remote switches running Fabric OS 6.1.0 or
later, the command succeeds; however, on the remote switches, fabric mode fails
and a raslog message is displayed on those switches.
If end-to-end monitors are present on remote switches running Fabric OS 6.0.x, the
command succeeds.
Rl l-ST1 1 -B30 : admin> perfttmon --show dom 3 5
Src WWN Ost WWN MB/sec Potential E-Ports
22 : 00 : 00 : 04 : cf : bd : 89 : 5f 10: 00 : 00 : 05 : 1 e : 57 : 7c : a4 51 . 850 16 9
22 : 00 : 00 : 0 4: cf : 92 : 5c :al 1 0 : 00 : 00 : 05 : 1e : 57 : 7c : a4 44. 867 16 9
~{ Top Talkers - Fabric Mode for RI l - STll -B30
Select... Display Top 5 •J Refresh Interval 10 Seconds~ Flow .:J Pause I~

Current Top Talkers
ITx+Rx (M ... ...-ls10 !source Name JSO;e Swltchi?ort ...,. fDID loestination Name l0estination Swltchll'o1
58.418 6206EF 22:00:00:04:CF:92:5C:A1 R11-ST1 1 -B51 l20:06:00:05:1 E:A1 :6A:B7 030100 10:00:00:05:1 E:S7:7C:A4 R11-ST11 -B30/20:01 :OO:OS:1E:OB:EA:
l57 .885 6 206E8 22:0():00:04:CF:BD:89:5F R11-ST1 1 -B51l20:06:00:()5:1 E:A1 :6A:B7 0301 ()() 10:00:()0:()5:1 E:57:7C:A4 R11 -ST11 -B30/2():01 :00:05:1 E:()B:EA:
Top Talker Fabric Mode Implementation (cont.)

• Display the top flows on a switch for a given Domain ID:
- perfttmon -- show dom domainid [n] [wwn I pid]
- Default: Top 10 flows displayed in WWN format
- Can display up to 32 of the largest flows, depending on platform 1
Rll-ST11-B30 : admin> perfttmon --show dom 3 5
Src WWN Ost WWN MB/sec Potential E- Ports
22 : 00 : 00 : 04 : cf : bd : 89 : 5f 10 : 00 : 00 : 05 : 1e : 57 : 7c : a4 51 . 850 16 9
22 : 00 : 00 : 04 : cf : 92 : 5c : al 10 : 00 : 00 : 05 : 1e : 57 : 7c : a4 44 . 867 16 9

Footnote 1: Maximum number of flows displayed is dependent on the hardware

platform; 32 flows for Brocade 300, 5100, 5300, and FC8-xx port blades; 16 flows
for Brocade 4100, 4900, 5000, 7600, and FC4-xx port blades; 4 flows for Brocade
7500.
Configuring Top Talker with DCFM

• Setup Top Talker monitors under the Monitor/Performance menus
M4M·iil1!0Hl1U
fe<lormance • .J V'tttw !Jillallon Or...U

!echnlceill ~
Event Poides
•j tjstorical Dela Coledion
End-Io-fnd Meriors
Event ~1lcal1on ·r ~•a Th'esholds
~Set\4)
Al Levels ~ Configui-etlon Iop Telce<S ..------j__.
El .p rSl11_st2
El '9 Swlch Gr0\4l L.ogs B.eal-Tine Gref>h
El 0 R11-ST02-830 2 40 .11---
B.eport
__ s - - -- l ltstorical 'iiras>h .3.C
13 Q R11-ST02-851 2 40.1 0 Track Fabric Qianges ..,__H§.l
_ orical
__ _ _ _ ___,3.c
Report
©20:01:0005:11::7
4 20:02:00:05:1 E:7
4 20:0A:00:05:1E:7
J> 22:000020:37:C
4 20:05:00:05:1 E:7
El 0 ~ rSl11_Sl2_m44
El R11-ST02-M44 2 40.18 97Mc0ATA Sphereon 44 ... 16 09.09
•
:: 2009 Brocade Commun ca! ens Sys'e'TIS l"IC • 8

Al R1g~·s Reserved
Setup Top Talker with DCFM

1. Select Monitor drop down menu
2. Select Performance
3. Choose Top Talkers
Configuring Top Talker with DCFM (cont.)

• F_ Port Top Talkers Mode
- Select F_ Ports for mo nitoring
; {. Top TcJlker Selecto•r
: Selotted (31 POrl•. 9 swlches ·~

Device Type 1Clan Find ~ _. Swl:ch,l«lme Fobrlc.Jllame f
rsl11~~-=:J ~ 20:0Ul0:05; R11-ST02-830 rsl11 _s12
:.i:IM.lilllloM:llloll.-1:11.1~
__fj 2Q;OJ .OOQS·l Ej9
8 O ~ rs111_S12.Jn« rs111_s12_rn.c4 ~1
8 OR11·ST02.M44 R11.S102-M44 Swich ~
U1I 20:05:08:00:88:E3:33 <I J

Setup Top Talker with DCFM (cont.)

1. Select Top Talkers Mode drop down menu
2. Select F_Ports

• Fa bric Top Talkers Mode
- Select a switch in each fabric for monitoring
~{ Top Talker 5electorr £1
Top Telcers - febrie .,..
AvM&llle ~~~...,., I : Selecttd (9 swteheS (~)

AILcvels Nome = ' - ' - ' = - ' = = - i Fird · Ch N • fp Adch$$ IFebrle Nelne
je c;. rs111_s12 rin1_s12 1-Sl02.SS1 10:00:0005:... 10 ~255.240.17 1_s12
v R11-Sl , -930 ""'~::.
0 R11-ST02-851
0 rsl11 _st2_rn44

Al R1gh's Reserved
Setup Top Talker with DCFM

1. Select Top Talkers Mode drop down menu
2. Select Fabric
3. Choose a switch in the fabric(s) desired

;.',; Top Talkers · Fabric Mode lor R l 1·5T02·BSI l!I[!] Ei
Started TIWJ Oct 08 22:30:21 POT 2009
Select... Oispey Top S ... Refresh lnle.vel 10 Seconds • Flow
Last
O....e<t Top Tal<ers
rx+Rx CM .. ....rs.i Sou-ce Nome JSou-ce SWll oo Destriet1on N.=1Desli'>ollCln s.J
0.0 020100 111:00:0005: ... R11,Sl02·B OHJEt.T 1 2?00 0020: ... R11-ST02-B...
I.:.
~ I] !For F_Port monitors
Top S
lOSeeonds
Top6
Seconds
Top7
Seconds
Top8
Seconds
Top9
Seconds
Top10
IMh.te
Top11
Top 12 ~
Tep Tal<er Sunmory

_T_x+Rx •.•_:Jl
_ A_v_ ._L_as1_ 0ccu'
_ _e<1 _ _ _~·s.i
__0cctnnces ----fSou-ceName _ISou-ceswt ...Joo TDe.stnation N JDes1ne1Jon s..f
0.0 Tl>J Od 08 2 ... 3 020100 10:00:00:05: ... R11-ST02-B... ()106E4 22:00:0020: ... R11-ST02-B ...

Al R1gh's Reserved
BROCADE
EDUCATION SOLUTIONS
Bottleneck Detection
• The bottleneck detection feature identifies devices attached to the
fabric that are slowing down traffic (slow drain device)
- Device is slow to process received frames and send back credit
returns
- Achieved throughput ~nto the slow drain port is lower compared to
intended throughput
2 GbiVsecbw
2 GbiVsec link
- 8 GbiVsec link

A latency bottleneck is a port where the offered load exceeds the rate at which the
other end of the link can continuously accept traffic, but does not exceed the
physical capacity of the link.
Latency bottlenecks can be caused by a device attached to the fabric that is slow to
process received frames and send back credit returns. A latency bottleneck due to
such a device can spread through the fabric and can slow down unrelated flows that
share links with the slow flow.
The following example, while technically a bottleneck, is simple

oversubscription/overutilization. The bottleneck detection feature does not look for
this.
3 Gbit/sec bw
> 8 Gbittsec
"' 3 Gbit/sec bw
~ 2 Gbit/sec link
4 Gbit/sec link
.,_.... 8 Gbit/sec link
Revision 01 10 7 - 23
Slow Drain Device

• A slow destination returns credits slower than the sender wants to
send frames to it
• Slow drain can exist at any link utilization level from 0°/o to under
1 OOo/o
- Not just high utilization scenarios
• Slow drain spreads into the fabric and can slow down unrelated
flows in the fabric
Ji Slow drain device

~ Slow drain flow
~ Affected flow

Bottleneck Detection (cont.)

• Does not require a license
• Supported on Condor2/GoldenEye2 ASIC Fibre Channel ports only

- F_and FL_Ports only
- F_Port trunks are not supported
• Bottleneck detection is disabled by default, and must be explicitly

enabled for each port that is to be monitored
• Supported in Access Gateway

Condor2/GoldenEye2 F_and FL_Ports in a 48000 are supported.
Reporting Slow Drain

• Two options for reporting slow drain
- RAS log if the metric value crosses a configurable threshold
- On-demand inspecting via CLI
• RAS log identifies port and value of metric at a point in time
• CLI reports value of metric over a period of time (up to 3 hours)

Al R1gh's Reserved
Slow Drain Metric

Affected?
I-second
1 --· --·--
inteIYals
Areraging
inte1Tal ( 12
seconds) )
0 ... Time
The metric value for this example is 4/12*·100% = 33.33%

The averaging interval is configured in seconds, the threshold is a percentage value

represented by a fraction between O and 1.
An affected second is any 1 second period where the port has zero buffer credits
and a frame waiting to transmit.
In the example the averaging interval is configured for 12 seconds, during this
interval there were four affected 1 second periods giving a metric of 33.33%.
If the threshold was configured to be .3333 or less then a RAS log message wou Id
be generated.
Revision 01 10 7 - 27
Configuration - CLI Only

• To enable bottleneck monitoring without RASLOG alerts on a
specified port range (this may be useful because the history can be
obtained through the CLI):
switch : admin> bottleneckmon --enable 2/24 - 26
• To enable bottleneck monitoring on all ports of a switch with
RASLOG alerts using default values for threshold and time
(preferred use case):
switch : admin> bottleneckmon --enable - alert '' * "
• To enable bottleneck monitoring on a port range with RASLOG
alerts using custom values:
switch : admin> bottleneckmon -- enable - alert - thresh . 1
- time 240 1 - 15

Alerts Enabled (Y) or disabled (N).
Threshold The severity threshold for triggering an alert. This threshold

indicates the percentage of one-second intervals affected by the
bottleneck condition within a specified time window. The severity
threshold is expressed as a fraction between 0 and 1.
Time (s) The time window in seconds over which the bottlenecking
percentage is computed and compared with the threshold.
Quiet Time (s) The minimum number of seconds between consecutive alerts.
The -qtime option can be used to throttle alerts by specifying the
minimum number of seconds between consecutive alerts.
The --status option displays a listing of ports for which bottleneck detection is
enabled.
Configuration (cont.)
• To display 30 seconds of bottleneck statistics with a 5 second
interval period:
switch : admin> bottleneckmon --show -interval 5 -span
30 2/24
===========================================================
Mon Jun 15 18 : 54 : 35 UTC 2009
-----------------------------------------------------------
Pe r ce ntage of
Fr om To affected secs
Jun 15 18 : 54 : 30 Jun 15 18 : 54 : 35 80 . 00 %
Jun 15 1 8 : 54 : 25 Jun 15 1 8 : 54 : 30 40 . 00 %
Jun 15 1 8 : 54 : 20 Jun 15 1 8 : 54 : 25 0 . 00 %
Jun 15 18 : 54 : 15 Jun 15 18 : 54 : 20 0 . 00 %
Jun 15 18 : 54 : 10 Jun 15 18 : 54 : 15 20 . 00 %
Jun 15 1 8 : 5 4 : 05 Jun 15 18 : 5 4 : 10 80 . 00 %

Setting a threshold of 0.1 and a time window of 30 seconds specifies that an alert
should be sent when 10% of the one-second samples over any period of 30
seconds were affected by bottleneck conditions. The -qtime option can be used to
throttle alerts by specifying the minimum number of seconds between consecutive
alerts.
Syntax :
bot t l eneckrnon --enable [-alert ] [ - thresh t hreshold ]
[-time window] [-qtime quiet_ time ]
[slot/]port list [[slot/ ] port list ]
bottleneckrnon --disable [slot/]port list
[ [slot/] port list] ...
bottleneckrnon -- show [ - interval interval size]
[-span span size] [slo t/]port
bottleneckrnon - status
bottl eneckrnon --help
Revision 01 10 7 - 29
Summary
• Top Talker monitors help to profile the fabric by determining the
flows (SID/DID pairs) that are the highest users of bandwidth
• Top Talkers measure bandwidth usage data in real-time and
relative to the port on w hich the monitor is installed
- Requires Advanced Performance Monitoring license
• Bottleneck detection helps to find F_Port dev ices that may be
experiencing slow drain and affected traffic w ithin the fabric
- Does not require a license
• A slow drain device is slow to process received frames and send back
credit returns

BROCADE
EDUCATION SOLUTIONS
CFP380 Internal Use Only NPIV and Access Gateway
Revision 0110 8-1

Objectives
• After completing this module and associated lab, attendees will he
able to:
- Describe and implement NPIV
- Discuss Access Gateway features and functionality

Revision 0110 8-2

BROCADE
EDUCATION SOLUTIONS
Revision 0110 8-3

N_Port ID Virtualization (NPIV) Benefits

• Virtual servers require secure access to storage in the same way
as physical servers
• Without NPIV, a single· physical server connection is unable to
provide independent storage access to individual virtual servers
• All storage ports and Logical Unit Numbers (LUNs) are exposed to
all virtual machines, which reduces security and manageability
r=========J~VM_2o o_l I
Without NPIV. zoning would need to

use the physical HBA's 1/1/WN,
which allows every virtual machine to access all LUNs

N_Port ID Virtualization (NPIV) allows a single FC_Port to appear as multiple,

distinct ports providing separate port identification and security zoning with the
fabric for each operating system image as i f each one had its own physical port.
Revision 0110 8-4

NPIV Benefits (cont.)

• NPIV enables a single Fibre Channel port to appear as multiple,
distinct ports
• Provides separate port identification within the fabric for each
operating system image behind the port (as if each operating
system image has its own unique physical port)
• The virtual port has the same properties as an N_ Port, and is
therefore capable of registering with all services of the fabric
~ 8Gbit/sec
SI ~ FCDisk
~-
Brocade
HBA
Cl VM_Zone_ 1
0 VM_Zone_2
0 VM_Zone_3
• VM_Zone_4
Four separate logical traffic
nows inside single cable

Revision 0110 8-5

NPIV Overview
• Available on all Brocade 4 and 8 Gbps switches
• Enabled on a per-port basis
• Each NPIV device is assigned a unique:
- Device PIO
- PortWWN
- NodeWWN
• To the fabric, the NPIV device acts the same as all other physical
devices in the fabric
• NPIV is defined in the FC-LS T11 standard

NPIV devices connected to the same switch port must have a unique 24-bit address
as well as a unique device WWPN.
Revision 0110 8-6

NPIV Zoning
• Standard fabric zoning and storage LUN masking can be used with
virtual machines to isolate storage ports and LUNs to the
appropriate virtual server just as they are with physical servers
• To perform zoning to the granularity of the virtual N_Port IDs,
WWN-based zoning must be used
~
~
LUN masking
performed at the
/ _,,, storage controller level
~
ID VM_lone_1
D VM_Zone_2
Zoning performed at the
10VM_Zone_3
fabric switch level • VM_Zone_4

You can also use domain, port zoning for an NPIV port, but all the virtual PIDs
associated with the port are included in the zone. A port login (PLOGI) to a non-
existent virtual PIO is not blocked by the switch; rather, it is delivered to the device
attached to the NPIV port. In cases where the device is not capable of handling
such unexpected PLOGls, you should use WWN-based zoning.
Revision 0110 8-7

NPIV Scalability
• Each NPIV-enabled port on the switch can support up to
255 devices1
• For the shared area ports on 48-port blades (ports 16 through 47),
the limit is 127
• The number of NPIV devices supported on shared area ports (48-
port blades) is reduced from 127 to 63 when Virtual Fabrics mode
is enabled

Footnote 1: Default value in Fabric OS is 126. To support more devices, the value
must be changed using configure.
Revision 0110 8-8

NPIV Scalability (cont.)

• To specify the number of virtual N_Port_IDs per port or per switch ,
use the configure command with either of the following
parameters:
- Maximum logins per port 1
- Maximum l ogins per swi t ch2
switch : admin> configure

Confi gure . ..
Fabric parameters (yes , y , no , n) : [no)
Virtual Channel parameters (yes , y , no, n) : [no]
F- Purl luyin pdrdmeler::; (ye::; , y, nu , 11) : [nu] y
Maximum logins per switch: (1 .. 25200) (3200)
Maximum loqins per port: (1 . . 255) [126)
Logins per second : (0 .. 100 ) [O J
Login stage interval (milliseconds) : (0 .. 10000) [0 ]
<truncated output>

Footnote 1: Use this parameter to set the number of virtual N_Port_IDs per port to
a value between 0 and 255. The default setting is 126.
For the Brocade 48000 director with an FC4-48 port blade or the Brocade DCX or
DCX-4S Backbone with an FC8-48 port blade: For ports 0 through 15 on the FC4-
48 and FC8-48 port blades, the maximum number of virtual N_Port_IDs per port is
255; for ports 16 through 47, the maximum number is 127.
Footnote 2: Use this parameter to set the number of virtual N_Port_IDs per switch
to a value between 0 and 126 multiplied by the number of ports you specify when
setting this parameter. The default setting is 16 multiplied by the number of ports
specified. If no ports are specified then all ports on the switch are used.
Revision 0110 8-9

Requirements to Implement NPIV

• Switches - NPIV needs to be supported on the switch connected to
the HBA. All Brocade FC switches currently support NPIV, that are
running Fabric OS v5.1.x or later.
• HBAs - HBAs must support NPIV. The following vendors and
models of HBAs are supported:
- Brocade - Any 4 or 8 Gbps FC HBA
- Emulex:
• 4 Gbps HBA running firmware level 2.70a5 or later
• 8 Gbps HBAs running firmware 1.00a9 or later
- Qlogic - Any 4 or 8 Gbps HBA
• Storage - N PIV is completely transparent to storage arrays

Enabling NPIV
• Enable/disable NPIV on a FC port with the port cf gnpi vpor t
command 1
• NPIV is enabled by default
• The portcfgshow command displays the NPIV status for each
port
switch :admin> portcfqshow
Ports of Slot 0 0 1 2 3 4 5 6 7 B 9 10 11 12 13 14 15
-----------------+ --+ -- +--+--+----+--+-- +--+---- +--* -- +--+---- +--+--+--
Speed AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN
AL_PA Offset 13
Trunk Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON
Long Distance
VC Link !nit
Locked L_Port
Locked G_Port
Disabled E_Port
ISL R RDY Mode
RSCN Suppressed

Footnote 1: Command syntax is po r tcfgn pi vport [slot/] p ort mode

Enables or disables NPIV on the specified port. Specify mode 1 to enable or 0 to
disable the feature.
Revision 01 10 8 - 11
NPIV port show

• Use the port show command to view the NPIV attributes and all
the N_Port (physical and virtual) port WWNs that are listed under
portwwn of device(s) connected
switch : admin> port show 2
portName :
portHealth : HEALTHY
Aut.hentication : Hone
portDisableReason : None
portCFla9s : 0:-:1 ....r::7:'1
portflags : Ox20b03 PRESEHT ACTIVE F _PORT G_PORT U_PQR, L!!!:!::JLOGICAL_ONLINE LOGIN NOELP ACCEPT FLOG!
portType : 18 . 0
POD Port : Port is licensed
portState : 1 Online
Protocol : FC
portPhys : 6 In_Sync portScn: 32 F_Port
port 9'~me r at i on number: 0
st.ate transit.ion count. : 0
PWWN of HBA (physical)
port Id : 010200
portlfid : 43020013

NPIV swi t chshow

• For a regular F_Port, the port WWN of the N_Port is returned
• For an NPIV F_Port, there are multiple N_Ports, each with a
different port WWN
• The command output indicates whether or not a port is an NPIV
F_ Port, and identifies the number of virtual N Ports behind it
switch : admin> switchshow
Dev st02 b300: admin> switchshow
switchName : switch
switchType : 71. 2
switchState : Online
switchMode : Native
switchRole : Principal
switchDom~in : l
switch Id : fffcOl
switchWwn : 10 : 00: 00 : 05 : le : Oa : S3 : 6d
zoning : ON (NewZoneConfi9)
switchBeacon : Off
0 0 010000 id NS No_Li9ht re
1 010100 id NS Online fC f-Port 10 : 00 : 00 : 05 : le : 56: c8 : 2a
2 2 010200 id NS Online I re r-Port l N Port + 3 NPIV public I
<truncated output >

NPIV po rt lo gi ns h o w
• Use the portloginshow command to display the login
information for the virtual PIDs of a port
• Following is sample output from the portloginshow command:
swi t ch : aclmi n> portl o g i n s h ow 2

Type PIO World Wide Name credit df sz cos
=====================================================
fe 0102 1 1o : 00 : 00 : 05 : l e : Ob : ef : dl 50 2112 8 scr=Ox 3
fe 0102 ' 25 : c6 : 00 : 0c : 29 : 00 : 0c : 76 50 2112 8 scr=Ox3
fe 0 1 02 25 : c6 : 00 : 0c : 29 : 00 : 0e : 76 50 2112 8 scr=Ox3 FLOG ls
fe 0102 25 : c6 : 00 : 0c : 29 : 00 : 10 : 76 50 2112 8 scr=Ox3
ff 0 1 021 10 : 00 : 00 : 05 : 1e : Ob : ef : dl 0 0 8 d - id=FFFFFC
ff 0102 10 : 00 : 00 : 05 : 1e : Ob : ef : dl 0 0 8 d - id=FFFFFA
ff 0 1 0 21 . 25 : c6 : 00 : 0c : 29 : 00 : 0c : 76 0 0 8 d id=FFFFFC
-
ff OlO j 25 : c6 : 00 : 0c : 29 : 00 : 0c : 76 0 0 8 d - id=FFFFFA
ff 0 1 02 25 : c6 : 00 : 0c : 29 : 00 : 0e : 76 0 0 8 d - id=FFFF FC PLOGls
ff 0102 25 : c6 : 00 : 0c : 29 : 00 : 0e : 76 0 0 8 d - id=FFFFFA
ff 0102 ' 25 : c6 : 00 : 0c : 29 : 00 : 1 0 : 76 0 0 8 d - id=FFFFFC
ff 0102 , I 25 : c6 : 00 : 0c : 29 : 00 : 10 : 76 0 0 8 d - id=FFFFFA
1

Use this command to display port login status received from devices attached to the
specified port.
For each login, this command displays the following fields:
Type - Type of login can display one of the following:
• fd - FDISC, Discover F_Port Service Parameters or Virtual N_Port login.
• fe - FLOGI, Fabric Login to Fabric F_Port.
• ff - PLOGI, Port Login to specific N_Ports or well-known addresses like Name
Server.
PIO - The 24-bit Port ID of the attached device.
WorldWideName - The port's world wide name.
credit - The credit for this login as appropriate. This is BB (buffer-to-buffer) credit
for Flogs and EE (end-to-end) credit for PLOGls.
df_sz - The default frame size for this login.
cos - Class of Services supported. This can be a combination of the following bits:
4 - Class 2 is supported.
8 - Class 3 is supported.
Further information about each login is displayed after these columns, including the
Port ID of the well-known address or N_Port that was the target of the PLOGI, if
applicable.
Revision 01 10 8 - 14
NPIV nsshow
DEV2- ST01- 300:admin > nHhoV
~ Pid cos PorWarM HodeNasie TTL (S.K:)
010200 ; 31 10: 00 : OO: OS : 1~: Ob :e! : d l to:
n::4.3 : r c:P
Po t ~S!r'mb : (291 " Broc•dii- 8.::5 I 1. 1 . 1 I
f abr ic Port Name : 20 : 02 : 0'J : OS :l~ : Oa : 83 : 6d
P•r.anent Port HalM : lO: OoO : OO : OS: le : Ob : e r::d l

Po r t. I ndex: 2
ShAr • Are<.\: Ho
Device Sh<.lred in Ot her AD : Ho
Redirect : No
010201; 3 f s : ctl: OD : Oc : .'.!9: 00: Oc : 7ft s :c6: 00 : Oc : 29: 00 : Ob: 76;
FC43 : FCP
Port.Symb: (281 " Broc;1de- 8.ZS I 1 .1. l I
F.l.bdc Port tl.ima : :o: OZ: OO: OS : l•: Od : 83 : 6d
a;;;~nent Por t u~: 10: 0i0: 00 : 05 : lt!> : Ob : e (:dl I
E lhdiX . ... PWWN of VMs (logical)
Shar e Are(!.: tlo
Device Shar ed in Other AD : No
~~lrect :
010202;
No
3 ; 25 : c6: 00 : Oc : 29: 00: Oe : 76; 25 : c6 : 00 : Oc : 29: 00 : Od: 76;
FC4~ : f CP
Port:Symb: (281 " Brocade- 8.2S t 1.1 . l I I •
Fabdc Port Haft'e : 20: 02 : 0(): 0S :le : Oa : 83 : 6d
Per manen t Port HaJM : 10: 0(): 00 : 05: le : Ob : e f: d l
Po r t: Index: 2
Shat:• Ar•": Uo
Device Shared in Ot her AD : Ho
Redirect : No
<truncated output>

BROCADE
EDUCATION SOLUTIONS
Access Gateway - Overview

• Access Gateway (AG) is a Fabric OS feature that enables:
- Seamless connectivity to a fabric
- Enhanced scalability
- Simplified manageability
• Designed to connect numerous servers with minimal impact to any
existing fabric
• Focus is connectivity, bandwidth is shared
• Included in the base Fabric OS - no separate license is required
• Attached F_ Port devices must be Fibre Channel Protocol (FCP)
initiators or targets
- Not supported: loop devices, FICON , virtual iSCSI initiators

The Brocade Access Gateway is a Fabric OS feature (no license required) that lets
you configure your Enterprise fabric to handle additional N_Ports instead of
domains.
Switches in AG mode are logically transparent to the host and the fabric. It
increases the number of hosts to have access to the fabric without increasing the
number of switches in the fabric. This simplifies configuration and management in a
large fabric by reducing the number of domain IDs and ports.
Revision 01 10 8 - 17
Access Gateway - Overview (cont.)

• Access Gateway ports can be configured as N_Ports, which
connect to the edge fabric
• No change in domain count - improves the scalability of the fabric
• Hosts/HBAs are mapped (through NPIV) to the N_Ports, and
connect to the edge fabric through the N_Ports
• No fabric management or zoning on the Access Gateway
Hosts
Fabric
• N Port
.'i F Port
[ ·: :· F_Port (with NPIVenabled)

The Brocade Access Gateway allows multiple Host Bus Adapters (HBAs) to access
the fabric using fewer physical ports. Instead, certain Access Gateway ports are
configured as N_Ports, with the attached hosts mapped through the N_Ports via the
N_Port ID Virtualization (NPIV) protocol. The Brocade Access Gateway is a device
management tool and provides only a subset of Fabric OS commands. Therefore it
does not consume critical fabric elements (e.g. domain IDs) that could inhibit
scalability.
The Access Gateway feature was introduced in FOS v5.3
Supported Platforms
• Supported on the Brocade 300, 5100, VA-40FC 1 and embedded
blade server switches2
~~ .i!~;~ :~.·~~:,::~~:':'.·:·::~::~:~~~~[~~:::•;,~
Brocade 300 Brocade 51 00
Brocade VA-40FC Brocade 4xxx and 5xxx Series

Embedded Blade SelfVer Switches
• The VA-40FC is a compact form factor version of the 5100 which

supports high-density server racks and also provides reversible
airflow options (either port-side or power-side exhaust)

Al 1R ghts Reserved
Footnote 1: the VA-40FC ships with AG mode enabled by default but AG mode can
be disabled and put back into Fabric mode
Footnote 2: Current embedded switches:
8 Gbps Switches
Dell M5424 24-port for Dell PowerEdge M1 OOOe
Fujitsu 5450 26-port for PRIMERGY BX900
Hitachi 5460 26-port for BladeSymphony BS2000
HP 5480 24-port for B ladeSystem c-Class
HP 5481 24-port HP Virtual Connect for Bladesystem c-Class
HP 5410 12-port for EVA 4400-S (storage switch)
IBM 5470 20-port for BladeCenter
4 Gbps Switches
Dell 4424 24-port torr PowerEdge M1 OOOe
Fujitsu 4016 16-port for PRIMERGY BX600
Hitachi 4016 16-port for 81000 and B ladeSymphony BS320
HP 4024 24-port for B ladeSystem c-class
HuaWei 4018 18-port Embedded Switch
IBM 4020 20-port for BladeCenter
NEC 4024 24-port torr SigmaBlade
AG Provides Scalability
• Multiple F_ Ports on an AG are mapped to a single N_Port on the
same AG
• Several N Ports on an AG can be connected to a fabric
• Every connection from an AG to a fabric can support a maximum of
255 devices, providing scalability for device attachment
e F_Port
e N_Port
Fabric

Revision 0110 8- 20
Access Gateway Use Cases

• AG works well in large server count environments where
management of multiple fabric domains is increasingly complex
and limiting
• Mixed-vendor SAN configurations can utilize their full capabilities,
without the restrictions of interoperability modes
Domain ID
5 Access Gateway
Domain ID
36 DomainlOs Domain IDs
1-4 1-4
Tradition al Brocade Blade server SAN Switches Brocade Blade Selvef SAN Switches in Access
Attached to SAN Fabric = 36 Domain IDs Gateway Mode Attached to SAN Fabric = 4 Domain IDs

Access Gateway- Details (cont.)

• When a switch is configured as an Access Gateway, the following
are some of the features that are disabled:
- Advanced Performance Monitoring - Management Services
- Extended Fabrics - Name Services (SNS)
- FICON (includes CUP) - Port mirroring
- IP over FC - SMl-S
- Zoning
These features are on ~y disabled on the AG, but are still available in
the rest of the fabric.
• All switches must have all available POD licenses installed before
configuring a switch for AG mode

Revision 0110 8- 22
BROCADE
EDUCATION SOLUTIONS
Revision 0110 8- 23
Default Port Mapping

300
Access
• Access Gateway uses a port map to Gatewa
PortO (.~ t
direct traffic from host HBAs to the I.-------•
NI Ports that connect to the fabric Port 1 J:
Port 2 ...._______
I Port 16
Each F_Port has a Primary N_Port that is J, - - - - -• Port 17

used to connect to the fabric
The port map and N_Port configuration
can be changed
::J> I
Port S '. • Port 18
• Enabling Access Gateway enables the

Port 6 Jl.:.-____ 1
- - -· Port 19
Port 7 ( ;,---
default port group. In this 8300 example:
N Ports: 16 - 23
••
Two F_Ports are mapped to each N_Port
•
Port14 ('.2
J • Port23
Port1s •::- 1
• N_Port
1: '.I F_Port
(- Mapped Online

Brocade Total Parts F_Por!s N_par!s Default F_ 111 N_Part Mapplnc

Madel
300 24 ().15 16 ·23 0. 1 mapped to 16

2. 3 mapped to 17
4. 5 mapped to 18
6. 7 mapped to 19
8. 9 mapped to 20
10. 11 mapped to 21
12. 13 mapped to 22
14, 15mapped to 23
200E 16 0-11 12·15 O. 1. 2 mapped to 12

3. 4. 5 mapped to 13
9. 10. 11 mapped to 15
4012 12 0- 7 8-11 o. 1 mapped to 8

2. 3 mapped to 9
4. 5 mapped to 10
6. 7 mapped to 11
4016 16 0-9 10-15 0. 1 mapped to 10

2. 3 mapped to 11
4. 5 mapped to 12
6. 7 mapped to 13
8 mapped to 14
9 mapped to 15
4018 18 4-11 0-3 4. 5. 12 mapped to 0

8. 9, 14. 16 mapped to 2
10. 11. 15. 17 mapped to 3
4020 20 1- 14 0.15- 19 1. 2 mapped to 0
3. 4 mapped to 15
8. 9 mapped to port 17
10. 11 mapped to 18
12. 13. 14 mapped to 19
Revision 0110 8- 24
Brocade Total Ports F_POO.s N_Ports Defa.Ult F_to N_Port Mapping

Model
4024 24 1-16 0, 17-23 1, 2 mapped to 17

9, 10 mapped t o 18
3, 4 mapped to 19
11, 12 mapped to 20
5, 6 mapped to 21
13, 14 mapped to 22
7, 8 mapped to 23
15, 16 mapped to o
4424 24 1-16 0, 17-23 0, 17-23

1, 2 mapped to 17
3, 4 mapped to 18
5, 6 mapped to 19
7, 8 mapped to 20
9 , 10 mapped to 21
11, 12 mapped to 22
13, 14 mapped to 23
15, 16 mapped to O
5100 40 0-31 32-39 O, 1, 2, 3 mapped to 32

4, 5, 6, 7 mapped to 33
8, 9, 10, 11 mapped to 34
12, 13, 14, 15 mapped to 35
16, 17, 18, 19' mapped to 36
20, 21. 22, 23 mapped to 37
24, 25, 26, 27· mapped to 28
28, 29, 30, 31 mappedto39
5424 24 1-16 0 , 17-23 0, 17-23

1, 2 mapped to 17
3, 4 mapped to 18
5, 6 mapped to 19
7, 8 mapped to 20
9, 10 mapped to 21
11, 12 mapped to 22
13, 14 mapped to 23
15, 16 mapped to o
5450 26 ~25 0, 19-25 1, 2, 17 mapped to 19

Not all ports 3, 4, 18 mapped to 20
may be 5, 6 mapped to 21
aooessible_ 7, 8 mapped to 22
9, 10 mapped to 23
11, 12 mapped to 24
13, 14 mapped to 25
15, 16 mapped to o
Revision 0110 8- 25
Brocade Total P1i>rts F_Ports N_Ports Default F_to N_Port Mapping

ModeJ
5470 20 1-14 0 , 15-19 O, 15-19 are N_ pOrts witt1 failover

enabled, tailback enabled and JDG
p0licy
1, 2 m apped too
3, 4 mapped to 15
5, 6, 7 mapped to 16
8, 9 mapped to 17
10, 11 mapped to 18
12, 13, 14 mapped to 19
5480 24 1-16 0 , 17-23 0, 17-23 are N__ports wtth failOver

enabled, tailback enabled and JDG
policy
1, 2 mapped to 17
9, 10 mapped to 18
3, 4 mapped to 19
11, 12 mapped to 20
15, 16 mapped to O
5, 6 mapped to 21
13, 14 mapped to 22
7, 8 mapped to 23
Revision 0110 8- 26
Access Gateway- Port Mapping (cont.)

• Access Gateway uses NPIV to
assign the 24-bit FC address
based on the port map
- F_Ports (devices) share the same
domain and area values as the
N_Ports to which they are mapped
- The last byte is assigned in the
order in which the devices login to
the fabric
• Example:
- Port 20 address =040500
- Host 5 address = 040501
- Host 6 address= 040502

NPIV must be enabled on the switch that the Access Gateway is connected to.
The example above shows 8 hosts (Host_1 through Host_8) connected to a Brocade 300. It
is configured as an Access Gateway with the default port map. The hosts are mapped to
the N_Ports as follows:
• F_Port O (Host_1) and F_Port 1 (Host_2) are mapped to N_Port 16
• F_Port 4 (Host_3) and F_Port 5 (Host_4) are mapped to N_Port 18
The FC addresses associated with the N_Ports and F_Ports (attached devices) are:
• The FC address for N_Port 16 = 030000. F_Port 0 (Host_1) logs in first and receives
the address 030001; F_Port 1 (Host_2) logs in second and receives the address
030002.
030102.
040502.
the address 040601 ; F_Port 13 (Host_8) logs in second and receives the address
040602.
Access Gateway- Initialization

• Access Gateway is logically transparent to the fabric, so the
initialization of an Access Gateway occurs in stages:
1 . N Ports come online first
2. F_Ports are mapped to N_ Ports according to the port map
3. If the N_ Port does not come onlline and failover is enabled, F_Ports
are remapped
4. F Ports are enabled
5. Hosts log in to the fabric

Revision 0110 8- 28
Access Gateway- Device Login

1. The N_Port of the AG logs into the fabric using a FLOGI
2. Fabric responds with a FLOGI ACC
3. Host issues a FLOGI request to the Access Gateway F_Port to which it is
attached
4. Access Gateway passes the Fabric Lo91in request to the N_Port to which the
F_Port is mapped and the N_Port transforms the FLOGI request to a Fabric
Discovery (FDISC) 1 request, which is transmitted to the fabric
5. Fabric Discovery Accept (FDISC ACC) response is received from the fabric
6. F_Port transforms the FDISC ACC response to a Fabric Login Accept (FLOGI
ACC) response, which is transmitted to the host
e F_ Port - 0 FLOGI
e N_ Port FLOGIACC
- -tUt--FLOGI- -- -- ----• ---FOISC-----------..

- ---LOGIACC • AG ..--------- FOISCACC·-· -----
Servers Fabric

Footnote 1: A Fabric Discovery (FDISC) is typically used when an initiator receives

an RSCN and wants to determine the current state of the target that caused the
RSCN. If the current login session to the target is valid, the initiator can continue
operations with affected target; otherwise, the initiator attempts to re-login. If login to
the affected port is not possible, the initiator can implicitly log out of the affected
port, thereby freeing resources. The FDISC process allows F_Ports to log in targets
and establish sessions.
Revision 0110 8- 29
Access Gateway- N_Port Failover

• If an N_ Port goes offline, the
Access Gateway N_Port failover
policy allows hosts to be
automatically remapped to
another online N Port
Host_3
- F Ports connected to failed 030101
N_Port are evenly distributed Host 4

across N Ports connected to the 03010 2
same fabric Host 5

040501
- F Ports receive a new FC address Swilch_B
Host_6 Domain 10 = 4
based on the new N Port 040502
- Enforced at N Port initialization as

well (cold failover)
- The default configuration requires
all N Ports to be connected to the
same fabric
- Enabled by default; managed on a
per-N_Port basis

The following sequence describes how a failover event occurs:

1. An N_Port goes offline (ISL failure, attached switch port or switch goes offline).
2. All F_Ports mapped to that N_Port are disabled.
3. If the N_Port failover policy is enabled, the disabled F_Ports are remapped to
an online N_Port. The F_Ports are evenly distributed among the remaining
online N Ports.
4. The F Port is re-enabled on the new N Port.
5. The host establishes a new connection with the fabric.
Cold failover: After the switch comes online, the N_Port must successfully log in to
the attached fabric within 6 seconds, or the mapped F_Ports are remapped to the
remaining N_Ports. An unsuccessful login can occur because the N_Port is not
connected to the enterprise fabric; NPIV is not enabled on the fabric port to which a
N_Port is connected; or the Fabric Login request from N_Port is rejected by
enterprise fabric with a reason other than "LS_LOGICAL_BUSY".
In the example above, port 0 on Switch_A goes offline (highlighted in red). This
causes N_Port 16 on the Access Gateway to go offline. The failover policy now
goes into effect, causing F_Port 0 to be remapped to N_ Port 18, and F_Port 1 to be
remapped to N_Port 20. After the devices attached to these F_Ports re-login to the
=
fabric, the devices obtain new FC addresses (Host_ 1 030103, Host_2 040503). =
Revision 0110 8- 30
Access Gateway- N_Port Failback

• If an N_ Port comes back online,
the Access Gateway N_Port
tailback policy automatically
remaps F_Ports back to the
originally mapped Primary Host_3
030101
NI Port
Hosl_4
- Only the originally mapped 030102
F Ports fail back Hosl_S
040501
- With multiple N_ Port failures, only
Switch B
F_Ports that were mapp,ed to the Hos1_s Domain 10 = 4
040502
recovered N_Port experience
failback1 Host_7
040601
- Failed-back F Ports return to
Host_8
their original FC address 040602
- Enabled by default, can be e N Port - Mapped Online

changed ; managed on a per- ('; F -P
.. _ort _• • • Fallover Route Offt1ne
N Port basis [ ·: :· F_ Port (With NPIV enabled)
C 2009 Brocade Commun cal ans Systems nc 31

Al 1R ghts Reserved
Footnote 1: The Access Gateway remaps only those F_Ports that were originally
mapped to the recovered N_Ports. F_Ports mapped to still-failed N_ Ports remain
remapped.
Access Gateway- Implementation

• In the example that is used in the next several slides, a Brocade
300 is deployed as an Access Gateway
- Hosts are attached to ports 1, 2, 5, and 6 of the AG
- N_Ports 16, 17, 18 and 19 are connected to ports 0, 1, 2 and 3 on a
Brocade 5100
I Ports 16-19 1
0 N_Port
0 F_Port

This example is used to demonstrate the commands issued to put a Brocade 300 in
Access Gateway mode and show various command outputs. It is not intended to be
a "best practice" example.
Revision 0110 8- 32
Access Gateway - Enabling

• Prepare by uploading the current switch configuration
B300 : admin> configupload <truncated output>

B300 : admin> switchdisable
B300 : admin> ag --modeenable

WARNING : Access Gateway mode c hanges the standard behavior of
the switch . Please chec k Access Gateway Admi n i s t rator ' s Gu i de
before proceeding . En abling agmode will remove all the
conf i guration data o n t he sw i tch i nc l uding zoni ng
conf i guration and secu rity database . Please bac kup your
configuration using configupload .
This operation wil l reboot the switch .
Do you want to continue? (yes , y , no , n) : [no] y
Access Gateway mode was enabled successfu lly
Switch is being rebooted ...

The ag --modeenable command enables the Access Gateway mode on the

Brocade 300. The switch must be in a disabled state to run this command. If the
platform does not support the command, the command output is:
Access Gat eway mode is not supported on t h is p lat f orm .
In the example above, the Brocade 300 uploads the switch configuration, disables
the switch, and then enables the Access Gateway mode.
Revision 0110 8- 33
Access Gateway- Enabling (cont.)

• After the switch reboots, verify the Access Gateway mode with the
ag --mo d e show command
• Remember, many Fabric OS commands and features are no
longer available
B300 : admin> ag --modeshow

Access Gateway mode is enabled
B300 : admin> cfgshow

Erro r : This command i s not supported in AG mode

The ag --modeshow command displays the Access Gateway mode on the

Brocade 300. The command has no arguments.
Revision 0110 8- 34
Access Gateway - Default Port Map

• Review the mapped ports with the ag --mapshow command
B300 : aclmin> ag --mapshow

N_Port Configured_F_Ports Current F Ports Failover Failback PG_ID PG_Name
16 0; 1 1 1 1 0 pgO
17 2; 3 2 1 1 0 pgO
18 4; 5 5 1 1 0 pgO
19 6; 7 6 1 1 0 pgO
20 8; 9 None 1 1 0 pgO
21 10; 11 None 1 1 0 pgO
22 12 ; 13 None 1 1 0 pgO
23 14 ; 15 None 1 1 0 pgO

The ag --mapshow command displays the mapping between the N_Ports and the
F_Ports. Each line displays the following information:
• N_Port - Port numbers of ports locked in N_Port mode.
• Configured F_Ports - List of F_Ports that are mapped to the corresponding
N Port.
• Current F_Ports - Shows the F_Ports that are currently conne-cted to the
fabric on the corresponding N_Port. In the case of failover, the Current F_Ports
and Configured F_ Ports differ.
• Failover and Failback - Indicates whether or not N_Port policy is enabled (1)
or disabled (0).
W hen the Access Gateway mode is enabled, all N_Ports belong to the default port
group, pgO.
Revision 0110 8- 35
Access Gateway- Default Port Map (cont.)

• Display detailed configuration information with the a g -- sho w
command
B300 : admin> ag --show

Name 8300
NodeName 10 : 00 : 00 : 05 : 1e : Ob : a0 : 5f
Number of Ports 24
IP Address(es) 10 . 255 . 240 . 105
Firmware Version v6 . 3 . 0
N Ports 4
F Ports 4
Policies enabled pg
Persistent ALPA Disab l ed
Port Group information :
PG ID PG Members PG Name PG Mode
0 16 ; 17 ; 18 ; 19 ; 20 ; pgO
21 ; 22 ; 23
<output continued on n·e xt sl.ide>

The ag --show command displays detailed information about the N_Ports and
F_Ports, including the FC address of the devices. General information includes the
switch name, the switch WWN, the number of switch ports (2 4), the IP address, the
firmware revision , the number of N_Ports ( 4) and current F_Ports ( 4 ) and the
policies enabled (pg). Port Group 0 is the default port group.
Revision 01 10 8 - 36

<output continued from previous slide> Brocade
5100
Fabric Information :
ports
Attached Fabric Name N Ports attached
to 300
10 : 00 : 00 : 05 : 1e : 9c : b1 : 73 16 ; 17 ; 18 ; 19
N Port i nformation :
Port Port I D Attached PWWN FO FB F Ports
16 Ox010000 20 : 00 : 00 : 05 : 1e : 9c : b1 : 73 1 1 1 0 . 255 . 240 . 106 1;

17 Ox010100 20 : 01 : 00 : 05 : 1e : 9c : bl : 73 1 1 1 0 . 255 . 240 . 106 2;
18 Ox010200 20 : 02 : 00 : 05 : 1e : 9c : b1 : 73 1 1 10 . 255 . 240 . 106 5;
19 Ox010300 20 : 03 : 00 : 05 : 1e : 9c : b1:73 1 1 10 . 255 . 240 . 106 6;
F Port information :
Port Port ID Attached PWWN N Port Preferred N_ port Login Exceeded?
1 Ox010001 10 : 00 : 00 : 05 : le : 57 : 7c : 7d 16 None No
2 Ox010101 10 : 00 : 00 : 05 : 1e : 57 : 7c :ab 17 None No
5 Ox010201 10 : 00 : 00 : 05 : le : 56 : c8 : 39 18 None No
6 Ox010301 10 : 00 : 00 : 05 : 1e : 56 : c8 : 35 19 None No
---------------------------------------------------------------------
HBAs
attached
to 300

Use the N Port informat ion to review:

• The N_Ports are ports 16, 17, 18, and 19.
• By reviewing the N_Port FC addresses (PortID), the domain ID of the
Brocade 5100 is 1 (OxOl ), and the switch ports used on the Brocade 5100 are
ports 0 (OxO l OO ), 1 (Ox0 1 0 1 ), 2 (Ox0102 ), and 3 (Ox0103 ).
• All N_Ports have failover (FO) and tailback (FB) enabled (1 ).
• The per-port N_ Port WWN (Port WWN ) is 20: PP: 00: 05 : le : 04: 25 : 4 5 ,
where PP is the Brocade 5100 port to which the N_ Port is attached, and
oo: os : le : o 4 : 2 s : 4 s is the last 6 bytes of the Brocade 5100 switch WWN.
• The IP_Add r value is 1 o . 255 . 2 4 4 . 91, indicating that all the N_Ports are
attached to the same Brocade 5100.
Use the F_Port information to cross-check the port map:
• F_Port port 1 (Ox010001 ) mapped to N_ Port port 16
• F_Port port 2 (Ox010101 ) mapped to N_ Port port 17
• F_Port port 5 (Oxo10201 ) mapped to N_Port port 18
• F_Port port 6 (Ox010301 ) mapped to N_Port port 19
Revision 01 10 8 - 37

• The swi t c hsho w command adds the F Port and N Port FC addresses
B300 : admi n> swi tch show
<truncated output>
switchMode : Access Gateway Mode
switchWwn : 10 : 00 : 00 : 05 : 1e : Ob : a0 : 5f
switchBeacon : OFF
Index Port Addr ess Media Speed State Proto
0 0 030000 id N4 No_Light FC
1 1 030100 id NB Online FC F- Port 10 : 00 : 00 : 05 : 1e : 57 : 7c : 7d Ox010001
2 2 030200 id N4 Online FC F-Port 10 : 00 : 00 : 0S : le : S7 : 7c : ab Ox0101 01
<truncated output>
5 5 030500 id N4 Online FC F-Port 10 : 00 : 00 : 05 : 1e : 56 : c8 : 39 Ox010201
6 6 030600 id N4 Online FC F-Port 10 : 00 : 00 : 05 : le : 56 : cS : 35 Ox010301
<truncated output>
16 16 031000 id NS Online FC N-Port l0 : 00 : 00 : 05 : le : 9c : bl : 73 OxOlOOOO (AoQ)
17 17 031100 id N4 Onl i ne FC N- Port 10 : 00 : 00 : 05 : 1e : 9c : b1 : 73 Ox010100 (AoQ)
18 lS 031200 id N4 Online FC N- Port 10 : 00 : 00 : 05 : 1e : 9c : bl : 73 Ox010200 (AoQ)
19 19 031300 id NS Online FC N-Port 10 : 00 : 00 : 05 : 1e : 9c : bl : 73 Ox010300 (AoQ)
< truncated output>

The swi tchshow command adds the NPIV-generated FC address for the F_ Ports
and N_ Ports. Use these addresses to verify the port map:
• F_Port port 1 (OxOlOOOl ) mapped to N_ Port port 16 ( Ox 5aOOOO )
• F_Port port 2 (Ox010101 ) mapped to N_Port port 17 (Ox5a 0100 )
• F_Port port 5 (Ox010201 ) mapped to N_ Port port 18 (Ox5a0200 )
• F_Port port 6 (Ox010301 ) mapped to N_ Port port 19 (Ox5a 0 30 0 )
AoQ = Application oriented QoS. Indicates that the link is capable of QoS. It should
appear on F and N_Ports on switches that are connected to either Brocade HBAs
with an active SAO license or Access Gateways that have a Adaptive Networking
license.
Revision 01 10 8 - 38

• On the Brocade 5100 switch that is attached to the Access
Gateway, the swi tchshow command displays details about the
N Ports
Number of
B5100 : admin> switchshow online hosts
switchName : 85100 from 300, +1 for
switchWwn : 10 : 00 : 00 : 05 : le : 9c : bl : 73 300 N_Port
<t runcated output>
0 0 010000 id NS Online FC F- Port 1 N Port + 1 NPIV public (AoQ)

1 1 010100 id N4 Online FC F-Port 1 N Port + 1 NPIV public (AoQ)
2 2 010200 id N4 Online FC F-Port 1 N Port + 1 NPIV public (AoQ)
3 3 010300 id NS Online FC F-Port 1 N Port + 1 NPIV public (AoQ)

The swi tchshow command on the Brocade 5100 shows:

• Ports 0, 1, 2 and 3 are connected to the Brocade 300 in Access Gateway
mode.
• The number of NPIV public devices is. the number of online hosts on the 300,
plus 1 for the N_Port connecting the 300 to the 5100.
Revision 01 10 8 - 39
Access Gateway - Changing the Port Map

• Mapped F_ Ports must be deleted before they can be added to
a nether N Port
• Delete F_Ports 1 and 2 from currently mapped N_ Ports:
B300 : admin> ag --mapdel 16 1
F-Port to N-Port mapping has been updated successfully
F-Port to N-Port mapping has been updated successfu l ly
• Add F Ports 1 and 2 to N Port 19:

B300 : admin> ag --mapadd 19 "1;2" 1
F- Port to N- Port mapping has been updated successfully
• An unconfigured N_ Port can set the F-to-N_ Port mapping with the
ag --mapse t command

The ag --mapdel command deletes one or more F_Ports from a specific N_Port on
an Access Gateway. The command has two arguments:
• N- Port: The specific N- Port.
• F_Port _ lis t : A semicolon-separated list of F_Ports whose mappings are
being remov,ed from the N_Port. When specifying multiple F_Ports, use '"' to
surround the semi-colon separated list; for a single F_Port, '"' marks are not
required. To identify ports 2, 4, 5, and 6, specify an F_Port list of "2; 4- 6".
The ag --mapadd command adds one or more F_Ports to a specific N_Port on an
Access Gateway. The command arguments are the same as with ag --mapdel.
The ag --mapset command creates an N-to-F_ Port mapping on an unconfigured
N_Port on an Access Gateway. The command arguments are the same as with ag
--mapdel and ag --mapadd.
Footnote 1: When trying to map more then one port at a time, if one port fails to
map .all the ports will fail to map. If a port already has a mapping, it must be
unmapped before it can be remapped with a mapadd or the new mapping process
will fail.
Revision 01 10 8 - 40
Access Gateway- Changing the Port Map (cont.)

• Review the updated port mapping
B300 : admin> a g --mapshow

N_Port Configured_F_Ports Current F Ports Failover Failback PG_ID PG_Name
16 0 None 1 1 0 pgO
17 3 None 1 1 0 pgO
18 4; 5 5 1 1 0 pgO
19 1; 2 ; 6; 7 1 ;2; 6 1 1 0 pgO
20 8; 9 None 1 1 0 pgO
21 10; 11 None 1 1 0 pgO
22 12 ; 13 None 1 1 0 pgO
23 14 ; 15 None 1 1 0 pgO

In the example above, the Access Gateway port map is displayed. The default port
map has been altered so that F_Ports 1 and 2 are now mapped to N_Port 19.
Revision 01 10 8 - 41
Access Gateway - Failover and Failback

• Failover can be displayed, enabled , and disabled per N_Port:
ag -- failovershow [N_Port ]
ag -- f ailoverenabl e <N Port>
ag -- f ailoverdisable <N Port>
• Failback can be displayed, enabled , and disabled per N_Port:

ag -- failbackshow [N Port]
ag -- f ailbackenabl e <N Port>
ag -- f ailbackdisable <N Port>

For all the commands above, the only argument is the N_Port on which failover or
tailback is to be displayed, enabled, or disabled. For the show commands, omit the
port argument to display the settings for all N_Ports.
Revision 01 10 8 - 42
Access Gateway - N_Port Management

• Display the current N_Ports with the port cf gnport command 1
B300 : admin> portcfgnport

Ports 0 1 2 3 4 5 6 7 8 9 1 0 11 12 13 14 15
--------------------+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--
Locked N Port
1 6 17 18 19 20 21 22 23
+--+--+--+--+--+--+--+--
ON ON ON ON ON ON ON ON

Footnote 1: The port cf gnport command is used to monitor and manage

N_Ports on an Access Gateway. The only configuration parameters for an Access
Gateway N_Port are port speed, persistent disable, and NPIV enabl,e .
Revision 0110 8- 43
Access Gateway- N_Port Management (cont.)

• Disable a port as an N_ Port with port cf gnport <port> O
• Enable a port as an N_ Port with portcfgnport <port> 1
• If the port is currently mapped as an F_Port, it first must be
deleted 1
• Once the N_ Port is enabled, F_ Ports should be mapped to the new
N_ Port using the ag --mapadd command

B300 : admin> portcfgnport 15 1
B300 : admin> portcfgnport
Ports 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
--------------------+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--
Locked N_Port . . . . . . . . . . . . ON
16 17 18 19 20 21 22 23
+--+--+--+--+--+--+--+--
ON ON ON ON ON ON ON ON

Footnote 1: Before configuring port 15 as an N_Port, it must be deleted from the

existing port map. Port 15 is currently mapped to N_Port 23.
Revision 0110 8- 44
Access Gateway - Disabliing

• Before disabling Access Gateway mode, first upload the switch
configuration, then disable the switch
B300 : admin> configupload <truncated output>

B300 : admin> ag --rnodedisable
WARNING : Disabling agmode will remove all the configuration data on
the switch including N_Port configuration and F Port to N Port
mapping . Please backup your configuration using configupload .
This operation will reboot the switch .
Do you want to continue? (yes , y, no , n) : [no ] y
Access Gateway mode disabled successfully
Switch is now rebooting...

The ag --modedisable command disables Access Gateway mode. The switch

must be in a disabled state to run this command. The command has no arguments.
Revision 0110 8- 45
Access Gateway- Disabliing (cont. )

• After the switch reboots, restore the switch configuration by
downloading the pre-Access Gateway switch configuration file
B300 : admin> switchdisab1e
B300 : admin> configdown1oad <truncated output>
B300 : admin> switchenab1e
• Pre-Access Gateway state of the Brocade 300 is now restored

Revision 0110 8- 46
BROCADE
EDUCATION SOLUTIONS
Revision 0110 8- 47
Access Gateway- Policies

• Port Grouping (PG)
- Limits N_Port failover to occur only within a user-defined group of ports
- Enabled by default
• Auto Port Configuration (APC) 1
- Automatically detects F_ Ports and N_ Ports
- Automatically maps the ports
- Disabled by default
• Advanced Device Security (ADS)
- Discussed in Security module
• Each policy must be in an enabled state before it can be used

The details for each policy will be presented later in this module.
Footnote 1 : The Brocade documentation refers to the Auto Port Configuration
policy as ,,APC' even though the command output shows "auto". Auto Port
Configuration is discussed in the appendix of this module.
Revision 0110 8- 48
Access Gateway- Policies (cont.)

• Use the ag --policyenable/--policydisable command to
enable/disable a policy (pg , auto , ads )
• Use the ag --po l icyshow command to display policy states
B300 : admin> ag --policyshow

AG Policy Policy Name State
Port Grouping pg Enabled

Auto Port Configuration auto Disabled
Advanced Device Security ads Disabled

A switch in Access Gateway mode supports three policies:

1. Port Grouping
2. Auto Port Configuration
3. Advanced Device Security
The policy must be enabled before it can be used. Port Grouping is enabled by
default; Auto Port Configuration and Advanced Device Security are disabled by
default.
Revision 0110 8- 49
Access Gateway - Port Grouping

300 Fabric 1
• Port Grouping limits N_ Port Access
Switch_A
Port O Gatewa Port 1& Port O
.•
Host_l Doma1nlD = 3
failover to occur only within a 030001 -- - - -. 030000
Port 1 ., - - - - -
user-defined group of ports Hos1..2 .
030002
- Allows Access Gateway to be Host_3 Port4
attached to multiple edge 040401
fabrics Host_4
040402
•
- Default port group is enabled :·tios1=s1
by default and requires all : 0602ill
Ports , • :·POri 20 •
·~
060200 1 ·> .
·P 2
Fabric 2
N Ports to be connected to
the same fabric
•• • ••• ,
: Host 6 J
' 060202
·-
Port9 •
Host_7 l°"I' ~ l_~_g_,.>

•
• Port 22
.
ort
..
·..•
Switch_C
Domain 10 = 6
L J
030701 ~- . 030700 Switch_D
- User-defined port groups Port 13 •• ~ Domain ID= 3
0
Host_8 Port 7
must be created to attach to 030702 ....
more than one fabric ~ N_Port · Group 1 - Mapped Online ••
l • N_Port - Group2
() F_Port
~
l ·::• F_Port (with NPIV enabled)

Al 1R ghts Reserved
In the example above, the Access Gateway is attached to two fabrics. Ports 16 and
18 are attached to Fabric 1 and ports 20 and 22 are attached to Fabric 2. To ensure
the F_Ports mapped to ports 16 and 18 failover only within Fabric 1, ports 16 and
18 are put into a Port Group (Group 1). Siimilarly, to ensure the F_Ports mapped to
ports 20 and 22 failover only within Fabric 2, ports 20 and 22 are put into a separate
Port Group (Group 2).
Revision 0110 8- 50
Access Gateway- Port Grouping (cont.)

300 Fabric 1
• NI Port failover and failback Access
Switch_A
Domain ID= 3
occur only within a Port Group
• If all N_ Ports in a Port Group go
offline, the mapped F_ Ports are
disabled
• Up to 15 Port Groups per Access
Gateway Fabric 2
Sw1tch1_C
Oomainl0=6

Al 1R ghts Reserved
In the example above, we see the result of N_Port failover with Port Groups:
• If port 16 fails, F_ Ports 0 and 1 failover to Port 18 - the only remaining port in
Group 1. Without port groups, one of the hosts would have failed over to port
20 or 22, which is not attached to Fabric 1. After the failover, the Pl Os of the
attached hosts are updated (Host_ 1 changes from Ox030001 to Ox040403;
Host_2 changes from Ox030002 to Ox040404).
• If port 22 fails, F_Ports 12 and 13 failover to Port 20 - the only remaining port
in Group 2. Without port groups, one of the hosts would have failed over to port
16 or 18, which is not attached to Fabric 2. After the failover, the PIDs of the
attached hosts are updated (Host_? changes from Ox030701 to Ox060203;
Host_8 changes from Ox030702 to Ox060204).
Revision 01 10 8 - 51

• Use the ag --pgshow command to display Port Group
information
• Default setting matches the default port map behavior:
- Port grouping is enabled by default
- All N_ Ports are initially part of the default Port Group (pg o)
- Ports can be added or removed from the default Port Group
- Default Port Group cannot be removed or renamed
Brocade 300 default port group

B300 : adrnin> ag --pgshow
PG ID PG Members PG Name
0 16 ; 17 ; 18 ; 19 ; 20 ; pgO
21;22 ; 23

The command output above shows the default Port Group (pgO) on a Brocade 300
with the default port map in place.
Revision 0110 8- 52

• Use the ag --pgcreate command to create a Port Group
B300 : admin> ag --pgcreate 1 " 16 ; 18" -n Groupl
Port Group 1 created successfully
B300 : admin> ag --pgcreate 2 "20;22" -n Group2

Port Group 2 created successfully
B300 : admin> ag --pgshow

PG ID PG Members PG Name
0 17 ; 19 ; 2 1 ; 23 pgO
1 16 ; 18 Groupl
2 20 ; 22 Group2

Any N_Ports not put in a user-defined Port Group remain in the default Port Group
(pgO ).
Revision 0110 8- 53
Access Gateway - Preferred N_Port

• Each F_ Port can identify a Preferred N_ Port for failover
- Acts as an alternative N_Port if the mapped N_ Port (Primary N_Port)
fails
- Allows users to know exactly which N_ Ports will be used - and thus,
what the Fibre Channel Physical ID (PIO) will be
• Default: An F_Port has a Primary N_Port, but no Preferred N_Port

Preferred N-Port provides an alternate N_ Port for F_Ports to failover to. The
F_Ports must have a primary N_Port mapping before a preferred (secondary)
N_Port can be configured.
Revision 0110 8- 54
Access Gateway- Preferred N_Port (cont.)

• When the Primary N_ Port fails, the affected F_ Ports are
automatically failed over to the Preferred N_ Port
• If you specify a Preferred N_ Port, failover only occurs to that port

- If the Preferred N_ Port is offline, the F_Port goes offline as well
• Preferred N_ Ports work with Port Groups:

- Preferred N_Port must be part of the same Port Group as the Primary
N Port
- If there is no Preferred N_ Port defined, the F_ Ports failover across the
N_Ports w ithin their Port Group (as discussed earlier)

Revision 0110 8- 55

• Display the Preferred N_ Port for an F_Port with the
ag --pref show command
• Default setting: All F_ Ports have a Primary N_Port, but no

Preferred N Port
B300 : admin> ag --prefshow

F Ports Preferred N Port
Preferred N_Port is not set for any F_Port

The ag --p r ef show command displays the current Preferred N_IPorts on the
Access Gateway. The command has no arguments.
There are currently no Preferred N_Ports - the default setting.
Revision 01 10 8 - 56

• Set the Preferred N_Port for an F_ Port with the ag --pref se t
command
B300 : admin> ag --prefset "0; 1 " 20

Pr eferr ed N Port i s set successfully for the F_Ports[s]
B300 : admin> ag --prefshow

F Ports Preferred N Port
0;1 20

The ag --p r e f set command sets the Preferred N Port for an F Port.
Revision 0110 8- 57
BROCADE
EDUCATION SOLUTIONS
Revision 0110 8- 58
Access Gateway - F_Port Trunking

• Trunking aggregates the bandwidth of the ports within the trunk
group
• Available in Fabric OS v6.1 .0 or later
• Needs to be configured as trunks do not automatically form
• Has the same requirements as ISL Trunking
- Trunking license on both AG and edge fabric switch
- Port group to port group
- Same speed 300
Access Gateway Edge Switch

N_Port/F_ Port Trunking provides the same benefits that ISL Trunking does in the
fabric. It aggregates the bandwidth of each link within the Trunk. Each link has an
N_Port on one side (the Access Gateway) and an F_Port on the other (Edge Switch
in the fabric). In addition, all N_ Ports within a Trunk use the same 24-bit address.
Ports used for Trunking on the edge switch must be enabled using the
portcfgtrunkport command.
Revision 01 10 8 - 59
Access Gateway- F_Port Trunking (cont.)

• Trunks use a shared port area ID/index called a Trunk Area (TA)
- Ports to be used for the F_Port Trunk are configured on the edge
switch
- When configuring a TA, an index from one of the trunk port members is
used for the TA
- Remaining F_Port Trunk members share the same TA

The ports in an F_ Port trunk use a shared port area. All associated N_ Ports in the
Trunk share the same index.
Revision 0110 8- 60

300
• On the AG, all N_Ports within the
·s
Access
Port 0 ,G•tewa
same F_Port trunk area share the ~3~~~ Port o
,......._P
I
~ort;.;...1'-i,.:~
Port16
Master
~
same Port ID Hos1_2
PoL. ~
d?
030002
• F_ Ports on the AG configured to
any of the Trunk member N_ Ports
Host_3 '°"" :Port
.
030003
Port s, ,
Slave
030000
1
are mapped to the trunk master

N Port
• Frames received on any of these
Host_4
030004
Host_S
040501
j.J Switch_B
Host_6
F Ports are load balanced across 040502
Domain ID =4
all member N Ports Host_7

040503
• Example:
Host_8
N_Ports 16 and 18 share PIO= 040504
030000
N_Ports 20 and 22 share PIO=
040500
C 2009 Brocade Commun cal ans Systems nc 6•

Al 1R ghts Reserved

• Supported on B-Series switches only
• F_ Port masterless trunking is supported on ALL ports (0-47) on the
FC8-48 blade; Only ports 0-15 ar,e supported on the FC4-48 blade
• To configure the TA ports on the Edge switch:
- Disable the ports
- Enable the F_Port Trunk with the porttrunkarea --enable command
BSlOO : admin> porttrunkarea --enable 0-1 -index 0
- Enable the ports
- Show the TA/Tl port configuration with the porttrunkarea --show
command
BSlOO : admin> porttrunkarea - -show enabled

Port Type State Master TA DI
0 F-port Master 0 0 0
1 F-port Slave 0 0 1

Use this command to:

• assign a static trunk area (TA) to a port or port trunk group
• remove a TA from a port or group of ports in a trunk
• display masterless F_port trunking information
Masterless F_port trunking interoperates between the Access Gateway (AG) and
Condor-based platforms. It is designed to ( 1) prevent reassignments of virtual
addresses when F_ports come back online after going offline and (2) to increase
N_port bandwidth. Assigning a static TA to a port or trunk group enables F_port
masterless trunking on that port or trunk group. When a TA is assigned to a port or
trunk group, the ports will immediately acquire the TA as the area of their process
IDs (PIO). Likewise, when a TA is removed from a port or trunk group, the ports will
revert to the default area as their PIO.
Revision 0110 8- 62
• Enable F_Port Trunking on a port or group of ports

porttrunkarea - - enable <slot/port[ - Range]> - index <index>
• Disable F_Port Trunking on a port or group of ports

porttrunkarea - - disable <slot/port [-Range ] >
• Disable F_Port Trunking on all ports

porttrunkarea --di sable a l l
• View F Port trunk details

porttrunkarea - - show d isabled
porttrunkarea - - show enabled
porttrunkarea - - show all
porttrunkarea - -show trunk
porttrunkarea - - show <slot/port[ - Range]>

A non-bladed switch uses the area ID. A bladed switch uses the index number.
Revision 0110 8- 63
Access Gateway- F_Port Trunking Failover

• If trunk master goes down, F_ Ports
are mapped to the new master Host 1
03oo01
Nl_ Port without any PIO change; no Host_2
030002
failover happens
Host_3
- Does not cause fabric disruptions 030003
(no RSCNs)
• Failover is triggered when last I
member in the trunk goes offline
I
- When trunk goes offline, F_Ports Domain ID= 4
are failed over to other N Ports as

per failover policy and preferred
failover settings
- Mapped Online
( e N_Port - - . Failed Port Maps
.;) F Port - FailOver_Pa_lh_s _ _
c F= Port T;,;;th NPIV enabled)

Revision 0110 8- 64
BROCADE
EDUCATION SOLUTIONS
Revision 0110 8- 65
Access Gateway - Cascading

• Connects two Access Gateways, linking one end as an N_Port and
the other end as an F Port
- Core AG: Access Gateway connected to the fabric
- Edge AG: Access Gateway connected to the devices
• Higher over-subscription
while consolidating ports Core AG
to main fabric
• No license requirement
Fabric
(!) N_Port
O F_Port

Access Gateway Cascading provides more flexibility in configurations, more efficient

use of available ports, and additional cable and SFP consolidation.
Revision 01 10 8 - 66
Access Gateway- Cascading (cont.)

• Configuration restrictions:
- Must enable the Port Grouping policy on both the Edge and Core AG
- Only one level of cascading is supported 1
- Trunking between the Edge AG and Core AG is not supported
- Use the Advanced Device Security policy on the F_Ports directly

Footnote 1: Several Edge AGs can connect into a single Core AG to support higher
consolidation ratios.
Revision 01 10 8 - 67
Persistent ALPA
• Prior to Fabric OS v6.3.0, PIDs were dynamic, meaning that every time a
device logged in, it received a new PIO
- Some operating systems cannot tolerate changing PIDs
• Fabric OS v6.3.0 supports a persistent PIO value, whereby a device will
get the same PIO it had when it originally logged in
- The device must log in using the same N_Port on the AG
• More details are provided in the Appendix
Hosts
r. N_Port
\) F_Port
·: '.• F_Port (with NPIV enabled)

Revision 0110 8- 68
Summary
• F_ Port to N_ Port mapping is set by default
• N_ Port failover and tailback is configurable
• N_ Port Grouping allows a single Access Gateway to connect to
multiple fabrics
• F_ Port Trunking aggregates bandwidth between the Access
Gateway and the edge fabric
• Cascading connects two Access Gateways for scalability

Revision 0110 8- 69
BROCADE
EDUCATION SOLUTIONS
Revision 0110 8- 70
BROCADE
EDUCATION SOLUTIONS
Revision 0110 8- 72
Firmware Upgrade/Downgrade Considerations

• If using APC Policy
- Firmware upgrade from Fabric OS v6. 2.x, or previous, to v6.3.0 will set
automapbalance to its default settings:
• Enabled on an N Port online event
• Enabled on an F Port offline event
- Firmware downgrade from Fabric OS v6.3.0 to a previous version will
disable any automapb alance settings

Al R1gh's Reserved
Revision 0110 8- 73
Firmware Upgrade/Downgrade Considerations (cont.)
• If using PG Policy
- Firmware upgrade from Fabric OS v6.2.x, or previous, to v6.3.0 will
disable lb mode, and thereby disable aut omapbalance
- Firmware downgrade from Fabric OS v6.3.0 to a previous version will
disable any automapb a l ance settings
- Enabling lb mode willl set automapbalance to its defaults
• Enabled on an N Port online event
• Enabled on an F Port offline event

Al R1gh's Reserved
Firmware Upgrade and Downgrade

• Firmware upgrade from FOS v6.0 to v6.1
- If Trunking is enabled on edge switch with v6.1 and Trunking license is
available on Access Gateway, trunk will be formed
• Firmware downgrade from FOS v6.1 to v6.0
- Not allowed if any F_Port trunk is active
- Trunking must be disabled before the downgrade
• Recommended method for upgrade in case of Trunking
- Upgrade both Access Gateway and edge switch to FOS v6.1, then
configure Trunking

Only one firmware version upgrade is supported in v6.x (upgrade from v6.0.x to
v6.1.x will work; v6.0.x to v6.2.x will not).
Revision 0110 8- 75
Managed Fabric Name Monitoring Mode (mfnm)

• This command changes the fabric name monitoring mode from
"default" to "managed"
• In both default and managed mode, the system queries the fabric
name1 once every 120 seconds
• If it detects an inconsistency it triggers a RASLOG message
- For example, if the port group is connected to multiple fabrics
• The difference betweern default and managed fabric name
monitoring is that in managed mode, failover is disabled for all ports
in the port group if the system detects an inconsistency in fabric
names
C 2009 Brocade Commun cat ons Syslems Inc 76

All R 'ghts Reserve<l
See the following page for a description of the additions to the "ag" CLI command.
Footnote 1: The "fabric name" of a fabric is the WWN of the principal switch of the
fabric. In AG mode, when you do an "ag --show" , the output will contain a
section that tells you which N-ports are connected to which fabric (according to
fabric names).
s1ai1e11·t:1min> ao - staow
N.ne : ~Mt.cti_Sll
Nod.NarM · 10'00:00:0$·1e:3S·etu7
Number d P'"'1> - 1e
lf>Addresl(n): 10.115.74.!i3
F'lf'Tl'M'al'e Vtni«I \6.0.0
N_P'Mt '
F_PMI: 10
Pdlctes~d : pg
P4nr&ltlfltAJ.PA ;~
Por'IGl'Cll.P~ormllilon ·
01;3 pgO .
2 0;2 seecndFaMc -
10:00«J:OS:1e;l4;01-d7 O;l ;U
Port PortlD Ad8ched FWWN FO f8 IP_Addi' F_Perts
00Jd3d0a(l()20;0a:OOOS:1• '37:11.• a 10 10,32.74 1094:5:6~
1 O:dldObOO20.0b0&05' 1L37 11•• 0 1 10,3z._74 109 7;8;9,

20l6d0c00 20~.05. 1•.37: 11 .. . 10 10.3.2.74.109 10.11.
3o...ed0dll0 20:0ctOO:OS:1e:)7 11·eaO I 10.32..7-t 109 12,13:
f"_Pon Information
• Qd!d0.01 21;0000'.0;Sb S3 •:S:cd 0 2 no

5 Cb:6dOe02 21:0t OO:e0·8b·a3 .3·Qd o 2 no
6 Ox&f0..00 21 :00:00 e0:8b~83:3e c.e 0 2 no
7 01M0b01 21 :01.00:e0.8b:-3:3e.oc 1 3 no
rr"""'""°"""'
Revision 01 10 8 - 76
BROCADE
EDUCATION SOLUTIONS
Revision 0110 8- 77
Persistent ALPA
• Prior to Fabric OS v6.3.0, PIDs were dynamic, meaning that every time a
device logged in, it received a new PIO
- Some operating systems cannot tolerate changing PIDs
• Fabric OS v6.3.0 supports a persistent PIO value, whereby a device will
get the same PIO it had when it originally logged in
- The device must log in using the same N_Port on the AG
Hosts
r. N_Port
\) F_Port
·: '.• F_Port (with NPIV enabled)

Revision 0110 8- 78
When a New Host Logs In

• The assigned PIO will be kept on the AG in the PWWN to ALPA
mapping table, used to maintain consistency
PWWN = 01.02.03:04:05.06:07.08 AG requests ALPA for tile host

dunnglogin
Gets a PIO With tile ALPA

value requested by AG
AG receives reply wrlh the Al.PA value;
Adds the ALPA value to the table

Al 1R ghts Reserved
The host does a login request to the AG, which forwards it on to the fabric. The
fabric responds with a PID, which the AG puts in its mapping table, persistently
assigning the PID to the PWWN for that host. The AG simultaneously sends the P IO
to the host.
Revision 0110 8- 79
If the Host Logs Out

• The PWWN to ALPA mapping will be maintained in the mapping
table on the AG
PWWN = 01.02:03:04:05:06:07:08 P"N'm - ALPA mapping present In

the table
Host Loos out

ALPAstill available
with the AG

If the host logs out, the PIO is kept in the mapping table on the AG.
Revision 0110 8- 80
If the Host Logs Back In on the Same N-Port

• If available, the same PIO will be given to the host when it logs
back in to the same N Port
PWWN = 01:02:03.04:05.06:07'08 AG searches If the PWVVN is

presentln the table. lfthere, requests
identicalALPAas mentioned
AG
AG receives reply with the ALPA

Has a same PIO value which value. sends host Identical PIO as
was assignee initially before

Al 1R ghts Reserved
If the host logs back in using the same N_Port on the AG, the AG wi ll search its
mapping table for the PWWN of the host. When it finds the PIO associated with the
PWWN, it will request that PID from the fabric, which will respond with the
requested PIO.
Persistent ALPA (cont.)

• For now, only the CLI can be used to configure this feature
• Any devices logged into the fabric when this feature is turned on
will retain their current ALPAs, as persistent ALPAs
• There is at least one instance where a device will be assigned a
different PIO even though it logs back in to the same port:
- In the event of an N_ Port failover the domain and area portion of the
PIO will change, but the ALPA portion will remain the same
• This feature is persistent across reboots of the AG

Al R1gn•s Reserved
Access Gateway uses a table to maintain a list of available and used ALPAs. When
the number of entries in this table is exhausted, the host receives an error message.
You can remove some of the entries to make space using the ag
deletepwwn f romdb command.
switch : admin> ag --deletepwwnfromdb PWWN
The max number of entries in the table for each port is set through the configure CU command, as follows :
switch : admin> switchdisable

switch : admin> configure
Configure ...
Fabric parameters (yes , y , no , n) : [no]

F- Port login parameters (yes, y , no , n) : [no] y
Maximum logins per switch : (1. . 25200) (3200)

Maximum logins per port: (1 .. 255) [255]
Logins per second : (0 . . 40) [OJ

Login stage interval (milli-seconds) : (0 . . 10000) [OJ
System services (yes , y , no , n) : [no]

Portlog events enable (yes, y , no , n) : [ no]
ssl attributes (yes , y , no, n} : [no]
rpcd attributes (yes , y , no , n) : [no]
cfgload attributes (yes , y, no , n) : [ no )
webtool s attributes (yes , y , no , n) : [ no]
system attributes (yes , y , no , n} : [no )
Revision 0110 8- 82
Modified CLI Command

• The ag command has been modified with new options to
accommodate the persistent ALPA feature
• Once enabled , the ALPA portion of all device PIDs become
persistent regardless of whether they were logged in before or after
the persistent ALPA feature was enabled
• ALPA persistence ensures that there is no inconsistency between
logged in devices
• The persistent ALPA feature is, by default, disabled
• ag --persistentalpaenable 1 I 0 mode - Configures the
persistent ALPA feature
- 1 I o - Specify 1 to enable persistent ALPA, specify 0 to disable the
feature
C 2009 Brocade Commun cat ons Syslems Inc 83

All R 'ghts Reserve<l
mod e - Specifies the manner in which the ALPA is obtained in the event
that the ALPA value is already taken by another host. Valid modes are:
- s Specifies a stringent ALPA request mode. In stringent mode, the login is
rejected if the ALPA is not available
- f Specifies a flexible ALPA request mode. In flexible mode, the host login is
accepted either with the requested ALPA value or with a differentALPA value if the
requested ALPA is not available
ag --printa l pamap F_Port - Displays the database entry for t he specified
port. An F_ Port must be specified. The output displays the PWWN-to-host-ALPA
mapping
ag --deletepwwnf r omdb PWWN - Removes the specified port WWN entry from
the database after the host has logged out
ag--clea r alpamap F_Port Clears the ALPA values for the specific F_ Port. This
command removes the PWWN-to-ALPA-value mapping from the database
Revision 0110 8- 83
BROCADE
EDUCATION SOLUTIONS
Revision 0110 8- 84
Access Gateway - Auto Port Configuration

300
• The Auto Port Configuration
feature brings plug-and-play
Port 1 •
functionality to the Access •
Port 2 ~.
Gateway
Port3+
• With Auto Port Configuration Port 4 \
enabled, there are changes to Port 5 !:.

Port6
Access Gateway behavior:
·.:·
Port 7 • Fabric_ 1
1. No pre-determined port role -
Based on the attached device or Port 9
Port 8~~Port 3
J
switch, each switch port Port 10
~-. cx!awi:n~o~g
automatically configures as F_Port Port 11 Port 2
• )
or N Port Port 12 •
..
Port 13 l, e N_Polt
F Port
Port 23
LJ • F_Port (with NPIV enabled)

When Auto Port Configuration is enabled on an Access Gateway, the first major
change is that port roles are not pre-configured as F_Port or N_Port. Instead,
Access Gateway switch ports react as regular switch ports: based on the attached
device or switch, the port automatically configures as an F_Port or N_Port. Users
can now attach devices or switches to any port, without needing to know whether a
port is locked as an N_Port or not.
In the example above, hosts have been attached to an Access Gateway with Auto
Port Configuration enabled. The following ports are configured in different modes
than the default Port Map:
• Ports 7 and 8 are configured as N_Ports because they are attached to a
switch. If Automatic Configuration was disabled and the default Port Map was
in effect, these ports would be F_Ports.
• Ports 12 and 13 are configured as F_Ports because they are attached to hosts.
If Automatic Configuration was disabled and the default Port Map was in effect,
these ports would be N_Ports.
Revision 0110 8- 85
Access Gateway-Auto Port Configuration (cont.)

300
2. No pre-determined Port Map
- F_Ports do not have a Primary
N Port
- F_Ports are evenly mapped
(round-robin) across all available
N_Ports
Port 3 Switd'l A
·: '.• Domain ID = 9
.r------- { •
. ,;)
Port 13J , e N_Port
F_Port
Port23
L ·: :• F_Port (with NPIV enabled)

The second major change with Automatic Configuration is there is no pre-configured

Port Map. F_ Ports are assigned to the available N_ Ports on a round-robin basis, not
based on a user-defined Port Map.
In the example above, hosts have been automatically mapped to the N_Ports as
follows:
• Ports 1 (Ox090301 ), 4 (Ox090302), and 12 (Ox090303) are mapped to
N_Port 7 (Ox090300).
• Ports 2 (Ox090201 ), 5 (Ox090202), and 13 (Ox090203) are mapped to
N_Port 8 (Ox090200).
Revision 0110 8- 86

300
3. No fixed port map -An
F_Port is not guaranteed the
same N Port as
devices/switches are
added/removed
When F Ports and N Ports
are added or removed , the Fabric_1
port map is automatically . ~Ort 3 Switch A
updated to ensure an even ++--~. ·' Domain 9 ID=
distribution (round-robin) of
F- Ports across N- Ports
Port23
__ ••
• ___.
·::• F_Port (with NPIV enabled)

The third major change with Automatic Configuration is that the Port Map is not
fixed - that is, an F_Port does not always have the same Primary N_ Port. As hosts
or switches are attached to the Access Gateway, the Port Map is automatically
readjusted to ensure an even distribution of F_Ports across the N_Ports.
In the example above, a new switch port is attached to port 9, creating a new
N_Port (PID = Ox090400). As a result, the F_Ports are remapped across the three
N_Ports. This results in four of the six hosts (noted with a* symbol above) changing
their PID:
• Host_3 has changed from Ox090302 to Ox090401 .
• Host_4 has changed from Ox090202 to Ox090302.
Revision 0110 8- 87

4. Failover, but no Failback -
Since there is no Primary *
NI Port, some failover-related
fe;tures also function differently *
- N Port Failover: Works the same* Fabric_ 1
Switc:h_A
as default port map Domain ID= 9
• No Preferred N_Ports or Port * Po113

Groups
• Only attach the Access Gateway
to one fabric
-
- N Port Failback: Devices do not
tailback!
* :-H~st)
: 0 90203
* .••
- Mapped Online
- - · Failed port maps
- Faitover paths

The final major change with Automatic Configuration is that there is N_Port Failover,
but no N_Port Failback. When an N_Port fails, the F_Ports mapped to that N_ Port
are automatically failed over, and the Port Map is automatically readjusted to ensure
an even distribution of F_Ports across the N_Ports. However, because F_Ports do
not have a Primary N_Port, there is nothing for a failed-over F_Port to fail back to.
The absence of a Primary N_Port also means that there are no Pref,erred N_Ports
or Port Groups when Automatic Configuration is enabled.
In the example above, a new switch port is attached to port 9, creating a new
N_Port (PIO= Ox090400). As a result, the F_Ports are remapped across the three
N_Ports. This results in all of the six hosts (noted with a* symbol above) changing
their PIO:
• Host_1 has changed from Ox090301 to Ox090201 _
• Host_2 has changed from Ox090201 to Ox090401 _
• Host_3 has changed from Ox090401 to Ox090202_
• Host_ 4 has changed from Ox090302 to Ox090402_
• Host_5 has changed from Ox090202 to Ox090203_
• Host_ 6 has changed from Ox090402 to Ox090403_
Revision 0110 8- 88

• Because the port map automatically updates, an F_ Port
experiences PID changes as devices, switches, or ports are added ,
removed , come online, or go offline
- F_Port must be tolerant of PIO changes!
• When Auto Port Configuration is enabled , for implementations
using Fabric OS versions prior to v6.3.0, Access Gateway
management changes:
- Cannot lock a port as an N_ Port
- Cannot configure an F_ Port to an N_Port
- Cannot specify Port Groups - these require fixed N_Ports
- Cannot specify Preferred N_ Ports
- Access Gateway can only be attached to one fabric

Revision 0110 8- 89
Extending APC Policy to Port Groups

• Fabric OS v6.3.0 adds two new port group modes to allow F_Ports
to connect to a specific port group (PG) and to connect a port
group to multiple fabrics:
- Login Balancing mode (LB)
- Managed Fabric Name Monitoring mode (MFNM)

The automatic Port Configuration (APC) policy provides the ability to automatically
discover port types (host vs. fabric) and dynamically update the routing maps when
a new connection is detected. This policy is intended for a fully hands-off operation
of Access Gateway. APC dynamically maps F_ports across available N_Ports so
they are evenly distributed. For example, when a port on an AG is connected to a
Fabric switch, the AG configures the port as an N_Port. If a host is connected to a
port on an AG, then the AG determines that it is connected and configures the port
as an F_Port and automatically maps it to an existing N_ Port with the least number
of F_Ports mapped to it.
When the APC policy is enabled, it applies to all ports on the switch. Enabling the
APC policy is disruptive and erases all existing F_Port-to-N_Port mappings.
Therefore, before enabling the APC
policy, you must disable the AG module. When you disable the APC policy, the
N_Port configuration and the F_Port-to-N_Port mapping revert back to the default
factory configurations for that platform.
Revision 01 10 8 - 90
Login Balancing Mode (LB)

• Prior to Fabric OS v6.3.0, APC was only a switch-wide policy
• Login Balancing mode allows the selective use of the APC policy
on a per-port group basis
• Allows the automatic balancing of F_Ports among N_Ports within a
port group for better performance and reliability
• If Login Balancing mode is enabled , and an F_Port goes offline,
logins in the PG are redistributed among the remaining F_ Ports
• If an N_ Port comes online, port logins in the PG are redistributed to
maintain a balanced N Port-to-F Port ratio
• This operation is disruptive
• Login Balancing must be explicitly enabled on a port group

See the following page for a description of the additions to the "ag" CLI command.
ag --pgcreate- pgid " N_ Portl [ ; N_ Port2 ; . .. ]" [ - n pgname] [ - m

"lb ; mf nm" J - Creates a port group with the ID pg i d and a specified list of
N_Ports to be included in the policy. The list must be enclosed in quotation marks.
Ports must be separated by semicolons. The Port Group ID must not exceed 64
characters. Optionally specify a name for the port group and a mode. Modes are
disabled by default.
ag --pgmapadd pg id " F_ Portl [ ; F_ Port2 ; ... ] " - Maps the specified
F_Ports to the PG identified by the pg id. Upon execution, the system identifies the
least loaded N_Port in the port group and maps the F_Ports to that N_Port. The port
list must be enclosed in double quotation marks. Ports must be separated by
semicolons.
ag --pgmapdel pgid " F_Po rt l[ ;F_Port2 ; ... ] " - Removes one or more
F_Ports that are part of the port group identified by the pg id from their mapping to
a corresponding N_ Port. The port list must be enclosed in double quotation marks.
Ports must be separated by semicolons.
ag --pgsetmodes pgid " lb;mfnm" - SetstheAPCmodesforthespecified
port group. The mode list must be enclosed inn double quotation marks and the
modes must be separated by a semicolon. Alternately the modes may be set at the
time the port group is created, with the pgcreate command. The following modes
are supported:
lb - Specifies the load balancing mode for the specified port group. If load balancing
mode is enabled and an F_ Port goes offline, logins in the port group are
redistributed among the remaining F_Ports. Similarly, if an N_Port comes online,
port logins in the IPG are redistributed to maintain a balanced N_Port to F_Port ratio.
This operation is disruptive. Load balancing mode is by default enabled in the
default port group 0. It must be explicitly enabled on any other port group.
mfnm - Enables the managed fabric name monitoring mode (MFNM) in the
specified port group. This command changes the fabric name monitoring mode from
"default" to "managed". In both default and managed mode, the system queries the
fabric name once every 120 seconds, and if it detects an inconsistency, for
example, if the port group is connected to multiple fabrics, it triggers a RASLOG
message. The difference between default and managed fabric name monitoring is
that in managed mode, failover is disabled for all ports in the port group if the
system detects an inconsistency in fabric names.
Auto Remap Control

• Fabric OS v6.3.0 is enhanced so that the rebalancing of F_ Ports-
to-N_Ports can be manually disabled, depending on user
requirements
• To prevent disruption, Fabric OS v6.3.0 allows control of the
automatic rebalancing of F_Ports for login distribution in the event
that an F_ Port goes offline, or an N_Port comes online
- The disruptive rebalancing of F_Ports might not be required in all
scenarios
• This is done using the agautornapbalance command to disable
the F_ Ports and distribute them among the available N_Ports
C 2CC9 B·:x:a::le Com ............ :a· ons Sys•e--s "'C 93

All R g'""S C?ese"Ved
With APC Policy and introduction of Login Balancing in PG Policy, the F_Port to
N_Port mapping in an AG can change dynamically depending on addition/removal
of ports. Some admins might want to control this automatic rebalancing feature -
which can be achieved using the agautomapbalance CLI.
A --force option was introduced in order to allow admins to run nightly/weekly

scripts per say - in case the automatic rebalancing is turned off, and system is
unbalanced (e.g.: 10 F_Ports are mapped to 2 out of 5 N_Ports because 3 N_Ports
came online at a later time).
CLI Description
agautomapbal.an ce - -for ce FORCED One-lime auto login dis tribution
In case the agautomapbalance option is disabled for
N_ Port addition, F_ Port removal or both, this
command will rebalance the F _ Port lo N _ Port
mapping in a forceful mannef" only O N CE.
agautomapbal.a n ce [ - -ena.bl.e I --disa.bl.e] All the above commands will have -pg# option so
[-fport I -nport ] they can optionally be enabled or disabled on a given
[-pgf ) port group in AG
This option is a llowed only when Load balancing
Policy is enabled on this particular PG
agautomapbal.ance [ - -enab l.e I --disab1-e) All the above commands should have -all option so
[-£port I - n port ] they can optionally be enabled or disabled per entire
r-al.1- J AG module (all PGs )
This option will enable/disable auto mapping on Port
Groups which have Load balancing Policy is
enabled.
Revision 0110 8- 93
CLI Examples
• agautomapbalance --enable - fport
- Will enable automatic rebalancing of F_ Ports when an online F_ Port goes
down
• agautomapba l ance - - disab l e -nport -pg 1
- Will disable automatic rebalancing of F_ Ports when a new N_Port comes
online in PG 1
• agautomapba l ance - - enabl e - fport - all
- Will enable automatic rebalancing of F_ Ports on all Port Groups in the AG
• agautomapbalance - - force
- W ill initiate one time rebalancing of F_ Ports (APC Policy)
• agautomapba l ance --force - pg 0
- Will initiate one time rebalancing of F_ Ports in PG 0

Al R1gh's Reserved
Revision 0110 8- 94
CFP380 Internal Use Only Fabric Security
Revision 0110 9- 1
Objectives
• After completing this module, attendees will be able to describe and
configure the following advanced security features:
• Policy distribution
• Fabric Configuration Server (FCS)
• Switch Connection Control (SCC)
• Device Connection Control (DCC)
• IP Filter Policies (IPFILTER)
• Advanced Device Security for Access Gateway

Revision 0110 9-2

BROCADE
EDUCATION SOLUTIONS
Revision 0110 9-3

Fabric Security Access Control Lists

• Fabric OS v6.3 supports the following Access Control List (ACL)
security policies: (No license required)
- Fabric Configuration Server (FCS):
• Restricts which switches can be used to change the configuration of the
fabric
- Switch Connection Control (SCC):
• Restricts which switches can join a fabric
- Device Connection Control (DCC):
• Restricts which Fibre Channel devices can connect to which Fibre Channel
switch ports
- Advanced Device Security (ADS):
• Policy based Device Connection Control (DCC) used to restrict device
access on the Access Gateway

M-EOS Security:
Switch Binding: (SANtegrity Binding license required) This is a list of devices and
switches that can log into a switch by specifying their WWNs in a switch
membership list.
Fabrii c Binding: (SANtegrity Binding license required) This is a list of switches that
can join a fabric by specifying their WWN and Domain ID in the Fabric Membership
list. This is required for FICON.
Port Binding: This binds a device WWN to a switch port. Only that WWN can log
into that switch port.
Revision 0110 9-4

Fabric Security Access Control Lists (cont.)

- IP Filter Policy (IPFILTER):
• Filters IP management interface traffic
- Authentication policy for fabric elements (AUTH): 1
• Public K,ey Infrastructure (PKI)
- Password database and user policy (PWD): 2
• Enables users and passwords configured on one switch to be distributed to
other switches

After Fabric OS v5.2, SFOS must be removed and policies will need to be recreated
using the new base Fabric OS policies.
Unlike the licensed Secure Fabric OS (SFOS) feature, the policies are part of base
Fabric OS.
• Not interchangeable with their equivalent SFOS policies
Footnote 1: AUTIH policy details are included in the appendix to this module.
Footnote 2: PWD policies are covered in the CFA280 course and not in this course.
Revision 0110 9-5

BROCADE
EDUCATION SOLUTIONS
Revision 0110 9-6

Policy Database Distribution

• Fabric OS lets you manage and enforce the ACL policy database
on either a per-switch or fabric-wide basis
• Policies are stored in a local database on each switch 1

- Fabric OS v6.0 and later, database size is 1 MB
- Use secdefi nes i ze to see the database size
• With the exception of the DCC and ADS policies, only one policy of
each type can be activ,e2

Footnote 1: The FCS, DCC, SCC and IPFILTER policies are grouped by state and
type while the PWD and AUTH policies are only grouped by type (there is no state
for these two policies).
A policy can be in the following state:
• Active-The policy is being enforced by the switch
• Defined-The policy has been set up but is not enforced
A group of policies is called a Policy Set.
Each switch has the following two sets:
• Active policy set-Contains ACL policies being enforced by the switch
• Defined policy set-Contains a copy of all AGL policies on the switch
When a policy is activated, the defined policy either replaces the policy with the
same name in the active set or becomes a new active policy. If a policy appears in
the defined set but not in the active set, the policy was saved but has not been
activated. If a policy with the same name appears in both the defined and active
sets but they have different values, then the policy has been modified but the
changes have not been activated.
Footnote 2: There are two IP Filter types, !Pv4 and 1Pv6, each with its own active
policy, but only one policy for each IP type can be active.
Revision 0110 9-7

Policy Database Distribution (cont.)

• Switch level distribution - Manual
- Each switch can be set to Accept or Reject individual
security policies 1
• Use the fddcfg command to show or set each policy for Accept or Reject
- All policies can be manually distributed to fabric switches.
•The IPFILTER and AUTH policies can only be manually distributed
• ADS is local to the AG only
• Fabric level distribution - Automatic

- Fabric-Wide Consistency2 policy can distribute the SCC, DCC, and
FCS policies automatically

Footnote 1: Policy distribution can be controlled at the switch level: Each switch
can be set to accept or reject each individual security policy
(FCS/SCC/DCC/IPFILTER/PWD/AUTH). It allows switches in a fabriic to have
different device access security settings-or no settings at all. The default switch
setting is to accept all policies.
Use the f ddcfg command to show or set each policy for Accept or Reject.
Footnote 2: The SCC, DCC, and FCS policies can be switch-centric (manually
distributed) or they can be automatically distributed to k·eep them consistent across
the fabric.
The Fabric-Wide Consistency policy dictates when a switch-level SCC and DCC
reject policy is allowed.
All other policies must be manually distributed using the distribute command.
Virtual Fabric considerations: ACL policies such as DCC, SCC, and FCS can be
configured on each logical switch. The limit for security policy database size is set to
1Mb per logical switch with an 8 MB maximum. FCS, DCC, sec, and AUTH
databases can be distributed using the -distribute command, but the PWD and
IPFILTER databases are blocked from distribution.
Fabric-wide consistency policies are configured on a per logical switch-basis and
are applied to the fabrics connected to the logical switches. Automatic policy
distribution behavior for DCC, SCC and FCS is the same as that of pre-v6.2.0
releases and are configured on a per logical switch basis.
Revision 0110 9-8

Policy Database Distribution (cont.)

• Fa bric-Wide Consistency has three states:
- Absent (not defined): Fabric-Wide Consistency is not
defined (default); manual database distribution is required using the
distribute command
- Tolerant: Switches are not required to have the same databases
- Strict: Switches in the fabric always have the same databases
• If one switch in a fabric has a strict policy, all switches in the fabric must also
have a strict policy
• Strict fabric-wide consistency policy are not supported in FC-FC Routed
fabrics in the edge or the backbone
• FC routers do not support the strict fabric-wide consistency policies

Fabric-wide consistency policy in the fabric:

1. Absent: SCC policies are manually distributed. Switches can have different SCC
polices.
2. Tolerant: Switches can have different SCC policies; however, any changes to
the sec policy are automatically distributed throughout the fabric, so if the
switches have their local-switch policy set to -Acq!t", all switches will always
have the same sec policy.
3. Strict: All switches must have the same SCC policy, and any switch with a
different policy is segmented from the fabric. Any changes to the SCC policy are
automatically distributed throughout the fabric.
Revision 0110 9-9

This example shows how to set a strict SCC and tolerant DCC fabric-wide
consistency policy.
switch : admin> fddcfg --fabwidese t "SCC: S ; DCC"

switch : admin> fddc fg --s h owall
Local Switch Confi guration for all Databases :-
DATABASE - Accept/Reject
sec accept
DCC accept
PWD accept
FCS accept
AUTH accept
IPFILTER accept
Fab r ic Wide Cons i stency Policy :- " SCC : S; DCC "
Policy Database Distribution (cont. )
Polley Distribution Commands

fddcf g --s howall Display the Accept/Reject
configuration for all databases
fddcfg -- localaccept <po l icy l ist> Configure switch to accept
distributions of the specified
polices
fddcfg -- localreject <po l icy l ist> Configure switch to reject
distributions of the specified
polices
fddcf g -- f abwideset <policy lis t > Sets the fabric-wide consistency
policy

Administrative Commands f ddc f g ( l m)

fddc f g • Manages the Fabric Data Distribution conf iguration parameters.
Description:
Use this command to manage the Fabric Data Distribution configuration
parameters.
These parameters control the Fabric-Wide Consistency policy in non-secure mode.
Operands:
--showall
Displays the accept/reject configuration of all policy sets and the fabric-wide
consistency policy on the switch.
-- l ocalaccept po l icy_lis t
Configures the switch to accept distributions of the policies in policy_ list. The
policy_list is a semicolon separated list of supported policy sets, for example,
- SCOJCC". Supported policies are Switch Connection Control (SCC), Device
Connection Control (DCC) and Password (includes account database and password
policies) (PWD).
--localreject policy_ list

Configures the switch to reject distributions of the policies in policy_ list. A database
cannot be rejected if it is specified in the fabric-wide consistency policy. The
policy_ list is a semicolon-separated list of supported policy sets, for example,
-SC(:DCC". Supported policies are Switch Connection Control (SCC), Device
Connection Control (DCC) and Password (includes account database and password
policies) (PWD).
--fabw i deset policy_list

Sets the Fabric-Wide Consistency policy. A database that is set to reject
distributions cannot be specified in the fabric-wide consistency policy. To set the
Fabric-Wide Consistency policy as strict, use the strictness indicator "S". To set the
Fabric-Wide Consistency policy as tolerant, omit the "S". A valid policy set should be
of the form "SCC:S;DCC". To set the fabric-wide policy to NULL (default) or no
fabric-wide consistency, use the policy Set"". Supported policies are Switch
Connection Control (SCC) and Device Connection Control (DCC).
• To display the fabric-wide consistency policy and the accept/reject configuration
for all databases:
switch : admi n> fddcfg --showall
Local Switch Configuration for all Databases :-
DATABASE - Accept/Reject
sec - accept
DCC - accept
PWD - accept
FCS - accept
AUTH - accept
I PFI LTER - accept
Fabric-Wide Consistency Policy : - " "

• To configure this switch to accept distribution of SCC & PWD databases:
switchadrnin> fddcfg --localaccept "SCC;PWD"
Local Switch Configured to accept policies.
• To configure this switch to reject distribution of SCC & DCC databases:
switch : adrnin> fddcfg - - localreject "SCC;DCC"
Local Switch Configured to reject policies.
• To set the fabric-wide consistency policy to strict for SCC and tolerant for DCC
("SCC:S;DCC"):
switch : admin> fddcfg --fabwideset "SCC : S;DCC;"
BROCADE
EDUCATION SOLUTIONS
Fabric Configuration Server (FCS)

• An FCS switch is not created by default, an FCS policy must be
created first
• Without an FCS policy, any switch in the fabric can make changes
to fabric-wide services
• One or more specific "trusted" switches
- Used for managing fabric-wide administrative features
- One FCS switch is configured to be the primary FCS switch
- One or more backup FCS switches are recommended to provide
failover ability in case the primary FCS switch leaves the fabric
- All other fabric switches are non-FCS switches

Fabric Configuration Server (FCS) List

• The first switch in the list
becomes the Primary FCS
Primary FCS
• Remaining switches in the list (1 11 Switch in List)
become backup FCS switches ..
~ ~
- If the first switch in the 1" Backup FCS

(2nd Switch in List)
FCS list is not reachable,
the next switch becomes ,,.-
-
the Primary 2nc1 Backup FCS
(3rd Switch in List)
-When a switch higher on

the list returns to the •••
fabric, it retakes the
primary role

Footnote 1 : While creating the FCS policy, the local switch WWN automatically
gets included in the list. Additional switches included in the FCS list are backup FCS
switches and other switches not included are non-FCS switches.
In this example output, the Primary FCS switch was disabled. The backup FCS
switch became the primary as indicated in the Primary column:
B5100 : admin> secpolicyshow
ACTIVE POLICY SET

FCS POLICY
Pos Primary WWN Did swName
1 No 10 : 00 : 00 : 05 : 1e : 36 : 2e : 62 - Unknown
2 Yes 10 : 00 : 00 : 05 : 1e : 04 : 24 : 8c 4 85100
DEFINED POLICY SET

FCS POLICY
Pos Primary WWN Did swName
1 No 10 : 00 : 00 : 05 : 1e : 36 : 2e : 62 - Unknown
2 Yes 10 : 00 : 00 : 05 : 1e : 04 : 24 : 8c 4 85100
FCS Details
• Once the FCS policy is configured , enabled, and distributed,
switch security and fabric-wide administrative command behavior
changes
• Only the Primary FCS switch can invoke commands that affect the
entire fabric 1 , which include:
- Create or modify FCS , sec and DCC policies in a Fabric-Wide
Consistency policy enforced fabric 2
- Distribute the password database3
- Zoning
• FCS policy must be consistent across the fabric
- If the policy is inconsistent in the fabric, then you will not be able to
perform any fabric-wide configurations from the primary FCS

Footnote 1: Only the Primary FCS switch can invoke any fabric-wide commands
(aliadd, zoneadd, cfgadd, cfgcreate, def zone, passwd, etc.).
• This includes all zone configuration commands.
• Non Primary FCS switches can invoke related show commands.
Footnote 2: Only the Primary FCS switch can add, create, delete, and remove SCC
and DCC policies in a fabric with a Fabric-Wide Consistency policy of tolerant or
strict.
Footnote 3: Once an FCS policy is created, activated, and distributed across a
fabric, only the Primary FCS switch can be used to distribute the password
database.
Revision 01 10 9 - 16
FCS Distribution
• The FCS policy can be automatically distributed 1 using the Fabric
Wide Consistency policy 2"" Backup
FCS Switch

Al 1R ghts Reserved
Footnote 1: The FCS policy can be automatically or manually distributed. Each

Fabric OS v5.3 or later switch can be configured to accept (default) or reject the
FCS policy distribution. Use the following command to reject the FCS database
distribution:
fddcfg --localreject FCS
The FCS policy, like other policies, is stored in a switch-local database. The FCS
policy can be in the following state:
• Active-The policy is being enforced by the switch.
• Defined-The policy has been set up but is not enforced.
A group of policies is called a =Poli~ Set'. Each switch has the following two sets:
• Active policy set-Contains ACL policies being enforced by the switch.
• Defined policy set-Contains a copy of all ACL policies on the switch.
FCS Configuration
• Steps to create an FCS ACL policy:
1. Create FCS policy using secpolicycreate
switch : admi n> secpo1icycreate "FCS_POLICY", "3; 4"
FCS POLICY has been created .
2. Activate FCS policy using secpolicyacti vate 1

• This command also saves the cilanges to the defined policy set
3. Distribute the FCS policy to switches configured to accept FCS

distribution using distribute 2 or Fabric Wide Consistency policy

Footnote 1: When a policy is activated, the defined policy either replaces the policy
with the same name in the active set or becomes a new active policy. If a policy
appears in the defined set but not in the active set, the policy was saved but has not
been activated. If a policy with the same name appears in both the defined and
active sets but they have different values, then the policy has been modified but the
changes have not been activated.
FCS enforcement does not apply to pre-Fabric OS v5.3.0 switches, they will be able
to initiate all operations but fabric-wide operations can fail if FCS policy is present
on the Fabric OS v5.3.0 switches in the fabric. Once the FCS policy is activated and
distributed to all fabric switches, only the Primary FCS switch will be able to
distribute subsequent policy updates across the fabric.
The fabric in this example only has a distributed FCS policy, no other policies in this
example were activated or distributed.
Footnote 3: Sample command:
distribute -p policy_ list -d switch list
switch : admin> distribute -p FCS -d " 1 ; 3 "
FCS enforcement for the distribute command is handled differently for FCS and
other databases in an FCS activated fabric.
1. The primary or any backup FCS switch can initiate the distribution of the FCS
policy.
2. Only the Primary FCS switch can initiate the distribution for other database
distributions.
BROCADE
EDUCATION SOLUTIONS
Setting SCC and DCC Policies

• Ensure that only specified switches and devices may join a fabric
• Unspecified switches and devices1 are denied access to the fabric
• Works at the device login stage - enforced earlier than zoning 2
Device Connection Control

(DCC) Policies
~~~~~~~-CLlT'
~.
~.
New Switch Switch Con nection Control
& Devices (SCC) Policies

Footnote 1: Switches not included in the SCC Policy will be segmented from the
fabric. Devices not included in the DCC Policy attempting to log into ports that are
included in DCC policies will disable the port, but for ports that are not included in a
DCC Policy allow any device to attach.
Footnote 2: Device access security is a step prior to and in addition to any zoning
in providing strong security in a fabric.
With zoning enabled, device-to-device communication is secured. However, new
devices and switches may still connect to and join an existing fabric, a key breach in
a larger security plan.
Switch Connection Control (SCC) Policy

• Used to restrict which switches can join the fabric
- Switches are checked against the policy each time an E_ Port-to-
E Port connection is made
- Only one sec policy can be created per switch
- To connect a Fibre Channel router to a fabric with an sec policy, the
front domain of the router must be included in the sec policy1
- By default any switch is allowed to join the fabric
10:00:00:05:1 e :53:fc:f0
10:00:00:05: 1e :08:0a:68
10:00:00:05:1e:08:0a:68

Footnote 1: FC-FC Routers are not supported in fabrics with =strtt' fabric wide
consistency policies.
Switch Connection Control (SCC) Policy (cont.)

• Steps to create an SCC ACL policy:1
1. Create SCC policy using secpolicycreate
2. Activate SCC policy using secpolicyacti vate
3. Distribute the SCC policy to switches configured to accept SCC

distribution using d istribute command or Fabric Wide
Consistency policy

Footnote 1 : To create an SCC policy:

1. Connect to the switch and log in
2 . Type secpolicycreate " SCC_ POLICY", " member ; ... ; member"
member indicates a switch that is permitted to join the fabric. Specify switches by
WWN, domain ID, or switch name. Enter an asterisk(*) to indicate all the
switches in the fabric.
For example, to create an SCC policy that allows switches with domain IDs 2 and
4 to join the fabric:
swi tch : adrni n> secpolicycrea t e " SCC_ POL ICY ", " 2 ; 4 "
SCC POLICY has been created
3. To save or activate the new policy, enter either the secp o licysave or the
s ecp olic yac t iva te command.
Revision 01 10 9 - 22
Device Connection Control (DCC) Policy

• Used to restrict which device WWN can connect to which
switch ports 1
- Each device WWN can be bound to one or more switch ports
- Multiple DCC policies can be configured
• Device and switch ports can be members of more than one DCC policy
- All devices are allowed to connect to all switch ports by default
11 :22:33:44:55:66:77:aa
aa:bb:cc:44:55:66:77:77
ff:ee:dd:44:55:66:77:ff

Footnote 1: The following restrictions apply when using DCC policies:

• DCC policies cannot manage or restrict iSCSI connections, that is, an FC
Initiator connection from an iSCSI gateway.
• You cannot manage proxy devices with DCC policies. Proxy devices are
always granted full access, even if the DCC policy has an entry that restricts or
limits access of a proxy device.
In this example we are allowing device with WWN 11 :.... :aa to connect on domain 1
ports 1 or 3. The [] are used to include the port numbers specified and the WWN of
any device currently connected on those ports. From the help file:
(1-6) =selects ports with index 1 through 6.
(*)=selects all ports on the switch.
[3,9] = selects ports with index 3 and index 9 and all devices attached to those
ports.
[1-3,5):: selects ports with index 1 though index 3 and index 5 and all devices
attached to those ports.
[*] = selects all ports on the switch and devices currently attached to those ports.
Revision 01 10 9 - 23
Device Connection Control (DCC) Policy (cont.)

• Devices in a DCC policy are only allowed to connect to
switch ports in the same DCC policy
• Devices not in a DCC policy may only connect to switch ports not in
a DCC policy1
• Devices can be initiators or targets
• When a DCC violation occurs, the related port is automatically
disabled 2
- In addition, an SNMP trap will be sent if SNMP trap listeners are
configured
- Disabled ports must be manually re-enabled using the portenable
command

Footnote 1: It is recommended a DCC policy is created containing all switch ports.

This will prevent devices that are not members of a DCC policy from being able to
access one another.
Footnote 2: DCC policies along with host authentication (AUTH) policies and
persistently disabling unused ports is an effective way to prevent host WWN
spoofing.
Configuring DCC Policies

• Steps to create a DCC ACL policy 1:
1. Create DCC policy using secpolicycrea te
• To create a DCC policy that automatically includes all attached devices
and the ports they are attached to use the [*] option
Secpolicycreate DCC_POLICY_ALL [*]
2 . Activate DCC policy using secpolicyacti vate
3. Distribute the DCC policy to switches configured to accept DCC

distribution using distribute command or Fabric Wide
Consistency policy

Footnote 1: DCC policies must follow the naming convention -OC_POLICY_nnn,"

where nnn represents a unique string. To save memory and improve performanoe,
one DCC policy per switch or group of switches is recommended .
Device ports must be specified by port WWN. Switch ports can be identified by the
switch WWN, domain ID, or switch name followed by the port or area number. To
specify an allowed connection , enter the device port WWN, a semicolon, and the
switch port identification.
The following methods of specifying an allowed connection are possible:
- deviceportWWN;switch WWN (port or area number)
- deviceportWWN;domain/D (port or area number)
- deviceportWWN;switchname (port or area number)
To create a DCC policy:
1. Connect to the switch and log in.
2. Type secpolicycreate " DCC POLICY_ nnn", " member ; ... ; member"
DCC_POLICY_nnn is the name of the DCC policy; nnn is a string consisting of
up to 19 alphanumeric or underscore characters to differentiate it from any other
DCC policies. Note: To create a snapshot DCC policy use"*" instead of a
member list.
3. To save or activate the new policy, enter either the secpolicysave or the
secpolicyacti va te command.
BROCADE
EDUCATION SOLUTIONS
IP Filter Policies
• Set of rules applied to management interfaces
• IP packets that are not in the rules are denied
- Example: to limit the permitted management stations
IP MGMT
IP Filter Policy
~ EQf1 ~
10.255.250.50 23 (Telnet) Permit

IP Filter Policies (cont.)

• There are two types of IP Filter policies
-1Pv4
- 1Pv61
• If an IP Filter policy is active, there is an implicit deny all for IP
traffic. Adding rules to the policy will enforce the specified
configurations.
• IPFILTER policies permit or deny the traffic through the IP
management interfaces according to policy rules 2
• Policy rules can be based on IP address, port number, or protocol
• There can be up to 6 different IP Filter policies defined 3
• Only one IP Filter policy for each IP Filter policy type (I Pv4 or 1Pv6)
can be active

Footnote 1: Fabric OS v5.3 and later loads default IP Filter policies for both 1Pv4
and 1Pv6.
Footnote 2: Policy rules will be discussed in more detail later.
Footnote 3: There are also two default policies, one for 1Pv4 and another for 1Pv6
and some implicit policies that enable communication like syslog, from the switch
out. More information on implicit policies will be covered in following slides.
Default IP Filter Policy Rules

There is an implicit
deny action for
every Dest Port not
BSlOO : admin> ipfilter --show default_ ipv4 on this list.
Name : default_ipv4 , Type : ipv4 , State : active
Rule Source IP Protocol Dest Port Action
1 any (SSH) tcp 22 permit
2 any (Telnet) tcp 23 permit
3 any (rpc) tcp 897 permit
4 any (Secure rpc) tcp 898 permit
5 any (sunrpc) tcp 111 permit
6 any (http) tcp 80 permit
7 any (https/SSL) tcp 443 permit
8 any (SNMP) udp 161 permit
9 any (sunrpc/UDP) udp 111 permit
10 any (NTP) udp 123 permit
11 any tcp 600 - 1023 permit
12 any udp 600 - 1023 permit
Protocol names
added to output

1Pv4 destination ports not explicitly defined in this policy will be blocked. Port 25 is
not listed so any smtp port 25/tcp Simple Mai l Transfer access to this
switch will be blocked.
When FOS v5.3 is initially loaded, two default IP Filter policies are activated:
• One for each policy type (default_ipv4 and default_ipv6)
• The default IP Filter policies cannot be deleted or changed
• When a user-defined IP Filter policy is activated, the default IP Filter policy
becomes deactivated
The Protocol names were added to the graphic.
A reference list of some protocol ports can be found in Windows at

c:/windows/system32/drivers/etc.
1Pv6
1Pv6 addresses consist of 128 bits (3.4 x 1Q38 total addresses), compared to 32 bits
(4.3 x 109 total addresses) allowed in 1Pv4.
1Pv6 addresses are represented as 8 colon-separated 16-bit hexadecimal digits.
Fabric OS supports static 1Pv6 addresses for management interfaces only.
Consecutive zeros may be dropped. For example, : 8 : is interpreted as : ooos :
and : : is : 0000 : An 1Pv6 unicast address: 2001 : DBS :: 8 : 800 : 2ooc : 417A .
Network Prefix: The netmask prefix is specified as the number of bits that comprise
the network portion of the address. The prefix is specified using a I followed by a
number. An 1Pv6 address and netmask: 2001 : DB8 :: 8 : 800 : 2ooc : 417A/64 .
Gateway: Default gateways will be learned from the network
When being configured as a local address, both an address and prefix MUST be
specified. The prefix is equivalent to a CIDR subnet mask in 1Pv4. There is no
implicit -alssfull" prefix length as in 1Pv4.
Breakdown of 2001 : DBS :: 8 : 800 : 2ooc : 417A/64
Network portion of address: 2 001 : DBS : o : o
Host portion of address: 8 : 800 : 2ooc : 41 7A
The prefix 64 is common for Ethernet LANs .
Because 1Pv6 addresses are so long, Fabric OS services have been upgraded to
allow for an 1Pv6 address or valid DNS name. Use the dnsconfig command to
configure a DNS server. The dnsconf ig command will allow for 1Pv6 addresses.
1Pv6 and DNS Support have been added to the following commands: aaaconf ig ;
configdownload; config upload ; dnsconfig ; fabricshow ;
fcrfabricshow ; firmwaredown lo ad ; ipaddrset ; ipaddrshow;
ipfilter ; seccertutil ; secfabricshow ; snmpconfig ; supportftp ;
supportsave; syslogdipadd; and tsclockserver . Web Tools support
has also been added for 1Pv6 management access. Secure Fabric OS IP policies do
not support 1Pv6.
IP Filter Rules
• An IP Filter policy is comprised of a set of rules
- Each rule has an index number identifying the rule
- There can be a maximum 256 rules within an IP Filter policy
• Each rule contains the following elements:

- Source Address: A source IP address or a group prefix
- Destination Port: The destination port numb,e r (0 through
49151) or name, such as: Telnet, SSH,
HTTP, HTTPS1
- Protocol: The protocol type. Supported types are TCP or UDP
- Action: The filtering action taken by this rule, Permit or Deny

Footnote 1: For an IP Filter policy rule, users can only select destination port
numbers in either the well-known or the registered port number range, between 0
and 49151 , inclusive. This means that customers have the ability to control how to
expose the management services hosted on a switch, but not the ability to affect the
management traffic that is initiated from a switch. A valid port number range is
represented by a dash, for example 7-30. Alternatively, for some -st.p:>rted
services", service names can also be used instead of port numbers.
TCP and UDP protocols are the only valid selections. Implicitly, ICMP type 0 and
type 8 packets are always allowed to support ICMP echo request/reply on
commands like ping and trace route .
For the action, only permit and deny are valid.
Revision 01 10 9 - 31
IP Filter Rules (cont.)

• An active IP Filter policy is applied to the IP packets entering
through the incoming management interface1
- When a packet arrives, it is matched against rules in a filter in top
down order until a match is found
- If a match is found against source IP, destination port, and protocol the
action permit or deny for that rule is taken
- If the packet does not match any of the rules the default action is to
deny

Footnote 1: If a match is found against the first rule source address, destination
port and protocol, the corresponding action for this rule is taken, and the
subsequent rules in this policy will be ignored. If there is no match, then it is
compared against the next rule in the policy. This process continues until the
incoming packet is compared against all rules in the active policy.
If none of the rules in the policy matches the incoming packet, the two implicit rules
will be matched against the incoming packet. If the rules still don't match the packet,
the default action, which is to deny, will be taken.
IP Filter Rules (cont.)

• For every IP Filter policy, the following two rules are always
implicitly appended to the end of the policy (will not see this in the
policy)
Source Destination
Protocol Action
Address Port
Any 49152-65535 TCP Permit
Any 49152-65535 UDP Permit
• This ensures TCP and UDP traffic to dynamic port ranges is

allowed , so that management IP traffic is not affected (syslog,
RADIUS and FTP)

These implicit rules ensure that needed management ports are left open.
• The Well Known Ports are those from 0 through 1023.
• The Registered Ports are those from 1024 through 49151
• The Dynamic or Private Ports available are those from 49152 through 65535.
Configuring IP Filter Policies

• IP Filter policies are managed using the ipf il ter command
• To create an IP Filter policy 1:

ipfi1ter --c1one <po1icyname> -from defau1t_ipv4
- Best practice is to create an IP filter policy as a clone of an existing

policy
• <policyname> is a unique string composed of maximum 20

alpha, numeric or underscore characters
• The name default is reserved (default ipv4 and

default ipv6 )
Footnote 1: The ipf il ter CU can also be used to show, clone, save,
activate, and abort IP policies. Additionally, use the command to add or delete
rules that will filter IP traffic.
B5100 : admin> ipfi1ter he1p
Usage : ipfilter
--help: display the ipfilter synopsis
--create <poiicynarne> -type <ipv4 I ipv6>: create an IP filter
policy
--cione <policynarne> -from <src_policyname>: create an IP
filter policy as a copy of existing policy
--show [policynarne]: display one or all IP filter po l icy
--save [policynarne] : save one or all IP filter policy
--activate <policyname>: activate an IP filter policy
--delete <poiicynarne>: delete an IP filter policy
--addrule <policynarne> -rule <rule_number> -sip <source IP> -
dp <dest_port> -proto <protocol> -act <permit I deny>: add a
rule to an IP filter policy
--delrule <policyname> -rule <rule number>: delete a rule
from an IP filter policy
--transabort: aborts an open IP fi l ter transaction
Revision 01 10 9 - 34
BROCADE
EDUCATION SOLUTIONS
Advanced Device Security (ADS) Policy

• Restricts device access to the fabric at the Access Gateway
- Similar to the DCC Policy
• Only specified hosts are allowed to login to the Fabric
- Allowed host login can be configured per F_Port by specifying their
Port WWN
11 :22:33:44:55:66:77:aa
aa:bb:cc:44 :55:66:77:77
ff:ee:dd:44 :55:66:77:ff
ADS_POLICY
F_Port WNNs Allowed
11 :22:33:44:55:66:77:aa

Advanced Device Security Allow List

• Enabling ADS policy creates a default entry list of devices for each
F_ Port in the ADS policy
• The list allows a set of port WWNs to login to the specified F_ Ports
- Each list can also be configured to ALL ACCESS ("*" ) or NO
ACCESS ("") 1
• After enabling the ADS policy, by default, every F_ Port list is
configured to ALL ACCESS
• Unauthorized access is rejected and user is informed through
RASLOG
• Existing device logins are not disturbed
• Only configurable from CLI (no Web Tools or DCFM support)

Footn ote 1: ALL ACCESS allows for any device to attach while NO ACCESS
denies all devices to connect.
Advanced Device Security Allow List (cont.)

• ADS is disabled by default
• To enable: ag --pol i cyenable ads
• By default all ports are set to ALL ACCESS
• Creating an ADS policy behaves the same way as a DCC Policy,
allowing defined devices, while denying undefined devices
B300 : admin> ag --adsshow

F Port WWNs Allowed
0 ALL ACCESS
1 ALL ACCESS
2 ALL ACCESS
3 ALL ACCESS
< output truncated>

--ads show Displays the current configuration of the Access Gateway. This
includes all N_Ports and F_Ports that are currently online, failover
and tailback settings as well as any online F_Ports that are currently
mapped to N_Ports. Failover and tailback policies are displayed as
enabled (1) or disabled (0).
Configuring ADS Policies

• Set the ADS policy allow list of the specified F_ Ports
B300 : admin> ag --adsset "1;2"
11
20: 03: OS: 00 :88:35:a.O:12;21: 00: OO:eO: Sb: S8: 01: Sb"
WWN l i st set successfull y as the Allow Lists of the F_ Port(s]
• Add devices to the ADS policy Allow List for specified F_Ports
B3 00 : admin> ag --adsadd "1;2" "50:00:00:e0:8a:88:02:al"
WWNs added successful ly to Allow Lists of the F_ Port[s]
• Remove devices from the ADS policy Allow List for specified F_ Ports
B300 : admin> ag --adsdel "1"
"20:03:0S:OO:S8:35:a0:12 ; 2l:OO:OO:eO:Sb:S8:01:Sb"
WWNs removed successfully from Allow Lists of the F_Port(s]

--ads set +
_Port[;F_Port2; ... ]"''WWN[;WWN2; ... ]": Sets the list of devices that are
allowed to log in to the specified F_ ports. Devices are specified by their World Wide
Names. Lists must be enclosed in double quotation marks. List members must be
separated by semicolons. The maximum number of entries in the allowed device list
is twice the per port maximum login count. Replace the WWN list with an asterisk (*)
to indicate all access on the specified F_Port list. Replace the F_Port list with an
asterisk (*) to add the specified WWNs to all the F_Ports' allow lists. A blank WWN
list(~ indicates no access. ADS policy must be enabled for this command to
succeed.
--adsadd + _Port[;F_Port2; ... ]""WWN[;WWN2; ... ]": Adds the specified WWNs to the
list of devices allowed to log in to the specified F_ports. Lists must be enclosed in
double quotation marks. List members must be separated by semicolons. Replace
the F_Port list with an asterisk. (*) to add th,e specified WWNs to all the F_Ports'
allow lists. ADS policy must be enabled for this command to succeed.
--adsde l -£Port[;F_Port2;... ]"''WWN[;WWN2; ...]": Deletes the specified WWNs

from the list of devices allowed to log in to the specified F_ ports. Lists must be
enclosed in double quotation marks. List members must be separated by
semicolons.
Configuring ADS Policies (cont.)

• Set Allow list to "NO ACCESS"
B300 : admin> ag --adsset "11;12;13u "u
WWN list set successfully as the Allow Lists of the F_Port[s]
• Set Allow list to "ALL ACCESS"

B300 : admin> ag --adsset "12" "*"
WWN list set successfully as the Allow Lists of the F_Port[s)

Configuring ADS Policies (cont.)

• Updated allow lists
B300 : adrnin> ag --adsshow
F Port WWNs Allowed
0 ALL ACCESS
1 50 : 00 : 00 : e0 : 8a : 88 : 02 : al
2 20 : 03 : 08 : 00 : 88 : 35 : a0 : 12
2l : OO : OO : e0 : 8b : 88 : 01 : 8b
50 : 00 : 00 : e0 : 8a : 88 : 02 : al
3 ALL ACCESS
4 ALL ACCESS
5 ALL ACCESS
6 ALL ACCESS
7 ALL ACCESS
8 ALL ACCESS
9 ALL ACCESS
10 ALL ACCESS
NO ACCESS
I::
14
ALL ACCESS
NO ACCESS
ALL ACCESS
15 ALL ACCESS

BROCADE
EDUCATION SOLUTIONS
ADDITIONAL RESOURCES (
Available Fabric OS Security Web Based Training

• SFO 102 Secure Access Methods to Brocade Switches
• SFO 103 Introduction to Security in SANs with Access Control Lists
• SFO 104 Security Features Introduced in FOS v5.3
• SEC 112 Securing Your Brocade Fabric: Restricting Administrative
• SEC 113 Securing Your Brocade Fabric: Restricting Switch and

Device Access

Al R1gh's Reserved
Visit the Brocade Education website for additional information.

www.brocade.com/education
To register for courseware visit http://learning.brocade.com
Summary
• ACLs can be created to enforce f.abric security. Policy distribution
rules can be used to ensure desired levels of consistency of the
ACLs throughout the fabric.
• Fabric Configuration Server (FCS) policies create a list of
designated switches allowed to make changes to fabric-wide
services
• Switch Connection Control (SCC) policies are lists of switch WWNs
allowed to join the fabric
• Device Connection Control (DCC ) policies list device WWNs that
are allowed to login to specific ports
• IP Filter (IPFILTER) policies control IP traffic into the management
ports of the fabric switches

Summary (cont.)
• Fabric Element Authentication Policy (AUTH) creates PKI
relationships between fabric switches and devices using DH-CHAP
orFCAP
• Advanced Device Security (ADS) policies for Access Gateway list
devices that are allowed to login to specific ports

BROCADE
EDUCATION SOLUTIONS
BROCADE
EDUCATION SOLUTIONS
Authentication (AUTH) Policies

• Fabric Element Authentication policies (AUTH) authenticate
switch-switch and/or device-switch connections
• Fabric OS v5.3.0+ supports DH-CHAP and FCAP protocols for
authentication
- Protocols use shared secrets and digital certificates to authenticate
- The switch may be configured to negotiate FCAP, DH-CHAP, or both 1
• The AUTH policy supports two types of authentication policies:
- E_ Port and EX_Port2 authentication (used for switch-switch)
- Device authentication (used for device-switch)
• All fabric element authentication configurations are performed on a
local switch basis
- Fabric-wide distribution of the policy is not supported

Al R1gh's Reserved
Footnote 1: You can use the command: authutil - - set <fcap I dhchap> to
set the authentication protocol which can then be verified using the command
authutil -show.
By default, the switch attempts FCAP authentication first and DH-CHAP second.
FCAP - Fibre Channel Authentication Protocol
Footnote 2: Authentication of extended ISLs between the edge and backbone

switches is considered peer-chassis authentication. Authentication between two
physical entities is required , so the extended ISL which connects the two chassis
needs to be authenticated. The certificates or shared keys must be installed on both
the edge and backbone switches. There is no configuration required for the front
and translate domains.
Revision 01 10 9 - 49
E Port Authentication
• Used for switch-switch authentication
- A secret key pair (DH-CHAP) or PKI certificates (FCAP) nave to be
configured/installed prior to activating the policy1
• E_ Port authentication supports four modes2 :
- On - Strict authentication is enforced on all E Ports
- Active - Tolerant state, can connect to a switch with any type of policy
- Passive - Default state, does not initiate authentication but will respond
- Off - Policy is off, no authentication supported
~ & ~
Certificate Certificate

Footnote 1 : If the PKI certificates are not installed prior to activating the policy,
authentication will fail and the link will be segmented.
Footnote 2 :
ON: Setting the AUTH policy to ON means that strict authentication is enforced
on all E_Ports. If the connecting switch does not support authentication or the
policy is switched to the OFF state, the ISL is disabled. During switch initialization,
authentication begins automatically on all E_Ports. In order to enforce this policy
fabric-wide, the fabric needs to have Fabric OS v5.3.0 switches only. The switch
disables the port if it is connected to a switch which does not support
authentication. Regardless of the policy, the E_Port is disabled if the DH-CHAP or
FCAP protocol fails to authenticate each other.
ACTIVE: In this state the switch is more tolerant and can connect to a switch with
any type of policy. During switch initialization, authentication begins on all E_ Ports,
but the port is not disabled if the connecting switch does not support
authentication or the AUTH policy is turned to the OFF state. The authentication
begins automatically during the E_Port initialization. A switch with this policy can
safely connect to pre-v5.3.0 switches, since it continues E_Port initialization if the
connecting switch does not support authentication. Regardless of the policy, the
E_Port gets disabled if the DH-CHAP or FCAP protocol fails to authenticate each
other.
Revision 01 10 9 - 50
PASSIVE (default): In the PASSIVE stat,e the switch does not initiate
authentication, but participates in authentication if the connecting switch initiates
authentication. The switch will not start authentication on E_Ports, but accepts the
incoming authentication requests, and will not disable if the connecting switch
does not support authentication or the policy is turned to the OFF state. This is the
safest policy for switches connecting to pre-v5.3.0 switches. That means v5.3.0
switches can have authentication enabled and this will not impact the pre-v5.3.0
switches. By default the pre-v5.3.0 switches act as passive switches, since they
accept incoming authentication requests. Regardless of the policy, E_Port is
disabled if the DH-CHAP or FCAP protocol fails to authenticate each other.
OFF: This setting turns off the policy. The switch will not support authentication
and rejects any authentication negotiation request from another switch. A switch
with the policy turned OFF cannot be connected to a switch with the policy tumed
ON. The ON state is strict and disables the port if any switch rejects the
authentication. DH-CHAP shared secrets must be configured before changing the
policy from the OFF to the ON state.
The behavior of the policy between two adjacent switches is defined as follows: If
the policy is ON or active, the switch will send an authentication negotiation
request to the connecting switch. If the connecting switch does not support
authentication or the policy is OFF, the request will be rejected. Once the
authentication negotiation succeeds, the DH-CHAP authentication will be initiated.
If DH-CHAP authentication fails, the port is disabled and this is applicable in all
modes of the policy.
Device Authentication
• Device authentication policies can also be categorized as F_Port,
node port or HBA authentication policies
• Check vendor HBA compatibility matrices for protocol support1
• Device authentication on the switch supports two modes2 :
- Off- Authentication is not required
- Passive - Authentication is optional
Certificate

Footnote 1: Supported HBAs: (See compatibility matrix for latest information)

• Emu lex LP11000 (Tested with Storport Miniport v2.0 windows driver)
• Qlogic QLA2300 (Tested with Solaris v5.04 driver)
• Brocade Fibre Channel HBA models 415, 425, 815 and 825
Footnote 2: The following are the available policy modes and properties:
OFF (Default): Authentication is not required. Even if device sends FLOGI with
security bit set, switch accepts the FLOG I with security bit OFF. In this case,
switch assumes no further authentication requests from device.
PASSIVE: Authentication is optional. If the attached device is capable of doing1
the authentication then the switch participates in authentication; otherwise it will
form an F_Port without authentication. In PASSIVE mode, an F_Port will be
disabled if the HBA shared secret does not match with the secret installed on the
switch. If the secret provided by the switch does not match the secrets installed on
the HBA then the HBA will disable the port on its side. On any authentication
handshaking rejection , the switch will disable the F_ Port with reason
-Atlnentication rejected". Since the F_Port authentication requires DH-CHAP
protocol, selecting the PASSIVE mode will be blocked if only FCAP protocol is
selected as the .authentication protocol. Similarly de-selecting the DH-CHAP
protocol from the authenticat ion protocol list will be blocked if the device
authentication is set to PASSIVE.
Revision 01 10 9 - 52
Setting AUTH Policies

• To configure E_Port Authentication:
- authu t il - - policy - sw <onlac t ive lpassiveloff>
swit ch : admi n > authutil --policy -sw active

Warning : Activating the authentication policy requires eith er DH-
CHAP secrets o r PKI ce r tificates dependi ng o n the protocol
sel ected . Othe r wise , I SLs will be segmented during next E-port
bring-up . ARE YOU SURE (yes , y , no , n) : [no] y
Auth Policy is set to ACT I VE
• To configure Device Authentication:

- authutil -- policy - dev <off I passive>
- See example in the notes

switch : admin> authut il --p oli cy - dev passive

Warn ing : Activating the a u thenticat i on policy requires DH-
CHAP secrets on b oth swi tch and device . Otherwise , the F-port
will be disabled during next F-p ort bring-up . ARE
YOU SURE (yes , y , no , n) : [no ] y
Device authen ticat i on is set to PASSIVE
BROCADE
EDUCATION SOLUTIONS
DCC Handshake
• During the login, the PWWN of the device is checked against the
DCC policy and either permitted or denied access

BROCADE
EDUCATION SOLUTIONS
Configuring IP Filter Policies in Web Tools

DEV_ ST01_0CX - Web Tools
o r- 0 Powe< 0
~
~
~
~
~
~
~
~
~
~
-
Sw1tchAdm1n
'!.'I DEV_ST01_DCX - Switch Administfdtion
PonAdmln
gB Admin Donulln
- - --- ·-··----··-·-··-· -··-·-·-·-·-·-- · -·-·-·--·-·------- - ...... -- ... _.. .... -·
1<t; FCR ::.1 DEV_ ST01_ DCX - Switch Administ ration
M Fabric IV
~ow Basie LI~
Wontor
IZI Perform Sw•chllama oev_STOl_OCX DomanO 4(0X4) WIMl 1000000S 1e43.1800 ft.'Od 09 2009 20 09 3S GUT-00 00
D Name Se Configure Rou1J1i1 Ex!ended Fabnc
PV6
Status
Ad.Ne
r
PV• Aar1e

Al 1R ghts Reserved
Configuring IP Filter Policies in Web Tools (cont.)

• Best practice is Conl\gol<e
Swld1
AOU11n9 -
llelwO<I<
Flbne AAA Serva
Fnnwlle Dow.-
T...:e FICOH CUP Secuny Poicles
u- Trunu
to Clone one of Sectlrty- ~ : Pfaltf PolcyAdttwustralJOn
Create Poley Show Poley Clone Poley

the default policies ACL
fWCP
and edit it to match AU1Mnliution

A~ts OtstroutJOn "
!..1 Clonr Policy '
IPSec Poltcles Currtent PollCy llal!'e defaut.. '!'_••
your corporate IPfllter
fle< llal!'e Type
Ell>emel IPSec
delaul_~
de!au ...pv•
PV6
PV• j
En:er Clone PollCy nal!'e 11ew d: =:J
=:J J
guidelines
!:••eel
Configure
Swtell
Seeurty Polleles ~ I: P Fler Poley Adnwi istrabon
ACL Create Poley ... Edi Poley ... Sh!P'"'°ley ...

FWCP 1
Authentication /
Accepts Ooslri>ubon f!1
IPSec Policies
:!lll"f N•ll"" !JVD<" ~.iw.
IPfiller NEW_ipv4 P\14 Oeli\ed
Ethernet IPSec defau._.,"6 PV
'"'6,,....---..,..
ACIN
~e--..
1defau1_~4 P\14 Acwe

Al 1R ghts Reserved

Configure Roumg Extended Fabne AAA Se<vice FICOH CUP
ell Nelw
I Seairfy Po1aes ~v p Fler Poley Adm.s1ra11on
ACL Create ~ Edt Po snow Polley •..

FWCP
_J
~!,J Edit JP Fdtel' Pol11CY x
- pts O.stti>ution f2I
~ame Type Status
Defiled
Acllve
:..1 Add Rule
1 I A)'
2tny Rllle Order
lany RPCOl891
4 H)' SfCURERPC Source P Address l t92. 168 I s(j
'""
--
Sany SU-PtRPC/111 ICO
..,.
-
6 1fl)' Wl/AWSQ
7 1ny !llTPS/U) lcp
Protocol lcp
8 1ny SllUPll61
""•
..,.
..,.
-
9 •ny s.~mRPCt111
ACIJOn
10 any 123
II ony 600-1023 lcp
121ny 600-1023
""•
I Sa•• Pilley I '""«!


• Web Tools can be used to Edit , Show , Delete , Clone
(copy) , Activate or Distribute IP Filter Policies
P Flier Pollcy - . 0 0 0
Create Poley ... Edi Poley ... Show Poley ... Detele Poley acne Polcy .. Adlvele Poley
~ Oistrb.tlon 0
tems: 3 tern Seleded: I
Flier Nonie
defOUll_lpv 4 PV4
defOUll_lj)v6 PV6 01str1bute Pohcy Conf1gurat10n
~Show IP Filler Pohcy
0 Ob:lrb.te to el the P FAer $141P0<1ed swtclle$
I Swtches For Setecllon
:s0
18
10:00;00:05.1 •d13".e«e«,RSl.6-ST03-"20(£)ft
moo.oo:os.10:11':24'8c(RSl6-ST03-4100)
10:00:00:05:1e:36:2d2(RSL6-ST03-4$()
010:00:00:05:1o:90:00:7~6.ST03-5000)
TB.r.ETl23
~7
~
SUffC/111
WWWll!IJ AddAI>>
HTTPSJ443
SUffC/111
123

Creating ACL Policies with Web Tools

itgi ·ii§i l1fj:£iWftib f'fi@M§Mii!.!.
S w t - R1 1-ST02-8S1 OomolnD: 1(0x1) \/WllN; 10-00:00:05 1e:7e dl.91
COnft!µe RO<Alng Eldencled F-.C AAA 5erY'ce Trece f1CON 0.. _ ,,y -
Swlch Netwon Frmwere ~...- • __. Ucense
AQ - . .,,, ConneclJon CO<Vol ~
Re, L Pofuy Conftgurntion W1zdrd £1
°"'
Acee
Pole)
Seledl'olcy(s)
1. Select l'lllity
r·
IPFitte,- f -
Ethc rnct IPSoc 2. Edit l'lllicy
l sec
C 2009 Brocade Co'Timun ca1 ans Systems nc 6•

Al 1R ghts Reserved
Creating SCC Policies in Web Tools

x
l. Select Policy TNs - creMe a secP<>lcy """*'II el SWichos In the fObrlc.

This cen be done ony when there ls no sec P'oicy c:tefhed on tho .swtcn.
I~=:n Jt----
3. DCC Policy
Defnod P<>lcy S<lt
Co1u1f&W'3tion
4 . FC S Policy S\Naches f or ~!On
Conli&w-ation '°
<>
~Q!Ml!!,O'IJ'!Jltd~-ll
1 0:00.00:05..1e:0er;6f Ta(R11.ST02..e30)
5. Coalflnnation
.......

Creating DCC Policies with Web Tools

All Pohcy Conr1guh-.t1on Wi.zard El
nu wtl creala a ~ poky for oech port il the fetric loc:ldng II down to t he device cornactnd or
l. Selec t Pollcy creeOng en ~ policy to dselow tJnf device to be connected to c Thb cen be done ontv when
2. s ec Polley
Co n.fig uration
~- DCCPollcy f--~-..=::=~~==,
1 lhef-e ere no oth« DCC polcies defined on the swtlch.
Creele
I FCSPolley
• Conllgw '1tloll _
4.
DCC Policy Configuration Dialog
- - - -
Con .fig uration
S. Cottllrmatlon
DCC Policy Name DCC POLICY name
Sw i ches & Device Members

B @ Devices
ID 1o:oo:oo:os:1e·s1:1c:
El 22:00:00:04:cf.92;5c:1
El 22:00:00:04:c t bd:89:!
B ~ Sw«ches
8 Q 1o:OO:OO:os·1e:Ob:ea:
Iii Porto - 11dex Ii o
Iii Port 1 - 11dex # 1

Creating FCS Policies with Web Tools (cont.)

ACL Pohcy Conr1guratmn Wizard El
l. Select Policy 0 Ttis wt aeete a FCS Policy heWlg el swlctoes W'l lhe IDie.
Ttis can be dooo>only wh!n there ls no FCS Poley defnect on Ille swlch.
2. sec Policy n-.r--.t4 ~ <:M

1151@MC®Htttt!1m@,@ I;
Configuration
3. DCC Policy
Configuration
1.$'TO'l.SSl(1, 10JXlOQ_05.le..7e..dl91)
1.sTD2~, IQOOOOO:S..le.Dllf.il le)
5. Confll1.lliltion

DCFM Fabric Binding (SCC)

For Fabric OS devices, enabling Fabric
~ i;:<M 'fjf!w
Binding activates Switch Connection
4) 0 a-rt Mpger
1--- - - - - -.--! FC S:t((cn
\llewAIY
Control (SCC) policy and sets Fabric
~Swilch
f"nnware~
Wide Consistency policy to strict and
R2Wl!I •
Al_L~
_ °'*
~-----~ LoglcelSWCcties insistent domain ID.
8 <P rsH 1_$12 i;:nctyptlon
s @swcc:n
13 VR11-ST02· 351~-----'-I' l'~'NNH ~SltelA ~Ac:Clot\
1000;oOos.1E:OA. ~- ..,--. -~,.lbne8frica'lg
20:01:
tO:Gloa:coao ~ t' ~r.1ine~
Ji 20:02:Wif!.I""~
20:0A;
~ 2200:00
Ji 20:05:00:
.
8 0 .(p rs111 _st2_m44 1--
[ICCN
------1
El 0R11-ST02-MCC Ml!mberlhPLal Oii r9" l_.1(2~4
4 10:00:00:05. 1 Alow*'rolti Malt\!!. Nenie _. Node'WllH DDrMn 0
0 21;00:00:20:371--
Potl
...;;._
Gf_....,.
_ _ _---! lt11.Sl0:2.830 1000.0Q:OS: 2 ,.,,_.st2 10-~ ft1 t .ST02.830 tO;OQ.OO;OS; 2
fCTr~ R1t-Sl02-8$1 1000.000S:
R11-sT02,,W.W 1000.0900
'
91(1) ...,,_...........
t#11J(Z tel ft1t-Sf02....... 10.00.oe-m. 97(1)
f'Tr~
<4
I
~====1 ·

Replicating Security Policies with DCFM

;{view All - DCFM 10.3.2
i-=-~~--=--
FC~cn
lools
g:E Swtcn
_, 11$
.
1-=:.:.::..:.~~__:•..,........,,.._~
•
~
Restore
____,r_., Name
FWmware !!janogemonl o;;onr'!l"ellon Repostory

Rpng • Sche<Uo ~
A ,.........__ _ _____, L.og1oa1 swcc:nes
_ l_Lewta
El rsl11 _st2
El t9 swccn 0r....,
. '
L Overview
2. Configuration ; { Replicate Switch Secunty Policy (onftguration

E T)pe
Steps
3. Select Source Select the source swtcn below. T1he selected swtcn wl provide the con•
l. Overview To modify the confogurel<ln one swccn, select e SWllcn end did( Modify t
Switch
Con Cnnfi•mmlin
4. Select
Type
IEl o ~ ,.,11 _s«)S rstl1_st05
0 R11·ST11-830 SWlcn DOif
======t=========~ 3 . Select Source
R11.ST11-830
0 R1 1.sr11.es1 R11-sr11-es1 swccn oou
Switch
4. Select

Select Configure > FC Switch > Replicate > Security.

OR
Right-click a device and select Configuration > Replicate > Security.
The first step of the wizard, Overview, displays. There are seven steps in the
Replicate Switch Security Policy Configuration wizard:
Overview, which describes the wizard.
Configuration Type, which allows you to select the type of configuration you wish
to replicate.
Select Source Switch, which allows you to select the source device of the security
policy configuration you wish to replicate.
Select Destination Switches, which allows you to select the destination devices.
Only devices that can accept the selected security policy configuration display.
Validation , which lists the configuration settings that you can validate before you
replicate.
Summary, which lists the replication settings that successfully ran on all the
selected destination switches.
CFP380 Internal Use Only FCIP Appendix
Objectives
• After completing this module, attendees will be able to discuss:
- FCIP concepts and terminology
- FCIP infrastructure
- FCIP as a means of extending the SAN beyond the physical
boundaries of the Data Center

FCIP - Usage Drivers

• Applications for FCIP include:
- Backup, consolidation, mirroring, business continuity solutions
- Longer-than-FC-distance connectivity1
- Existing IP-based networks connect two sites, but not dark fiber
- FICON connectivity over long distance
• Fibre Channel continues to be the dominant choice for Storage
Area Networks (SANs)
- Fibre Channel provides a reliable , high-speed , and low-latency
transport for SCSI Initiators and Targets

Footnote 1: Distances that utilize native FC can span 500km; these solutions
incorporate dark fiber, C/DWDM, and form a single fabric.
For additional FCIP details, reference RFC 3821 - Fibre Channel Over TCP/IP
(FCIP).
Brocade does not recommend FCIP for use in every distance extension scenario:
no technical solution can be all things to all people. FCIP has inherent performance,
reliability, data integrity, and manageability limitations when compared to native FC
solutions. Delay and packet loss may create bottlenecks in IP networks. FCIP can
support very long distances, as long as the carrier network is extremely high
performance and reliable. FCIP is typically deployed when long-haul applications
are not business critical, and do not need especially high performance. FCIP may
not be suitable for tape, since tape usage will often fail i·f packets are dropped. In
addition to its performance limitations, FCIP troubleshooting and performance
analysis requires evaluating all aspects of the IP LAN and WAN networks in addition
to all FC nodes, switches, and routers, which can make it more complex to manage
than other extension options.
FCIP - Overview
• The Fibre Channel-over-IP (FCIP) protocol connects Fibre
Channel switches over a IP-based network
- IP packets generated by an FCIP-compliant port are 1Pv41 compliant,
so that they can navigate any IP network to reach the destination
end point
- Implementation uses standards-based TCP, it interoperates with
regular network equipment
Server
FCIP Platform
FCIP
(Bladed)
Platform

Footnote 1: Fabric OS v6.0 adds support for 1Pv6 addressing.

The TCP/IP stack (Presto) complies with following standards:
• RFC793 basic tcp state machine
• RFC879 maximum segment size (mss) selection
• RFC1191 path MTU detection
• RFC1323 window scaling and timestamp option
• RFC2018 SACK option
• RFC2581 congestion control (slow start and congestion avoidance)
• RFC2873 processing of 1Pv4 precedence field
• RFC2988 computing retransmit timer
• RFC3517 SACK-based loss recovery
• RFC3782 NewReno fast retransmit I fast recovery
In general, a Brocade FCIP solution is compliant with all relevant sections of host
requirements RFC1122 link layer and ARP (section 2), IP/ICMP (section 3) and UDP
and TCP (section 4).
FCIP - Overview (cont.)

• FCIP is a tunneling protocol that allows a transparent
interconnection of geographically distributed SAN islands through
an IP-based network
- Allows remote disk access, tape backup, and live mirroring
• From the fabric view, an FCIP link is an ISL, transporting all needed
FC control and data frames between switches - the IP network is
invisible to the fabric
• The FC fabric and protocols are invisible to the TCP/IP network
Server
FCIP Platform
FCIP
(Bladed)
Platform
Server

FC traffic over an IP network

• Interconnection of islands of Fibre channel storage area networks over IP-
based networks
• Distance Extension over IP LAN/WAN/MAN
FCIP Infrastructure - Brocade 7500E SAN Router

• The Brocade 7500E SAN Router is an entry level 1U platform that
supports advanced FC Routing and FCIP functionality
- 2 GbE ports - FCIP connections (VE_ Ports) with one tunnel per port,
data compression , and routing (VEX_Ports)
- Redundant hot-pluggable power supplies, fans, SFPs
- Supports non-disruptive firmware activation
- License upgradeable to full Brocade 7500 capabilities

Al 1R ghts Reserved
7500E Upgrade
Capability 7500 7500E
license
" ,, -{'. ,·,
Redundant Power Supplies and Fans :ij; -
Hardware-based Encryption .. ~ -
FC Tape Pipelining (over FCIP) . ·~ - "'
FICON (Disk and Tape)
,,
-** -** "'
-**
FC-based Extension with FastWrite TM ~ -
Qualified for local FC swiitching /;(; -
"'
FC Routing between fabrics
'
''
-~ ~ - "'
Call Home ·~ - "'
FC Routing for Fault Isolation '·'ti: "'.
Per GE Port Rate Limiting (Throughput
( l.'ptofG . \l;'J. 9iR;!"'f• • Up,to ilG
Throttling) J
f,..
~11
#of FC Ports ..... 18 '" I,··
2 ,
.··;l"~Ti:&.;; .c.. ....
;:;·:;;;~~z..~ ~
Connections or Tunnels (Remote Sites) .~·~·.~ llf\'-'9'.~'f ~j
Hardware-based Compression ~r.Jir; -
Open Systems Extension w/ FastWrite™ over
fl
"' -
FCIP
Storage-Optimized TCP ~
"' -
"'
FCIP Infrastructure - Brocade 7500 and FR4-18i

• The Brocade 7500 SAN Router and the Brocade
FR4-18i Director Blade both support all the FCIP
features discussed:
- Two 1 Gbit/sec GbE ports - Up to 8 tunnels/port,
VE- Port or VEX- Port1
- Per tunnel , can enable SACK, hardware compression,
and traffic shaping

The Brocade 7500 SAN Router and FR4-18i Director Blade are both designed for
FC Routing and FCIP solutions. Both platforms provide 2 Gigabit Ethernet ports (1
Gbit/sec) that support FCIP with 1-8 FCIP tunnels per port. For each tunnel, you can
enable SACK, hardware-based compression, traffic shaping, and FC routing.
Footnote 1: Can be either VE_ Ports or VEX_Ports within the same GbE port.
FCIP tunnels can be created between either of these platforms, but not between a
BrocadeAP7420 and either a Brocade 7500 or FR4-18i. The port hardware used to
implement the FCIP functionality is different.
Revision 01 10 10 - 7
Supported Platform-to-Platform Connections

• Supported platform-to-platform connections are:
- Brocade 7500(E)/FR41-18i VE_Port ~ Brocade 7500(E)/FR4-18i
VE Port
- Brocade 7500(E)/FR41-18i VEX_Port ~ Brocade 7500(E)/FR4-18i
VE Port

FCIP - Protocol Mapping

• FCIP transports FC frames across an IP network through an FCIP
tunnel, which is established between the two endpoints of the FCIP
connection
• FCIP is layered on TCP, so FC frames are encapsulated into TCP
packets
FCIP Tunnel
FC-IP i=-=11 FC-IP
FC-2 I I! TCP 11 TCP 11 FC-2
FC-1 I~ IP ij IP II FC-1
FC-0 I I! LINK I! LINK 11 FC-0
~ ij
I
To Fibre Channel
PHY
\.
v
PHY
I I
To Fibre Channel
IPNetw~

After FC frames destined for devices at the remote side are encapsulated into TCP
packets, a standard IP header is added to each packet. The packet is then sent to
the next hop (usually an Ethernet router).
FCIP Concepts and Terminology

• Before an FC frame is sent out vi.a FCIP over a Gigabit Ethernet
link, the transmitting FCIP port encapsulates the FC frame in the
payload of each of the four protocols in the stack: FCIP, TCP, IP,
and Ethernet
• The receiving FCIP port strips the Ethernet, IP, TCP, and FCIP
headers; reassembles the FC frame (if it was fragmented); and
forwards the FC frame into the FC fabric
• FC Frame

Brocade's FCIP implementation is compliant with applicable IETF TCP/IP

standards.
Brocade leverages T11 FCIP standard (RFC 3821) mechanisms.
Frame information:
• Ethernet frame format: 6 bytes for the destination address, 6 bytes for the
source address, 4 byte optional 802.1q VLAN Tag, 2 byte length/type. Ethernet
frames also use 4 bytes for CRC trailer.
• At a minimum, IP has 20 bytes of header, no bytes of trailer. There are at least
two IP header formats , one for 1Pv4 and another for 1Pv6. Brocade uses the
1Pv4 header.
• At a minimum, TCP has 32 bytes of header, no bytes of trailer. It is transmitted
in packets that may not occur on frame boundaries, so the header may not be
at the beginning of a frame. It generates the appearance of a virtual byte
string.
• TCP over Ethernet: Assuming no header compression (e.g. not PPP) add 20
1Pv4 header; add 20 TCP header; and add 12 bytes optional TCP timestamps
• FCIP is specified in RFC 3821 (ftp : I /ftp . rfc - edi tor . org/ i n -
notes/ rf c3821 . txt ). It uses the encapsulations specified in RFC 3643
(ftp : I I ftp . rf c - edi tor . org I in - notes/ rf c3 64 3 . txt ). That protocol
specifies a 24 byte header and a 4 byte frame begin marker and a 4 byte
Revision 0110 10 - 10
frame end marker. The encapsulated frame itself contains a 24 byte frame
header and a 4 byte CRC, plus possible pad bytes and possible optional
headers. It may contain up to 2112 bytes of data between mandatory header
and CRC, so you can see that it overlaps the standard Ethernet frames, which
are like 1018 bytes max.
• Then the FCP SCSI information units have an additional header on them, but
there is only one header for a multi-frame unit. That is specified by FCP-3
(found in the drafts section of www . t 1 o. org ). That header information varies
significantly, since the FC frame header (encapsulated in the FCIP) contains
control information that tells what the format is.
• For additional Ethernet h,eader, trailer information see
http : //sd . wareonearth. com/-phi l /net/overhead/
TCP provides reliable data transport and delivery (TCP Windows, ACKs, ordering,
etc.).
IP provides IP "routing" capability so that packet can find its way through the
network.
Ethernet provides physical network capability (Cat 5, MAC, etc.).
Revision 0110 10 - 11
FCIP Concepts and Terminology (cont. )

• An FCIP tunnel is created between two FC/FCIP Entities, each of
which is configured with the following:
- A static IP address for each link end-point (FC IP_ LEP) o r tunnel
- Two TCP ports (fixed) to send and receive data
- The WWN of the other end of the FC IP link (optional)
• Depending on the FCIP platform, each GbE port supports 1 or 1-8
FCIP tunnels
B FCIP Entity
FCIP_LEP
FCIP Entity
FCIP_LEP I FC Entity I
Storage
Server
M"Wple
FCIP_LEPs
(Tunnels)
{
:
} Multiple
FCIP_LEPs
(Tunnels)
,, I FC Frame I
'Tl
\
(")
FCIP_LEP FCIP_LEP IFC Frame I
0
;::i.
GbE Port GbE Port

FCIP tunnels are similar to FC virtual channels with one significant difference: FCIP
tunnels require IP addresses, a TCP port, TCP parameter and QoS 1information, and
optionally the expected WWN of the other end of the link.
The FC Entity components combine with FCIP Entity components to form an
interface between a FC fabric and an IP network.
• FC Entities contain FC specific components like FPGA's. These Field
Programmable Gate A rray's (FPGA's) determine if
compression/decompression is needed on a packet and if compressions is
needed, they forward to the correct circuitry (HIFN 9360) for
compression/decompression. The FPGA's also handle TxlD translation to
ensure that the IP packet goes to the right TCP connection on the correct GbE
Port.
• The FCIP Entity is responsible for FCIP protocol exchanges on the IP network.
The FCIP Entity contains FCIP control components, at least one and possibly
multiple FCIP_ LEPs (FCIP_LEPs), and an FCIP Data Engine (FCIP_ DE).
- The FCIP control components are responsible for FCIP protocol exchanges
on the IT network.
-An FCIP link end-point (FCIP_LEP) is used to connect one end-point of a
TCP connection to the TCP FCIP LEP at the other end.
- The FCIP Data Engine (FCIP_DE) handles FC frame encapsulation, de-
encapsulation, and transmission.
Once the tunneled FC frames are in the IP network, normal IP network routing
procedures are used to transmit them through the IP network.
Revision 01 10 10 - 12
Before creating a TCP connection to a peer FCIP Entity, the FCIP_LEP needs a
static IP address, a TCP port (TCP port 3225 is used for FCIP COS F traffic and
TCP port 3226 is used for COS 2,3 traffic),. the expected WWN of the other end of
the link, and TCP parameter a nd Quality of Service (QoS) information.
Revision 0110 10 - 13
FCIP Concepts and Terminology (cont.)

• An FCIP tunnel is represented in a Brocade fabric as a Virtual
E_ Port (VE_ Port)
- No different from E_Ports, except underlying transport is IP
• The VE_ Port emulates an E_ Port on either end of the IFCIP tunnel,
so that:
- Standard E Port link ~nitialization occurs
- The FCIP platforms at both ends of the link merge to form a single
fabric
• VE_Ports do not support ISL Trunking , but they do support
exchange-based routing (Dynamm c Path Selection)

Revision 0110 10 - 14

• Current FCIP platforms support FC routing over an FCIP tunnel ,
creating a Virtual EX_Port (VEX_ Port)
- Allows long-distance FCIP connections with fabric-to-fabric isolation
- VEX_Ports are no different from EX_ Ports, except underlying transport
is IP
• On FCIP platforms that support FC routing, there are these
additional rules:
- A single GbE port can have VEX_ Ports and VE_Ports configured on a
per tunnel basis
- A VEX_Port connects only to a VE_Port - it may not connect to
another VEX Port
• There can be multiple VEX-to-VE port connections to each Edge
fabric, but cannot have EX-to-E and VEX-to-VE connections to the
same Edge fabric

You can optionally create a FCIP link for faiilover: create the link, then manually
disable it; enable it on an as needed basis (if the FC connection fails).
Revision 01 10 10 - 15

• When making FCIP connections, the TCP/IP topology used to
connect the sites may be quite different from the FC topology that
is implemented by the FCIP tunnels
- WAN topology: follow the GbE cables
- FC topology: follow the FCI P tunnels
Physical topology: one port/site FC topology: Based on the FCIP tunnel definitions;
connected to the WAN each switch is connected to the others (loop)
Site 01 S"te 01 ISL from Site 01
to Site 03
Two FCIP tunnels from

Sito 02 Site 01 vs. one physical
link

As we create the Fibre Channel network topology which would be tunneled across
the FCIP network, remember that the IP network is transparent to the FC fabric. In
the diagram above, the physical topology (shown at the left) has three Brocade
7500s, each with a single GE port connected to the same WAN. The virtual
interfaces (FCIP tunnels), however, connect each Brocade 7500 to the other two
routers, creating the loop FC topology shown at the right.
Revision 0110 10 - 16

• A Brocade FCIP tunnel can have a guaranteed amount of bandwidth,
called the committed rate (best practice)
- Link cost= 2000 - committed rate (Mbit/sec), so the higher the committed rate,
the lower the link cost
- No guaranteed bandwidth (uncommitted rate): link cost= 2000
• If you are connecting more than one VE-to-VE FCIP tunnel between two
FCIP platforms, manage the per-tunnel bandwidths carefully1
- If several tunnels have the same link cost, exchange-based routing is used
- If one tunnel has more bandwidth than the others, it will have the lowest FSPF
link cost, so it will carry all the traffic
Tunnel 1: BW = 50 Mbits/sec -> Link Cost = 1950
ITunnel 2: BW = 100 Mbits/sec ->Link Cost= 1900
Tunnel 3: BW = 50 Mbits/sec -> Link Cost= 1950
Result: Tunnel 2 carries all the traffic
Tunnels 1 and 3 carry no traffic

In the diagram above, three FCIP tunnels connect two Brocade 7500s. If the
bandwidth on all the tunnels were the same, then each would have the same link
cost, and thus all the tunnels would be used for switch-to-switch traffic. In our
example, though, tunnel 2 has more bandwidth than tunnels 1 or 3. Because the
FSPF link cost for an FCIP link/tunnel is equal to 2000 - (tunnel BW in Mbits/sec),
tunnel 2 will have the lowest link cost (2000 - 100, or 1900). As a result, all inter-
switch traffic will traverse tunnel 2, leaving the bandwidth on the other two tunnels
unused. FCIP tunnels can specify a committed bandwidth as well, so that tunnel
bandwidths can he matched.
Footnote 1: If we had specified an uncommitted tunnel, then the tunnel would be
allocated 1 Mbps bandwidth (it can use up to 1Gbit/sec - committed rate tunnels)
and the link cost would be calculated by taking 2000 - BW (in Mbps) 2000 - 1 = =
1999.
Revision 0110 10 - 17

• On FCIP platforms, fault isolation can be added to an FCIP tunnel
by combining FC Routing to create a routed FC SAN
- VEX_Ports at one end of each tunnel, VE_Ports at the other end
- Issues at the recovery site cannot affect the primary site
- When mapping the connections, draw the FC tunnels carefully, cross-
connecting the GbE ports to improve availability
FCIP ports from
Tunneled IFLs are cross-connected blade in slol 10
across blades 10 ~re a~~~ny
48000 "A"
E - _,,..-::---
F
-- --
':I:- ------
Physical
Each physical FCIP port on the FCIP ports from
7500 forms two logical IFLs blade in slot 9
r--Fabric ID 02
~ta SAN A Edge)
~
_J l_
r Fabric ID 01
(Meta SAN A Backbon~
I

1 Four 1 Gbit/sec Ethernet ports come from two FR4-18i blades on the chassis
2 FCIP ports from blade in Slot 9 - 1Gbit/sec
3 FCIP ports from blade in Slot 10 - 1Gbit/sec
4 Tunneled IFLs are cross-connected across blades to maximize availability
In addition to using FCIP for distance extension, you can also use the FC router
feature to isolate the primary site from potential reliability issues. To accomplish this,
we configured the FCIP ports in Fabric ID 01 as VEX_Ports, and the FCIP ports in
Fabric ID 02 as VE_Ports, creating a routed FC SAN. With the tunneled EX_ Ports
being at the "primary" site. This isolates the primary site from both the "recovery"
site fabrics and from the WAN itself. Each physical FCIP port at the primary site
contains one virtual EX_Port. Each GE port can support up to eight virtual tunnels
per FCIP port today, but in this case the customer only needed one tunnel per port.
Those tunneled VEX_Ports "look" like they form Fibre Channel IFLs from the point
of view of the fabric, even though they physically cross an IP network.
Since there are only two FCIP ports on each Brocade 7500 vs. four each Brocade
48000 chassis, each physical FCIP port on the Brocade 7500 contains two virtual
EX_Ports instead of just one, resulting in the cross-connection depicted above.
Revision 0110 10 - 18
Observe that the two physical FCIP ports on 7500 "A" (domain 02) are labeled "E"
and "F". Each of those ports connects via FCIP to two different physical interfaces
on 48000 "A" (domain 01 ). Those ports - "A" and "C" - are located on different
blades, so that a blade, media, or cable failure or replacement will not cause a WAN
outage for fabric "A". Physical port "E" connects to physical ports "A" and "C" by
using two different logical sub-interfaces.
Revision 0110 10 - 19
FCIP Performance - Overview

• In FCIP-based connections, there are several performance
variables that need to be considered:
- Effective Bandwidth: Amount of available bandwidth varies
- Delay: Considerations include distance PLUS all other network
parameters, including:
• router hop count, router processing, and packet size
- Loss: Number of IP packet delivery failures requiring a retransmission
request from the receiving device
• FCIP platforms provide additional features that address these
considerations

Effective bandwidth is a measure of how much of the available bandwidth can

actually be used, taking into consideration dropped packets and retransmission due
to congestion and protocol inefficiency.
Delay or latency is the amount of time it takes a packet of data to get from one point
to another. Delay is often measured by sending a packet that is returned to the
sender; the packets round-trip time (RTT) is considered the delay in the network. In
an IP network, delay not only depends on distance, but also upon router hop count,
router processing, and packet size. A reasonable rule of thumb to estimate delay is
driving distance * 1.5.
Pack,e t Loss: The Internet Standards treat packet loss and congestion as synonyms.
Congestion is the prime cause of packet loss. Congestion occurs when Routers
discard incoming packets that can't be stored or transmitted because the average
sum of the inputs to a router exceeds the capacity of its output. Slightly reworded:
anytime the average feed from the Ethernet exceeds the capability of a router,
packets will be lost. When the output connection is a costly nation-to-nation or
satellite link, it becomes very expensive to make the pipe big enough so packets
won't be lost. If the pipe is too big, then you become concerned with effective
bandwidth. Other contributors that cause packet loss include bit errors, deliberate
discards, and router in/out times. See
http : I /www . gigabytex . com/whttcp . htm for additional information. TCP
assumes that all packet loss is caused by congestion and responds by reducing the
transmission rate.
Revision 0110 10 - 20
Different applications use protocols with different block sizes to transfer data. Block
access protocols access "blocks" of data in portions that are a multiple of the OS
system block. Consider using the following guidelines to determine block sizes:
Transaction data (4-8k block size); Office automation (16-32k block size); Data
warehousing (64 - 256k block size); CAD/Design (64-128k block size); Multimedia
(512k - 4M block size). Small block sizes of contiguous data mean more 1/0
especially if the data is spread across the disk; large block sizes that don't use all
the space read the whole block just to get a small piece of data.
Applications can be configured to allow multiple outstanding I/Os to occur before
requiring an acknowledgement. The #of outstanding I/O's is typically 1 to 16.
Revision 0110 10 - 21
FCIP Performance - Selective Acknowledgement

• Packet loss significantly degrades FCIP performance
• Each packet must be acknowledged (ACK) by the receiving port or
the lost packet must be retransmitted
• To mitigate packet loss, Brocade FCIP platforms support Selective
Acknowledgement (SACK)
- When SACK is disabled on a receiving VE_Port, each lost packet
requires a separate ACK packet to request retransmission
- When SACK is enabled on a receiving VE_ Port, the request for
retransmission of multiple lost packets can be combined i n a single
ACK packet1
• Fewer ACK packets, faster recovery time, better performance

Packet loss re-transmissions are compounded when errors are bursty.

Footnote 1: SACK improves loss detection, retransmission techniques, and
enables faster recovery.
Selective Acknowledgement (SACK) is an extension to a protocol which allows the
acknowledge reception of specific packets or messages.
The SACK option RFC 2883 [18] allows the receiver to acknowledge multiple lost
packets in a single ACK, enabling faster recovery. An FCIP Entity MAY negotiate
use of TCP SACK and use it for faster recovery from lost packets and holes in TCP
sequence number space.
Revision 01 10 10 - 22
FCIP Performance - Hardware Compression

• To improve FCIP tunnel performance, Brocade FCIP platforms
support hardware-based compression
- When enabled, an FC data frame is compressed before FCIP
encapsulation is performed
•The receiving port must then remove the Eth/TCP/IP/FCIP headers, and
decompress the FC data frame
- Disabled by default on Brocade FCIP platforms
- Caveats:
• If WAN components are already compressing or encrypting data streams,
do not enable compression on the Brocade FCI P platform
• Verify that data compression is suitable for the data

In general, the performance of IP networks is less efficient than FC fabrics.

Latencies are higher, congestion more prevalent, and the chain of headers on an IP
packet and lower packet size supportable across a WAN can often result in an
additional reduction in efficiency for FCIP vs. FC. Also, it is never safe to assume
that a 1Gbit LAN link will actually be able to sustain 1Gbit across a WAN. Data
compression can compensate somewhat for lower performance networks, but it is
not a panacea.
Hardware compression maximizes throughput over WAN links without performance
degradation. The compression feature is a per tunnel feature that allows the FC
data frames (but not FC command frames) to be compressed before they are sent
over the tunnel as FCIP frames. The FC command frames and all the frames that
are generated from or destined for control processor are not compressed, even
when compression is enabled on a tunnel. The compression is done by hardware on
a frame-by-frame basis and uses the LZS algorithm to compress. Compression is
performed on a FC data frame in the ingress direction before they are encapsulated
as FCIP frames. The latency introduced by the compression functionality depends
on the size of frame being compressed; for a 2184 byte FC frame, the latency
introduced due to compression is about 14 microseconds.
Before enabling compression on a Brocade FCIP platform, first verify that
compression is not already enabled in the WAN. If the WAN is already performing
compression, then compressing in the SAN will only lead to redundant cost and
management effort with no performance improvement.
Revision 0110 10 - 23
You should also verify that data compression will not inadvertently affect the data
type or file system. Many data types (images, databases, etc) or file systems may
be already compressed (or encrypted) by their application (e.g. Oracle), and thus do
not benefit from further compression. Remember that compression (and encryption)
have a computational cost and add latency, which could negatively impact
performance.
Revision 0110 10 - 24
FCIP Performance - Without Traffic Shaping

• An FCIP tunnel provides unlimited bandwidth (up to the maximum
link bandwidth) for host-based applications to drive data
- If an application drives more data into an FCIP tunnel than the
available bandwidth, the effective FCIP bandwidth has an undesirable
sawtooth pattern
Packets Drop at Maximum Connection

Bandwidth (10 Mbits/sec)
..
.... ·.t·... ...... Max: 10 Mbps
,.•• • I
.•.·· I
8(
.•. •·• I
-.~--
,•'
1-~
- GMbps
[~
Non-traffic
Non-traffic shaped
average Effectivej
shaped data
I
1
haped saw-
ooth effect
L Bandwidth
I
------~ Time

A host does not know how much bandwidth is available; most hosts would try to get
as much bandwidth as possible.
• The amount of bandwidth available is variable when there are multiple hosts sending
data through the IP backbone.
• For example, a host may send 2, 4, 8, 16, 32 packets without receiving an ACK from
the other side so as to get as much bandwidth as possible. This would continue until
the congestion threshold (maximum bandwidth of the connection) is reached.
• When the destination requests a re-transmission of frames due to missing frames or
time outs, the source host will slow down. This creates the saw-tooth effect in the
figure shown above.
• The type of applications running on the network also impact effective bandwidth. This
is why integrating storage traffic with other IP traffic of emails, documents, databases
and others is not always the best practice.
In the example above, the maximum bandwidth of the connection is 10 Mbits/sec.
The application attempts to fill this pipe, but as the solid red line shows, each time
the application hits the 10 Mbits/sec maximum, the pipe fills, throttling the
application, and causing the application to halt, resulting in the "rounded sawtooth"
formation. The receiving VE_Port sees the traffic as the dotted "sawtooth" effect,
indicating the 'TCP Slow-Start" effect as the port buffers fill , then empty, then refill
again. The effective bandwidth is only 6 Mbits/sec, so 40% of the available
bandwidth is not being used.
Revision 0110 10 - 25
FCIP Performance - Traffic Shaping (cont.)

• To improve effective bandwidth, Brocade FCIP platforms support
Traffic Shaping
- When traffic shaping is disabled, each FCIP tunnel tries to use all the
available link bandwidth
- When traffic shaping is enabled, each FCIP tunnel has the maximum
bandwidth available, resulting in more consistent performance
• Can also be paired with SACK
.. ....
...··..
8(
.•. •·• I
....~----t -.~--
,•'
1-~
- GMbps
[~
I
I .. Non-traffic shaped
Non-traffic average Effectivej
Traffic
Non-traffic
shaped data
:
1
haped saw-
ooth effect
L Bandwidth
shaped cilata
I
Time
In the example above, the FCIP tunnel has traffic shaping enabled, with the
maximum bandwidth set to 10 Mbits/sec, matching the maximum link bandwidth.
Now, as the application attempts to fill this pipe, the solid blue line shows the
application hitting the 10 Mbits/sec maximum once, then holding at this bandwidth.
The resulting effective bandwidth (shown as a dotted blue line) is now 9 Mbits/sec,
an improvement of 50% over the non-traffic shaped example.
Revision 0110 10 - 26
FCIP Performance - Jumbo Packets

• There is also a mismatch in the maximum sizes of Ethernet packets
(default: 1518 bytes) and FC frames (2148 bytes)
- To send a full-sized FC frame req uires two Ethernet packets,
increasing the chance of packet loss, and decreasing the effective
bandwidth
FCIP Platform
2148 bytes
'----~~~~~~-73_2_by_te_s~~./
1518 bytes

The default Maximum Transfer Unit (MTU) size of an Ethernet packet is typically
1518 bytes. This is smaller than the FC frame maximum of 2148 byt,e s, so a FC
frame would be broken into two Ethernet packets. The maximum MTU size of a
Gigabit Ethernet is larger than 1518 bytes.
Note that the combined size of the Ethernet, IP, TCP, and FCIP packet headers is
102 bytes (1518-1416).
In the example above , a full-sized FC frame (on the left) is encapsulated in two
standard-sized TCP/IP frames .
Revision 01 10 10 - 27
FCIP Performance - Jumbo Packets (cont.)

• To avoid this mismatch, some FCIP platforms support Jumbo
Packets
- With Jumbo Packets enabled, the Ethernet packet size is increased to
2384 bytes
- A full-sized FC frame can be sent in one TCP/IP packets, decreasing
the chance of packet loss, and increasing the effective bandwidth
- If the TCP/IP network does not support Jumbo Packets, the packets
will be fragmented
FCIP Platform
21148 bytes
'---~~~~~~-21_4_8b~yt_es~~./
2384 bytes

The maximum size of 2384 bytes accommodates 2148 bytes of FC frame data and
102 bytes of Ethernet, IP, TCP, and FCIP headers (2384 2148 + 236). =
In the example above, jumbo packets have been enabled on the FCIP platform. As
a result, the FC frame entering the FC port can be encapsulated in a single, 2384
byte Gigabit Ethernet packet.
Revision 0110 10 - 28
QoS on FCIP Links

• Fabric OS v6.0 +offers two ways. to preserve the QoS across the
FCIP tunnel:
- DSCP (Differentiated Services Code Point) is a means of classifying
or prioritizing layer 3 traffic
- L2CoS (Layer 2 Class of Service) is a means of classifying or
prioritizing traffic over a switched layer 2 network, primarily a VLAN.
This feature does not prioritize the flows within the SAN
• It only tags the egress frames so that the IP router can classify/prioritize
the traffic

Differentiated Services (DiffServ) is a standard for IP networks that classifies,

manages, and provides Quality of Service (QoS) guarantees for network traffic. T he
DiffServ class is specified by a six-bit Differentiated Services Code Point (DSCP)
value, and is managed by layer 3 Ethernet routers. The Type of Service (ToS) field
in the IP header contains the DSCP. The network nodes queue and forward traffic
based on this value. L2CoS (Layer 2 Class. of Service) is a means of classifying or
prioritizing traffic over a switched Layer 2 network, primarily a VLAN, and is defined
by IEEE 802.1. There is no reverse tagging of DSCP/L2COS in the FCIP ingress
side, since the VC is carried across the network within the frame. Which is to say,
since the QoS information is already in the frame when it reaches the IP router, the
router has no need to "retag" the frame with DSCP/L2COS information to route it.
Revision 0110 10 - 29
QoS on FCIP Links (cont.)

• To preserve the QoS across the FCIP tunnel, the VC on the FCIP
outbound frame will be mapped to both a DSCP and L2COS value
- These values are defaulted to pre-determined values, and can be
modified on a per-FCI P tunnel basis 1
• This mapping will only occur if the FCIP tunnel is configured with
VC QoS Mapping on
- If this mapping is on, the mapped values are simply placed into the
associated egress protocol headers, i.e. the IP headerNLAN tag

Footnote 1:
Default VC to FCIP QoS Mapping
VC# DSCP L2CoS

0 101110 (46) 111 (7)
1 000111 (07) 000 (0)
2 001011 (11) 011 (3)
3 001111 (15) 01 1 (3) ""'
Medium
4
5
010011 (19)
010111 (23)
01 1 (3)
011 (3) u
--
6 011011 (27) 000 (0)
OLW..1 (31)
Low
11 101111 (47) 100 (4)

12 110011 (51) 100 (4) High
13 100 (4)
~4 100 (i)
15 000 (0)
Revision 0110 10 - 30
1Pv6 Support
• Fabric OS v6.1 +supports 1Pv61 addresses on GbE ports for FCIP
- A GbE port on the Brocade 7500/FR4-18i may have 1Pv4 and 1Pv6
interfaces simultaneously
- 1Pv6 is not supported on the GbE ports on the FC4-161P and FA4-18
blades
• Caveats:
- When an 1Pv6 address is configured on a Brocade 7500 or FR4-18i
GbE port, IPSec may not be configured on that chassis or blade
- Compression is not supported on an 1Pv6 configured FCIP tunnel
- 1Pv6 packets may not be tunneled through an 1Pv4 network
- Tunnels must be 1Pv4 <-> 1Pv4 or 1Pv6 <-> 1Pv6

Footnote 1: 1Pv6 is a network-layer protocol for packet-switched networks that is

designated as the successor to 1Pv4. 1Pv6 addresses consist of 128 bits (3.4 x 1038
total addresses), compared to 32 bits (4.3 x 109 total addresses) allowed in 1Pv4.
Revision 0110 10 - 31
1Pv6 Support- Fabric OS v6.1

• 1Pv6 support is added to the following CLI commands 1 :
portcfg f ciptunnel portshow fciptunnel

portcfg f icon portshow ficon
portcfg iproute portshow ipif
portshow iproute
portcf gshow f ciptunnel
portcfgshow ipif
portcf gshow iproute

Footnote 1: Web Tools and Fabric Manager v6.0 also support 1Pv6 addresses.
Revision 0110 10 - 32
VE_Port Configuration - Overview

• Follow these basic steps to create a VE-to-VE_Port connection: 1
All steps must be performed on both ends of the link
1. Determine the required parameters for the VE_Ports and FCIP tunnel
2. Persistently disable the virtual FC ports associated with the tunnel 2
3. Create an IP interface - an IP address that will be used by the FCIP tunnel
4. Configure an IP route to specify an IP gateway (optional)
5. Verify the IP network between the two IP interfaces that will form the FCIP
tunnel
6. Create an FCIP tunnel
•Associate the local IP interface address with a tunnel ID, a committed rate ,
and a remote IP interface address at the other end of the link
•As needed, configure FCIP features (SACK, compression, etc.)
7. Verify configuration, enable the associated VE_Port, and validate functionality

Footnote 1: We will go through the steps to create a VE_Port -> VE_Port tunnel
and then go through them again AND add steps to create a VEX_Port-> VE_Port
tunnel. Note: For VEX-to-VE_Port connections, additional steps are required. We
will focus on the different aspects of the configuration each time.
Create an IP interface on the tunnel, each interface is automatically given an
instance number. Each IP interface requires:
• A static IP address
• MTU size specification
• TCP ports (3225/3226) - Note: these ports are automatically assigned (not
configurable)
Use the IP interface to create an FCIP tunnel. Each tunnel configuration requires:
• A tunnel number (0-7)
• The IP address at the remote end of the link
• A maximum bandwidth allocation value for that tunnel called committed rate
(comm_rate )
Footnote 2: In addition to persistently disabling the virtual FC port, the GE port can
also be disabled.
Revision 0110 10 - 33
Determine the VE Port and FCIP Parameters

• Before creating a VE_Port and an FCIP tunnel, make sure that you
have defined the basic parameters:
- IP Interface: GbE port, IP address, netmask, MTU
- IP Route (optional): Gateway IP address, netmask
- FCIP tunnel basics: Tunnel ID (determines the virtual FC port), source
and destination IP address, committed rate, and FCIP tunnel
parameters
• Remember: an FCIP tunnel requires two endpoints, so you will
need two of everything

Revision 0110 10 - 34
PersistenUy Disable the Virtual FC Ports

• Before beginning VE_Port configuration, persistently disable the
virtual FC port with the portcfgp ersistentdisable
[s l ot I] [ ge] por t command
- Use the G bE port and tunnel ID to determine which virtual FC port is to
be disabled
B48000 : admin> portcfgpersistentdisable 10/16
B7 5 00 : admi n > portcfgpersistentdisable 16
Server Storage
Brocade 48000
with FR4-181 Blade
Brocade 7500
WAN
Port 10fge0
(FC port 10/16)
VE_Port ---.. Port geO
(FC port 16)
VE_ Port

35
Revision 0110 10 - 35
Create IP Interface
• Create an IP interface using the portcfg command with the ipif
operand : portc fg ipi f [ slot /] g e port create args
- Required args for ipif include:
• Ipaddr - Unicast 1Pv4 address
• Ne tmas k - Contiguous 1Pv4 bitma:sk
• mtu size - 1500 through 2348 Bytes
- An MTU size greater than 1500 enables jumbo packet support; you will
get a message warning you to verify that this is supported
• Note: The ipaddrsho w command has unrelated Fibre Channe l
I P Address parameters; i paddr show parameters are used for
in-band management using IP/FC, not FCIP 1

IP Address - Default: None; Range: Unicast 1Pv4 address (formerly known as class A,B,C)
in dotted decimal format. Range is 1.1 .1.1 through 223.255.255.254; Mandatory/Optional:
Mandatory; Disruptive/Non-disruptive: Adding an IP address is non-disruptive. Delete
Request will be rejected for interfaces which have FCIP tunnels on them.
Net mask - Default: None; Range: Contiguous bitmask in 1Pv4 dotted decimal format;
Mandatory/Optional: Mandatory: Disruptive/Non-disruptive- see above (I P Address
information)
MTU size - Default: None; Range: 1500 through 2384 (large enough to contain a full sized
FC data with FC and FCR headers, and FCIP, TCP, IP, Ethernet headers). If an MTU size
greater than 1500 is configured on the Interface, proper steps must be taken to insure that
the entire IP network (all the routers/switches/hosts in the path) support that MTU size.
Here is an example configuring an IP interface with an MTU size greater than 1500, notice
the message:
NDA- T01- 48K : admin> portcfg ipif 10/geO create 192.168.10.0 255.255.255.0 2100
WARNING : You are t rying to configure MTU size greater than 1500 .
Pl ease make sur e that all devices i n your I P
network can support Max Ethernet Size frames
You can a lso use CL! "port c md - - i pperf" to
find out the actual PMTU .
Operation Succeeded
Revision 0110 10 - 36
Footnote 1: FC switches can be managed in-band using IP/FC, basically one

switch is set up as the gateway to access other switches for management purposes.
This is totally unrelated to the F'CIP configuration being discussed in this module,
but there is some confusion related to ipaddrshow and configuring an FCIP
tunnel, so it is worth mentioning here.
Revision 0110 10 - 37
Create IP Interfaces Example

• On the Brocade 48000, create an IP interface on port 10/geO with an MTU size of 2348:
B48000 : admin> portcfg ipif 10/geO create 192.168.20.48 255.255.255 . 0 2348
WARNING : You are trying to configure MTU size greater than 1500 .
Please make sure that all devices in your IP
network can support Max Ethernet Size frames
You can also u se cli "portcmd -- ipperf " to
find o u t the actual PMTU .
Operation Succeeded
• On the Brocade 7500, create an IP interface on port Geo with an MTU size of 1500:
B7500 : admin > portcfg ipif geO create 192.168.23 . 75 255.255.255.0 1500
Operation Succeeded
Setver Storage
Brocade 48000
with FR4-181 Blade
Brocade 7500
WAN
VE_Port VE Port
192.168.20.48 . . . - ... 192.168.23.75

This example creates IP interfaces in two different subnets using two different
mtu size specifications.
The mtu_size does not have to be the same at each end of the link. It specifies an
mtu_s ize for IP packets going out from specified IP interface.
Consider setting Path MTU to the largest size possible and then running IPPerf;
then scale back the MTU size and re-run IPPerf as needed.
Revision 0110 10 - 38
Create IP Interfaces Example (cont.)

• Verify the IP inte rface settings with the portshow ipif [slot/ ] port command:
B48000 : adrnin> portshow ipif 10/geO
Slot : 10 Por t : geO
Inter face IP Address NetMask MTU
0 192 . 168 . 20 . 48 255 . 255 . 255 . 0 2348

B7500 : admin> portshow ipif geO
Port : geO
Int erface IP Address NetMask MTU
0 192 . 168 . 23 . 75 255 . 255 . 255 . 0 1500
Storage
Brocade 48000
with FR4-181 Blade
Brocade 7500
WAN
VE_Port VE Port
192.168.20.48 . . . - ... 192.168.23.75

The portshow ipif [slot/] port command displays the interface ID, IP
address, netmask, and MTU slide for each IP interface.
Revision 0110 10 - 39
Define an IP Route (Optional)

• After defining the IP interface of the remote switch, (optionally)
define destination routes on an interface by configuring an IP route
- Add IP routes when crossing subnets or when you want to reach the
destination IP of the remote site tlhrough a preferred gateway IP rather
than the default gateway
- A maximum of 32 routes can be added per ge port1
- Use the portcf g iproute command:
portcfg iproute [slot/] [geO l gel] create <destination
ip_address> <netmask> <l oca l ga t eway> <metric>
- Specify the i p _address and netmas k of the destination tunnel
- Specify the IP address of the local gateway responsible for forwarding
frames to destination IP address (must be on same subnet as local
device)
- Use the default metric of 0 (if configuring more than one route you can
make the second one less preferred by setting a higher metric)

Footnote 1: Static Routes may be configured into the Presto Stack for IP routing on
the GbE WAN side. IP interface must be configured before we add a destination
route on an interface. A maximum of 32 routes can be added on one GbE port (geO
or ge1 ). Route additions do not require tearing down tunnels. The IP address and
gateway parameters use Unicast 1Pv4 address in dotted decimal format (1 .0.0.1
through 223.255.255.254); netmask uses contiguous bitmask in 1Pv4 dotted decimal
format. The associated metric (weight) value range is 0 through 255.
The specified IP address needs to be an actual IP address at the other end of the
link, not a subnet address.
When you create multiple routes to get to the other end of the link:
• To use a preferred gateway specify a metric of 0
• To configure an alternate, secondary gateways specify a higher metric value
• The higher the metric, the less preferred the route
Revision 01 10 10 - 40
Define an IP Route Example

• On the Brocade 48000, add a route on port 10/geO to the remote IP interface
192 . 168 . 2 3 . 7 5 through !local gateway 192 . 168 . 2 o. 1 with a default metric
of O:
B48000 : adrnin> portcfg ipr oute 10/geO c r eate 192.168.23.0 255 . 255.255.0 192.168.20.l 0
• On the Brocade 7500, add a route on port geO to the remote IP interface
192 . 168 . 2 o. 4 8 through local gateway 192 . 168 . 2 3 . 1 with a default metric
of o:
B7500 : adrnin> portcfg iproute geO create 192.168 . 20.0 255.255.255.0 192.168.23.l 0
B rocade 48000
with FR4·181 Blade
Fabnc
B
VE Pon VE_Port
192.168.20.48 • • - • .. 192.168.23.75

The portcfg iproute [ s lot ] /port create command configures an IP route

from a local IP address to a gateway IP address over an GbE port. The command
has the following required arguments:
• [slo t J /port: The port on which the command is to operate.
• ipaddr : The IP address of the route.
• ne tmask: The IP netmask.
• g ateway_route r: The IP address of the gateway router.
• me t ric: The gateway metric; if not specified the default metric of 0 will be
used.
In the example above, we specified IP routes at each end of the link because the
two IP interfaces that we configured are in d ifferent subnets.
Revision 01 10 10 - 41
Define an IP Route Example (cont.)

• Verify the IP route settings with the port show i prout e [slot/ J port command :
B4 8000 : admin> portshow iproute 10/geO
Slot : 10 Port : geO
I P Address Mask Gateway Metric Flags
192 . 168 . 23 . 0 255 . 255 . 255 . 0 192 . 168 . 20 . 1 0 Interface

B7500 : admin> portshow iproute geO
Port : geO
1 92 . 1 68 . 20 . 0 255 . 255 . 255 . 0 192 . 168 . 23 . 1 0 Interface

Brocade 48000
with FR4·181 Blade
Fabnc
B
VE_Port
192.168.20.48
--- VE_Port
192.168.23.75
C 2009 Brocade Commun cat ens Systems Inc

42
The por tshow ipif [ s l ot/ ] port command displays the interface ID, IP
address, netmask, and MTU size for each IP interface.
Revision 0110 10 - 42
Validate the IP Network

• Verify IP connectivity between the two IP interfaces with the
portcmd --ping [slot J /port command
- Always spec.ify local GbE port, the source IP (- s), and destination IP (-ct):
B7500 : admin> por tcmd --ping ge O -s 192 . 168 . 23 . 75 -d 192.16 8.20. 4 8
Pinging 192 . 168 . 20 . 48 from ip interface 192 . 168 . 23 . 75 on 0/0 with 40 bytes of data
Reply from 192 . 168 . 20 . 48 : bytes=40 rtt=Oms
Pi ng Statistics for 192 . 168 . 20 . 48 :
Packets : Sent = 4 , Received = 4 , Loss = 0 ( 0 percent loss)
Min RTT = Oms , Max RTT = Oms Average = Oms
Server Storage
~
Brocade 48000
with FR4-181 Blade
Brocade 7500 -
VE Port VE_Port
192.168.20.48 • - - - 192.168.23.75

The portcmd --ping [slot]/port command validates end-to-end IP connectivity over an

GbE port. The command has the following required arguments:
• [ s l o t J /port : The port on which the command is to operate (here, geO on
7500).
• -s: The source IP address for an IP interface on a local GbE port.
• - d : The destination IP address for an IP interface on a remote GbE port.
• -n num_requests : Specifies the number of ping requests. The default is 4.
• -q s e rvice_ type: Specifies the type of service in the ping request. The
default is 0 and service_type must be an integer from 0 to 255.
• - t t tl: Specifies the time to live. The default is 100.
• -w wait_ time: Specifies the time to wait for the response of each ping
request. The default is 5000 milliseconds and the maximum wait time is 9000.
• - z: Specifies the default packet size to a fixed size in bytes. The default is 64
bytes. The total size, including ICMP/IP headers (28 bytes without IP options)
cannot be greater than the IP MTU configured on the interface.
If no optional parameters are specified, the command displays the currently
configured values for the specified port.
Revision 0110 10 - 43
In the example above, a ping command is issued from the new IP interface on the
Brocade 7500, to the new IP interface on the Brocade 48000.The command output
shows that the ping messages are received and returned by the Brocade 48000,
verifying IP connectivity between the IP interfaces.
Revision 0110 10 - 44

• Verify the IP router hops between the two IP interfaces with the
following command : portcmd - -traceroute [slot] /port
• Always specify local GbE port, the source IP (- s), and destination
IP (-ct):
B7500 : admin> portcmd -- traceroute geO - s 192 . 168 . 23 . 75 - d 192 . 168 . 20 . 48
Traceroute to 192 . 168 . 20 . 48 from IP interface 192 . 168 . 23 . 75 on 0/0 , 64 hops max
1 1 92 . 168 . 23 . 1 0 ms 0 ms 0 ms
2 1 92 . 168 . 20 . 48 0 ms 0 ms 0 ms
B48000 : admin> portcmd -- traceroute 10/geO - s 192 . 168 . 20 . 48 - d 192 . 168 . 23 . 75

Traceroute to 192 . 168 . 23 . 75 from I P inte rface 192 . 168 . 20 . 48 on 10/0 , 64 hops max
1 192 . 168 . 20 . 1 16 ms 0 ms 0 ms
2 192 . 168 . 23 . 75 16 ms 0 ms 0 ms

portcmd -- traceroute [slot/ ]geport -s src_ip -d dst ip [-h

max_hops ] [- f first ttl] [-q type_of_service ] [-w timeout ] [-z
size]
Traces the IP router hops used to reach the host ds t _ ip from one of the source IP
interfaces on the GbE port. Valid arguments include:
• - s src_ ip: Specifies the local IP address to use for sourcing the probe
packets.
• -d dst _ ip: Specifies the destination IP address to which to probe the IP
router path.
• -h max_hops: Specifies the maximum hop limited used in the outgoing
probe packets.
The default of probing a maximum of 30 IP router hops. This operand is optional.
• -f first_ ttl: Specifies the starting time to live value to first_ttl. The
default is 1. --traceroute skips processing for those intermediate gateways that
are less than the first_ttl hops. This operand is optional.
• - q service type: Specifies the type of service in the ping request. The
default is 0 and service type must be an integer from 0 to 255. This
operand is optional.
• - w timeout : Sets the time, in seconds, to wait for a response to a probe.
The default is 5 seconds.
Revision 01 10 10 - 45
• - z size: Specifies the size, in bytes, of the trace route packet to use. The
default is 64 bytes. The total size, including ICMP/ IP headers (28 bytes without
IP options) cannot be greater than the IP MTU configured on the interface. This
operand is optional.
Revision 0110 10 - 46

• Verify the IP interface and FCIP tunnel settings with the IPPerf tool
and the portcrnd - - i pperf [slot J /port command 1
- Always specify local GbE port, the source IP (- s ) and destination IP (-d), and
whether th is port is the sender (-S) or receiver ( - R)
- Start the IPPerf receiver first (- R), then start the IPPerf sender (- s)
- If no time interval is specified, type Ctrl- C on the sender to stop
B7500 : admin> portcmd --ipperf geO - s 192.168 . 23.75 - d 192.168.20.48 - R
ippe rf to 192 . 168 . 20 . 48 from IP i nterface 192 . 168 . 23 . 75 o n 0/0 : 3227
Server Storage
·~·"''°° ~~
Brocade -iaooo
with FR4·181 Slade
Fabric
B
VE Port VE_Port
192.168.20.48 . . - - - 192.168.23.75

Footnote 1: The portcmd --ippe rf [slot ] /port command captures end-to-

end IP performance data over an GbE port and can be useful in validate a service
provider Service Level Agreement (SLA) throughput, loss and delay characteristics .
The command has the following required arguments:
• [slo t J /port : The port on which the command is to operate (here, geO and
9/gel on switch2)
• -s : The source IP address for an IP interface on a local GbE port.
• -d: The destination IP address for an IP interface on a remote GbE port.
• -s I -R: Whether this port is the sender (-s) or receiver (-R) of the test
packets. These parameters are mutually exclusive.
• -t: The time interval over which the test is to run; in minutes. Default: forever.
Specifies total time to run the test traffic stream, in seconds. If not specified ,
the test runs continuously until you explicitly abort the operation with Ctr l +c.
• - i: The interval between updated measurements (the calculation window); in
seconds. If less than the time interval, the updated measurements are
displayed onily once, at the end of the interval. Default: 30 seconds.
• - r : Committed rate to be used by the IPPerf packet generator; in Kbits/sec.
Default: 0 (uncommitted).
Revision 01 10 10 - 47
• - p: The TCP port to use when sending and receiving the test frames. If -s
was specified, this value is the remote port with whom the local port is to
connect; if - R was specified, this value is for the local port to listen for new
connections. Default: 3227
• - z: Default buffer size to use; in bytes. Default: MTU size specified for the
FCIP tunnel.
• If no optional parameters are specified, the command displays the currently
BW represents what the FCIP tunnel I FC .application throughput rather than the
Ethernet on the wire bytes.
WBW represents what the FCIP tunnel I FC application throughput rather than the
Ethernet on the wire bytes.
Loss(%) is the number of TCP retransmits. This number is an average rate over
the last display interval.
Delay (ms) is the TCP smoothed RTT and variance estimate in milliseconds.
Path MTU is the largest IP-layer datagram that can be transmitted over the end-to-
end path without fragmentation. This value is measured in bytes and includes the IP
header and payload.
In the example above, two switches are connected via FCIP (port geO on 7500, and
port 1 oI geO on 4 8 oOo). After creating IP interfaces on the appropriate GbE ports,
the IP Perf utility is launched, with 7 5 oo as the receiver, and 4 8 ooo as the send er.
The command output on each switch notes the "to" and "from" addresses, as well as
the slot/port indicator (slot 0 is the motherboard on the Brocade 7500).
Revision 0110 10 - 48

• The portcmd --ipperf [slot] /port command output on the
sender displays end-to-end IP path performance values
- Updated in real-time, according to the time and test interval
- Two bandwidth values: BW and WBW
B48000 : admin> portcmd --ipperf 10/geO -s 192.168.20 . 48 -d 192 . 168.23 . 75 -s
ipperf to 192 . 168 . 23 . 75 from IP interface 192 . 168 . 20 . 48 on 10/0 : 3227
30s : BW : ll3 . 03MBps WBW(30s) : 55 . 39MBps Loss( %) : 0 . 0 Delay(ms) : 1 PMTU : l500
60s : BW : 108 . 89MBps WBW(30s) : 83 . 0SMBps Loss( %) : 0 . 0 Delay(ms) : 0 PMTU : l500
90s : BW : ll2 . 59MBps WBW(30s) : 96 . 93MBps Loss( %) : 0 . 0 Delay(ms) : 0 PMTU : l500
<Truncated Output>
Server Storage
Broc1de 48000
with FR4·18i Blide Brocade 7500
VE_PO<I VE_PO<I
192.168.20.48 • - - - 192. 168.23.75

The portcmd --ipperf [ slot] /port command output presents the

following values (as seen in the last line above);
• 9 Os - The time elapsed since the beginning of the test.
• BW : 112 . 59MBps - The bandwidth measured in the current time interval.
• WBW (30s) : 96 . 93MBps - The weighted bandwidth (with the current interval
given a 50% weight) since the beginning of the test.
• Loss ( %) : o. o - The packet loss measured in the current time interval.
• De l ay (ms) : o - The round-trip time measured, in milliseconds, during the
current time interval rounded to the nearest 1 ms. A value of 0 ms indicates a
RTT of less t han 1 ms.
• PMTU : 1500 - The Path MTU, in bytes.
Revision 01 10 10 - 49

• Verify that the remote IP address has been resolved (ARP) 1 with
the port show arp [ slot/] port command:
B48000 : admin> portshow arp 10/geO

GE Port 10/geO
IP Address Mac Address Flags
192 . 168 . 23 . 75 00 : 05 : le : 37 : 3b : be Resolved

'V'
MAC address of remote Ge port (Brocade 7500)

Footnote 1: The Address Resolution Protocol (ARP) is the method for finding a
host's hardware address when only its network layer address is known. ARP is
primarily used to translate IP addresses to Ethernet MAC addresses. On the
Brocade 7500 and FR4-18i, address resolution is performed once a virtual port is
configured as an FCIP tunnel and the port is enabled .
The ports how arp [ slot ] /port command output displays the IP address
associated with the given MAC address, and any related flags. In the example
above, we see that port 1O/geO on the Brocade 48000 has resolved the IP address
of the IP interface on the Brocade 7500. The MAC address of port geO is
00:05: 1e:37:3b:be.
Revision 01 10 10 - 50
Create an FCIP Tunnel

• With the IP interface created (and IP routes defined, optionally),
create an FCIP tunnel on the configured local IP interfaces
• Use the portcfg fciptunnel operand : portcfg f ciptunnel
[slot/][ge]port create args [optional args ]
args for fciptunnel include:1
- tunnel_num - Identifies the tunnel number (0-7)2
- remote_ipaddr - Specifies the remote IP address
- local_ipaddr - Specifies the local IP address
- comm_rate - Specifies the committed rate, in Kbits/sec3 : specify 0 for
uncommitted or 1544 Kbps (T1 ) ~ 1000000 Kbps for committed. Default: 0
• The switch with lower local IP address will attempt to connect
(initiate) connection using the configured "remote IP address"
• The switch with higher local IP address will listen (accept)
connection only from configured "remote IP address"

Footnote 1: The required portcfg f ciptunnel arguments are listed on the slide
above. The optional arguments are discussed on subsequent slides. Changing FCIP
tunnel arguments is disruptive to the FCIP tunnel and therefore to the overlaid FC
Ports.
Footnote 2: If the t unnel_ num is not specified, configuration will automatically be
assigned the next available tunnel number.
Footnote 3: The mandatory committed rate for the tunnel on the GbE port does not
have a default value. The range is: 0 for uncommitted and 1544 Kbits/sec (T1) to
1000000 Kbits/sec for committed.
NDA-ST02-B48 : admin> portcfg fciptunnel 10/geO create 1
192 . 168 . 1.23 192.168.1 . 2 1000000
Available Bandwidth on this GigE Port = 845000 Kbps
Tun nels with uncommitted bandwidth take 1 000 Kbps
Minimum Committed Rate for tunnels is 1544 Kbps
Tunnel Bandwidth Exceeded
Revision 0110 10 - 51
FCIP Tunnel Arguments

• The port cfg fciptunnel command accepts a number of
optional arguments
1. Settings marked with * must be the same at both ends of the FCIP tunnel, or the tunnel will not be
established
2. -c : Enable hardware compression of outgoing data (default is off, range is on/off) *
3. -k timeout : Set the keepalive_timeout (default is 10 seconds, range is 8 - 7200 seconds)
4. -m time : Set the minimum retransmit time (default is 100 ms, range is 20ms through 5000 ms)
Note: Changing this parameter is disruptive
5 . -r retransmissions : Set the maximum retransmissions (default is 8 attempts, range is 1 - 8
attempts)
6. -n wwn : Set the remote switch WWN (if specified, the local switch accepts the incoming FCIP
tunnel only from a remote switch with the configured WWN, and only initiates a tunnel to the
specified switch)
7. -s : Disable Selective Acknowledgement (default is on, range is on/off)
8. - f : Enable Fastwrite (default is off, range is on/off) *
9. - f -t: Enable Tape Pipelining (default is off, range is on/off); requires Fastwrite to also be
enabled*

portcfg fciptunnel includes the following optional arguments. Changing

optional arguments is disruptive to the FCIP tunnel and therefore to the overlaid FC
Ports. Optional portcfg fc i ptu nnel arguments include:
• -c enables compression (default is off): Failing to enable compression at both
ends of a FCIP tunnel results in the tunnel not forming, and the following error
message is issued:*
• 2006/03/02-21 : 53 : 53 , [IPS - 1006] , 118 1 , WARNING, NDA-ST02-
B75 , Tunnel Configuration Mismatch for slot (0) port (O)
tunnel ID ( l ) reason (Compression ; mismatch .)
• -k timeout specifies the keepa 1 i ve _ timeout: default is 10 seconds,
range is 8 seconds through 7200 seconds. This is like a heartbeat timer. The
only instance that I can think of where this may need to be changed is when
going over a satellite, there could be other instances. Changing it will impact
performance, get your networking experts involved!
• -m time specifies the minimum retransmit time: Default is 1OOms, range is
20ms through 5000ms. Changing this parameter is disruptive. This value
represents how long to w.ait until data is retransmitted.
• -r retransmissions specifies the maximum retransmissions, default is 8,
range is 1 - 8 attempts. This value represents how many times. to attempt
retransmission of data before showing a failure.
Revision 0110 10 - 52
• -n wwn specifies the remote switch WWN. If the remote WWN is configured,
the switch only accepts the incoming FCIP tunnel with the configured WWN; it
also only initiates a tunnel to the desired switch. If the remote WWN is not
configured, the switch accepts FCIP connections from any other switch.
• -s disables Selective Acknowledgement (SACK): default is on, range is on/off.
Recall that SACK allows receiver to acknowledge multiple lost packets with a
single ack and thus enabling faster recovery. *
• -£enables Fastwrite: default is off, range is on/off. Recall that Fastwrite
allows the local gateway to buffer write 1/0 operations, allowing the FCIP
tunnel bandwidth to be optimized.
• -£ -t enables Tape Pipelining: default is off, range is on/off. Recall that Tape
Pipelining optimizes tape-oriented 1/0 operations. *
*indicates parameters that must be set to the same value at both ends of the
FCIP tunnel, or the tunnel cannot form.
Revision 0110 10 - 53
Create an Unencrypted FCIP Tunnel Example

• On the Brocade 48000: Port 10/g eO , tunnel ID = o (FC port 10/16);
Remote IP = 1 92 . 1 68 . 23 . 75 ; local IP = 192 . 168 . 20 . 48;
committed rate = 155000 Kbits/sec (OC-3); compression enabled
B4 8000 : a dmin> portcfg fciptunnel 10/geO create 0 1 92.168.23.75
1 92.168.20.48 155000 -c
• On the Brocade 7500: Port g eO , tunnel ID= O (FC port 16);
Remote IP= 192 . 1 68 . 23 . 75 ; local IP= 192 . 168 . 23 . 75;
committed rate= 155000 Kbits/sec (OC-3); compression enabled
B75 00 : a dmin> portcfg fciptunnel geO create 0 192.168.20.48
192 . 168.23.75 155000 - c
Server Storage
Brocade 48000
w ith FR4-18i Blade
Brocade 750(("
Fabric
B
VE_Port
192.168.20.48
Tunnel O(Port 10116) --- VE_Port
192.168.23.75
Tunnel O(Port 16)

54
In the example above, an FCIP tunnel is created between tunnel 0 on port 1O/geO
(FC port 10/16) on the Brocade FR4-18i, and tunnel 0 on port geO (FC port 16) on
the Brocade 7500. For this tunnel, compression is enabled, and the committed rate
is set to 155,000 Kbits/sec, which matches the bandwidth of the OC-3 link that
connects the two sites. This committed rate will prevent TCP slow start issues
related to trying to push more data through a pipe than it is capable of handling.
Revision 0110 10 - 54
Verify FCIP Tunnel Settings

• Verify the FCIP tunnel settings with the port show f ciptunnel
[slot/ J port command:
B48000 : admin> portshow f ciptunne l 10/ geO 0

Slot : 10 Port : geO
Tunnel ID 0
Remote IP Addr 192 . 168 . 23 . 75
Local IP Addr 192 . 168 . 20 . 48
Remote WWN Not Confi gured
Local WWN 10 : 00 : 00 : 05 : 1e : 36 : 03 : 80
Compression on
Fastwrite off
Tape Pipelining off
Committed Rate 155000 Kbps (0 . 155000 Gbps)
SACK on
Min Retransmit Time 100
Keepalive Timeout 10
Max Retransmiss i ons B
Status I nactive 1
Uptime 1 mi n ute , 56 seconds

In the example above, the ports how f ciptunnel 1 OI geO O command

displays information about FCIP tunnel 0 on port 1O/geO on the Brocade 48000. The
FCIP tunnel parameters match those set earlier.
Footnote 1: The Status : Inactive message indicates that the FCIP tunnel
has been created., but the associated virtual FC ports are not online.
Revision 01 10 10 - 55
Verify, Enable, and Validate the VE_Port

• Before enabling the VE_Port, ensure that fabric merge-related
parameters (domain ID, zoning, etc.) are set so that the fabrics will
merge successfully1
• When the fabric merge-related parameters are set correctly, enable
the VE Port
- Review the swi tchshow command output to determine whether
- portenable or portcfgpersistentenable should be used2
B7500 : admin> switchshow

... <truncated output> . ..
15 15 N4 No Module
16 16 Offline Disabled ( Persistent )
17 17 Offline Disabled ( Persistent )
<Truncated Output>
B7500 : admin> portcfgpersistentenabl e 16

Footnote 1: VE- Ports are virtual E- Ports established over a FCIP tunnel. Some of
the parameters that cause VE_ Ports to segment include domain overlap, zoning,
incompatible fabric parameters. Note that these are the same parameters that will
cause E_Ports to segment (see fabs tatsshow help information).
Footnote 2: In the swi tchshow output above, VE_Port 16 is persistently disabled.
Revision 0110 10 - 56
Verify, Enable, and Validate the VE_Port (cont.)

• Validate VE_ Port functionality with the same commands used to
validate E_Ports: swi tchshow, fabricshow , and
topologyshow 1
B7500 : admin > switchshow

<Truncated Output> Successful VE_Port
15 15 N4 No Module
16 16 Onlin e VE-Port 10 : 00 : 00 : 05 : le : 36 : 04 : 06
" B48000 " (downstream)
17 17 Offl ine Disabled (Persistent)
<Truncated Output>
geO id lG Onli n e

Footnote 1: The fab r icshow output associated with establishing VE_Port

connections are no different than a f a b ricshow output established over E_Ports.
B7500 : admin> topologyshow
l domains in the fabric ; Local Domain 10 : l
B7500 : admin> fabricshow

Switch ID Worldwide Name Enet I P Addr FC IP Addr Name
1 : fffcOl 10 : 00 : 00 : 05 : le : 37 : 89 : dl 10 . 128 . 128 . 25 0.0.0.0 >" 87500 "
Revision 0110 10 - 57
VEX_Port Configuration - Overview

• Configuring a VEX-to-VE_Port (isolation) connection adds some steps to
the process:
- As before, all steps must be performed on both ports at either end of the link 1
1. Determine the parameters for the VE_Port, VEX_Port, and FCIP tunnel
2. Persistently disable the virtual FC ports associated with the tunnel 2
Steps 1 - 6
are exactly
3. Create an IP interface the same
4. Configure an IP route to specify a preferred gateway (optional) as VE-VE
connection
5. Verify the IP network between the two IP interfaces
steps 1 - 6
6. Create an FCIP tunnel
- Associate the local IP interface address with a tunnel ID, a committed rate , and an
remote IP interface address at the other end of the link
- As needed, configure FCIP features (SACK, compression, etc.)
7. Configure VEX_Port and configure LSAN zones in Edge/Backbone Fabrics
8. Enable the VEX_Port and VE_Port, and validate functionality

Footnote 1: The VEX_Port configuration steps include .all of the VE_ Port
config uration steps, plus the highlighted steps, step 7 and part of step 8. We will
present only the new steps.
Footnote 2: In addition to persistently disabling the virtual FC port, the GE port can
also be disabled.
Revision 0110 10 - 58
Create VEX Ports and LSAN Zones

• Configure VEX_Port(s) with the portcfgvexport [s l ot/ J port
command
- Options include: enable admin capability (-a), set the Fabric ID (-f), other
settings
- Omit the port information - display the current settings
/ '\
B7500 : admi n> portcfgvexport 17 -a 1 -f 10 State is NOT OK
B7500 : a dmi n> portcfgshow 17 because
Por t 17 i nfo associated logical
~A_d_m_i_· n_=~~~~~~~~~e_n_a_b_l_e_d~~---. ----------- FC port is not
l~s_t_a_t_e_:~~~~~~~~~N _O
_T~O_K~~__J enabled yet
Pid f o r ma t : No t Appl i cable
Edge Fa bri c ID : 10
Front WWN : 50 : 00 : 51 : e3 : 75 : 8a : 8e : 10
Preferred Domai n ID : 160
Fabric Parameters : Aut o Negotia t e

The portcfgvexport command is similar to the port cf gexport command

described in an earlier module
portcfgvexport [slotnumber/ ]portnumber [ -a admin ] [ -f
fabricid] [-r ratov] [-e edtov] [ -d domainid ] [ -p pidformat] [-t
fabri c_ parameter]
This command has the following operands (sometimes referred to as arguments):
-a admin: Specify 1 to enable or 2 to disable the admin.
-f fabricid: Specify 1 to 128 for the fabric ID.
-r ratov: Specify the R_A_ TOV used for port negotiation (E_D_ TOV*2 -
12000).
-e edt ov: Specify the E_D_TOV used for port negotiation (1000 - R_A_TOV/2).
-d domain id: Specify 1 to 239 for the preferred domain ID.
-p pidforma t : Specify 1 for core, 2 for extended edge, and 3 for native port ID
format.
-t fabric_ parameter: Specify 1 to enable or 2 to disable negotiate fabric
parameters.
Revision 01 10 10 - 59
Create VEX_Ports and LSAN Zones (cont.)

• As with FC Routing over FC, use LSAN Zones to share devices via
FC Routing over FCIP
- Devices being shared can reside in the Edge or Backbone Fabric
- Same rules for LSAN zone names, member definitions, etc.
• Similar to physical FC ports, virtual FC ports enable as VE_ Ports

by default, and must be configured explicitly as VEX_Ports
• LSAN zones can be defined before or after V EX Ports are

configured - the order of these operations does not matter

Revision 0110 10 - 60
Enable and Validate the Ports

• Enable the VEX Port and VE Port with the
portcfgpersistentenable or portenable commands
• In the Backbone Fabric, verify the routed fabric settings as with EX-
to-E Port links:
- fcrfabricshow
- lsanzoneshow - s
- f crphydevshow
- f crproxydevshow
- f crrouteshow
- f crxlateconfig
• To verify the VEX-to-VE_ Port link, use the same commands that
you would use to verify an EX-to-E_Port link - swi tchshow,
fabricshow, and topologyshow

Revision 0110 10 - 61
Enable and Validate the Ports (cont.)

• In the Backbone Fabric (7 500 ), swi tchshow shows VEX_ Port
and VE Port information
- VEX_Port: The WWN of the GbE port on the Edge switch, the name of
the Edge switch (480 00), and the Fabric ID of the Edge Fabric (10)
- VE_ Port: The morphed WWN of the Edge switch, and the XD for the
imported LSAN devices from the Edge Fabric (DID= 1, FID = 10)
B7500 : admin> switchsh ow
<Truncated Output >
17 17 Online VEX-Port 10 : 00 : 00 : 05 : le : 36 : 04 : 06 " 48000 " (fabric id - 101

VE-Port 50 : 00 : 51 : e3 : 78 : a6 : 6f : 9b " fer xd 1 10"
<Truncated Output>

The domain name fer - xd- 1 - 10 can be translated as::

fer = Indicates a connection from the edge fabric to an FC router
xd = Indicates that this domain is a translate domain
1 = Domain ID of the translate domain
10 = Fabric ID (FID) of the edge switch, the Brocade 48000
Revision 01 10 10 - 62

• In the Edge Fabric ( 4 8 000), swi t chshow displays VE_Port
information
- The manufactured WWN of the Backbone router
- The front domain presented by the BB fabric in the Edge Fabric
(DID=160)
B48000 : admin> swi tchshow

<Truncated Output>
sw i tchDomain : 10
sw i tchWwn : 10 : 00 : 00 : 05 : 1e : 36 : 04 : 06
<Truncated Output>
Area Slot Port Media Speed State
241 10 17 -- Online VE-Port 50 : 00 : 5l : e3 : 78 : a8 : 5e : Oa " fer .fd 160"

(downstream)
<Truncated Output >

The domain name fe r fd 160 can be translated as:

fer = Indicates a connection from the edge fabric to an FC router
fd = Indicates that this domain is a front domain
160 = Domain ID of the front domain
Revision 01 10 10 - 63

• In the Backbone Fabric, the fabricshow output shows an XO,
but no FD
B7500:admin> fabricshow
1 : fffcOl 50 : 00 : 5l : e3 : 78 : a6 : 6f : 9b 0.0. 0.0 0.0.0. 0 " fer xd l 10 "

101 : fffc65 10 : 00 : 00 : 05 : le : 37 : 8a : 85 192 . 168 . 23 . 200 0 . 0 . 0 . 0 >" B7500 "- -
The Fabri c has 2 switches
• In the Edge Fabric, the fabricshow output shows both an XO

and an FD 1
B48000:admin> fa.bricshow
1 : fffcOl 50 : 00 : 51 : e3 : 78 : a6 : 6f : 9a 0 . 0 . 0 . 0 0. 0.0.0 " fer xd 1 100 "

10 : fffcOa 10 : 00 : 00 : 05 : 1e : 36 : 04 : 06 192 . 168 . 20 . 20 0 . 0 . 0 . 0 >" B48000" -
160 : fffcaO 50 : 00 : 51 : e3 : 78 : a8 : 5e : Oa 0 . 0 . 0 . 0 0. 0.0.0 " fer fd 160 "
The Fabric has 3 switches

Footnote 1: Notice that this is the same output displayed on E_Ports connected to
an EX Port.
Revision 0110 10 - 64

• An FD and XO are created in Brocade 48000 edge Fabric A
• An XO is created in the Brocade 7500 BB Fabric B
Server Storage Gateway 192.168.20.1 Storage
Brocade 48000
with FR4-18i Blade
Fabric
---
Fabric B
A
VE_Port VEX_Port
192.168.W .48 192.168.23.75
Tunnel 1 (port 10117) Tunnel 1 (port 17)

Al 1R ghts Reserved
Revision 0110 10 - 65

• The BB Fabric B topologyshow output shows the connection
between the VEX_Port (port 17) and the XD
B7500 : admin> topologyshow
2 domains in the fabric ; Local Domain ID : 101
Local Translate Domain 1 owned by port : 17
• The Edge Fabric A, the topo l ogysh ow output shows the FD and
xo1
B48000 : admin> topol ogyshow Metric= 1845 (2000-
/ - - + Domain : 160
3 domains in the fabric ; 155 [155000 Kbits/sec])
I Metric : 1845
Local Domain ID : 10
Domain : l \ Name : fer fd 160
\
Metric : 11845
\ Path Count : 1
Name : fer xd 1 100
\
Path Count : 1 Hops : 1
\
Hops : 2 I Out Port : 10/17
Out Port : 10/17 I In Ports : 1/8 1/15
/
~
<Truncated Output > continued here .... .. Total Bandwidth : 0 . 155 Gbps
(adjusted)

Footnote 1: Notice that the metric associated with fc r _ xd_ l _ lOO is 1 1845;
devices are "routed" towards the translative domain. The metric associated with
front domain 160 (fcr_ fd_ 160) has a committed rate tunnel metric of 1845, or
2000 - 155 (155000 Kbits/sec). If we had specified an uncommitted tunnel, then the
tunnel would be allocated 1 Mbps bandwidth (it can use up to 1Gbit/sec -
committed rate tunnels) and the link cost would be calculated by taking 2000 - BW
(in Mbps)= 2000-1 = 1999.
The 2000 metric was chosen so that these virtual FC Port routing metrics would
look similar to 1 Gbit/sec ISL metrics: start at 2000, if all the bandwidth is allocated
to one tunnel by giving it a committed rate of 1Gbit/sec or 1000000 Kbit/sec then
that tunnel would have the same metric as a 1 Gbit/sec FC link (2000 - 1000 =
1000).
Revision 01 10 10 - 66
Web Tools - GbE Port Configuration
• Manage and monitor the

GbE port in the Port PIUrf- P-
Administration
Services dialog Pctl-
MACAdiSns
geO
OOOSo t e.<O:bcldS
Perlr,.,. O!gf-l'o<I
• GigE Ports: FCIP port Perl si.....

Pon Slat
Ptrmtenl[)solbied
Ofllne
Yn
statistics, and IP and
FC IP tunnel settings 1=- '°""'

Al 1R ghts Reserved
Revision 0110 10 - 67
Web Tools - FCIP Settings

• To configure IP interfaces, click the IP Interfaces tab
• Click the Add menu item to configure IP interfaces
(Add IP Interface) 0enor• p~..1oce. Plb.Ces
!.dd
Add IP lnterfdce rE)

Gf Port # : gel
PA~e.s' I I
s..t>netMasJc • l~---
MTUSae' i1soo ~
Add J Close ]

Revision 0110 10 - 68
Web Tools - FCIP Settings (cont.)

lC
'"' IOl(ICH01·7SOO)
Ill !:II
81 901
10 00000518378185 on
In the FCIP Tunnel tab, the New, Edit FCIP Port, and
Edit Configuration menu items launch the GigE Port .!.l
Configuration wizard
G1gE Port# 0 · Configuration
1Ns wozard wl Qlide you tlY0<.9> Gigf Port cor/91'otlon. n hos the tolowrov steps :
1. Overview
I. Set..., tine locol IP tntetfeceswlich wi be usedbyf<IP Tu:inels
2. Configure IP Interfaces
z. Set..., the IP ROU;os f0< tho do>tlnotloo IP Addresses. lhs st<1> Is cptlonol.
11 no IP Routes are specfied, deld routes wl be used.
3. Configure IP Routes
4. Select TUMel
S. Corfun the GlgE Port settrogs &. save to the switch.
S. FC!P Tunnel Configuration
6. A iU'lmlW'f tJ al the changes made on the <wtch wl be rej>Orted .and
6. Confumation and Save to Switch Ol"'f errors enco.rtered clmO the set i.c> wl be flaooed.
WARXING : GiVf POtt Conllgur- b o cls>uptive process. Doto trOMl'er throucJh the GiVf
7. Report pOtt / fCIP Tunnel moy be Interrupted os o result o( the FlNJSH button oct1on.

Al 1R ghts Reserved
The wizard will walk you through the configuration steps displayed above.
Revision 0110 10 - 69
Web Tools - Port Settings

• Use the FC Ports tab in the Port Administration Services dialog
to configure VE_Port and VE_Port settings
~ ; giJ
~
J fC Perts IGi\ePMs I
I liil Pcr; O •
(W Pcrt I Heath
iiil Pert 2
!iii Pert 3 10 9 Persistent o mine EX· Port N4 Yes Yes lrue tlue
Iii Pert 1 11 10 No_Modute Omlne EX· Port N4 Yes Yes true tiue
Perts 12 11 No_Module om1ne U-Port N4 Yes Yes true oue
Pcrt6
13 12 Online Healthy F·POrt N2 Yes Yes 1rue t1ue
Pert 7
(W Pcrt8 1' 13 Persistent o m1ne U-Port N4 Yes Yes true tJue
Iii Pert 9 15 14 Persistent Omlne U-Port N4 Yes Yes true l!Ue
Iii Pert 10 16 15 Persistent o m1ne U-Port N4 Yes Yes llue liue
iii Pert ti
17 16 Persistent om1ne U-Port Yes Yes false l!Ue
iii Pert 12
Iii Pert 13 Persistent o m1ne U-Port Yes Yes false l!Ue
iii Pert l<f 19 18 Persistent om1ne U-Port Yes Yes false IJUe
Iii Pert 15 20 19 Persistent o m1ne U-Port Yes Yes false tJue
Clil Pert 16 21 20 Persistent Omlne U-Port Yes false
Yes tJue
llill Pert 17
1111Pert18 22 21 Persistent o mlne U-Port Yes Yes false l!Ue
1111Pert19 23 22 Persistent om1ne U-Port Yes Yes false tiue
-~~

Al 1R ghts Reserved
Select the port from the list at the left to monitor and manage port-specific general
information, port statistics, and FCIP tunnel information.
[f.:u"i'tsJI <.q Ports J
0 101(NDA·T01·7SO: •
liil Port 0 1.-------
liii Port 1
fdot Ena'! 01 abl !'_ersistent Poorsistent Ennbloo Disable En•.11
Configuration " - • e Enable Disable Irunklng Trunking NPJV
iii Port 2
!iii Port 3
Iii Port 4 General
!iii Port 5
iiil Port 6 Port Number 16
[iii Port 7 Port Name
(m!Port8
l (iii Port 9
Port protocol FCIP
1- (iii Port 10 Port WWN 20:10:00:05:1 e:37:8a:8 5
liii Port ti Port Media
[ii Port 12
liil Port 13 Port Type U-Port
liil Port 14 Allowed Port Type VE-Port
liil Port 15
t
mi
mi
mmPort 17
Bandwidth Allocaled
Long Distance Mode
0 1550
NIA
lilt Port 18 Desired Distance (km) NIA
ii'iJ Port 19
Port status Persistent Disabled
WI Pc.t ZO
t ml Port 21 Controllable Yes
I.ill Port ZZ licensed Yes
ml Port 23
I.ill Port 24 Heatth Oftllne
lilil Port 2S Pnrt Index 16
liil Port 26 Trunking Enabled false
ml Port 27
ml Port 28 NPIV Enabled true
> ml Port 29 Additional Port Info Persistently disabled port
~~~ :? ~
• ___J...!.J • ...!.J
I I AD: ADO User: adOO I Role: A<iMl
Revision 0110 10 - 70
Web Tools - FCR Settings
~o
<!> O
Use the Fibre Channel Router wizard to

configure FC Routing parameters -
including VIEX_Ports over existing FCIP
tunnels
'DA '>107 Bl~ fCR Admm "11cro~ofl l ntl"rnl"t hcplorror l!I~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Fibre Channel Router

-FalxlclD: 100
The tabrte that Chis- &-'\lr1tch (acflng 11 an FCR) ts part ons knOWn as ltd backbone fabric You can conntct mis S'Mltt'l to 0Che1
fabnu lhrough EX·Pon' These olh.er t11>r1cs are cantd 1ctge fabncs They remain f'IOlated and do not merge'Mth the 1>ac:kbon1 tabnc
AN t.abrtu. tncludmg tht batl<bone fat:lrit, haw Iii uniqu• Fa.blic tD from the FCR pers:pec'fttie
... output continued in notes
C 2009 Brocade Co'Timun ca1 ans Systems nc i •

Al 1R ghts Reserved
Here is a complete snapshot of the Fibre Channel Router wizard:

'.} NDA-STOZ-875 - FCR Admm - Microsoft Internet EKplorer 1111~ Ei
[~.Jj EX-Ports I LSAN F8i:>ries I LSAN Zone• I LSAN DeW:es I
Fibre Channel Router

Backbone Fabric ID: 1 00
The fabric that this switch (acting as an FCR) Is part of is known as the backbone fabric. You can connect this switch to other
fabrics through EX-Ports. These other fabrics are called edge fabrics. They remain Isolated and do not merge wilh the backbone fabric.
All fabrics. including the backbone fabric. have a unique Fabric ID from the FCR perspective.
An FCR allows sharing of devices between edge fabrics and between the e dge fabric and the backbone fabric.
To share devices between any two fabrics, you must create an LSAN zone in both fabrics containing the WllJNs (World Wide Names)
of the devices to be shared. LSAN zones are configured the same way as regularzones. The only difference between a regular zone
and an LSAN zone Is thatthe name of an LSAN zone should begin with LSAN_ or tsan_ (not case s ensitive) and It should contain
only port WVVN members.
As part of the configuration ofFCR, you should follow the procedure given below. Please note that, to view or
configure other switches, you wlll have lo launch Web Tools on those switches.
1. Ensure lhattne backbone fabric ID of the switch Is the same as that of ol11er FCRs In the backbone fabric. This panel
displays the backbone fabric ID.
2. Ensure that the ports to be configured as EX-Ports are etther nol connected or are disabled.
3. Configure EX-Ports by clicking lhe New task in the !ask bar under the EX-Ports lab. As part oflhis configuration,
supply a fabric ID for the fabric lo which the portwlll gel connected. You can choose any unique fabric ID. as tong as
it Is consistenlfor an EX-Ports Illa! connectto me same edge fabric.
4. Connect the EX-Ports to the proper edge fabric if they are not already connected.
5. Configure LSAN zones on the fabrics thatwlll share devices using the Zone Administration module of Web Tools.
6. View the EX-Ports, LSAN Fabrics, LSAN Zones a.nd LSAN Devices tabs to make sure that your configuration has succeed ed.
Revision 0110 10 - 71
CFP 380 Internal Use Only FCIP Appendix
FCIP Best Practice Suggestions

• Review application needs to determine FCIP performance settings
- SACK, compression, jumbo frames, committed rates, fastwrite,
and tape pipelining - as well as application settings1
• Create an implementation plan that includes:
- IP interface and route settings
- FCIP performance settings
- FC Routing settings: Fabric IDs, domain IDs, LSAN zone definitions
• Follow step-by-step implementation procedures introduced in this
module
• Design the solution to meet the business problem and then ensure
that the solution solves the problem !

Revision 0110 10 - 72
FCIP License-Related Behavior

• FCIP and IPSec are both licensed per Brocade 7500(E) and 48000
chassis (FR4-18i)
- Adding/removing these licenses is non-disruptive (no reboot needed)
• Without an FCIP license:
- Not permitted: Create/delete FCIP tunnels; portcmd -- ipperf,
portcmd --ping, portcmd --traceroute
- Permitted: Show all settings; create/delete IP interfaces, IP routes, and
ARP entries
• What happens if an FCIP license is removed?
- All previously defined FCIP tunnels continue to be enabled - even if the
port is disabled , and re-enabled
- Additional FCIP tunnels cannot be defined
- Parameters for existing FCIP tunnels cannot be modified

The switches that can utilize a Fabric OS FCIP license are the Brocade 7500(E) and
the Brocade 48000 with a Brocade FR4-18i blade.
Revision 0110 10 - 73
FCIP Configuration Parameters

• Virtual ports associated with the Brocade 7500 and the Brocade
FR4-18i have configuration (conf igshow) parameters
- Blade configurations are stored on a slot basis; if blades are swapped,
the configuration stays; the new blades take the old , corresponding slot
configuration
• Similar to prior Fabric OS releases, a Fabric OS configupload
command w ill not cause 1/0 disruption
- A configuration upload does not have any dependency on Brocade
FR4-18i or Brocade 7500 state
- All FCIP tunnel related configuration is saved 1

Configuration files have virtual port port cf g key values. Even though the
16 FCIP Virtual/ILogical ports exist statically, the portcfg key value pair for
these ports is optional and when not explicitly present in the config file, those
ports will be assumed to have default configuration . Below is a sample
configuration file FCIP section:
KEY VALUE
Port Config Mode
portCfg . SlO . PO . MODE : FCI P
IP Interfaces
portCfg . S10 . PO . IFO : Idx=O , Ip= OxcOa8 14 0a , Mask= Oxffff ffOO , Mtu=2348
portCfg . S10 . P0 . IFl : Idx=l , Ip=Oxc0a8 14 14 ,Mask=Oxf fff ff00 ,Mt u =2348
Arp Entries
portCfg . S10 . P0 . ARPO : Idx=O , Ip=c0a81764 , Mac=00 : 06 : 5b : eb : 35 : ef
IP Routes
portCfg . Sl0 . PO . ROUTEO :Idx=3 ,Ip=Oxc0a81764 ,Mask=Oxffffff00 , Gateway=O
xc0a81401 , Metric= O
portCfg . Sl0 . PO . ROUTEl : Idx=3 , Ip=Oxc0a817c8 , Mask=Oxffffff00 , Gateway=O
xc0a8140 1, Metric=O
Revision 0110 10 - 74
FCIP Tunnels
portCfg . Sl0 . PO . FCIPTUNNELO : Idx= O, Remip=Oxc0a81764 , Locip=OxcOa
8 1 40a , RemWwn=OO : OO : OO : OO : OO : OO : OO : OO , LocWwn=l0 : 00 : 00 : 05 :1e : 36
: 04 : 06 , Comp=l, FWrt=O , CommRt=l55000 , Sack=l , MinRetrTm=l00 , KpAlv
=1 0 , Max Retr=8 , PthMtu =O , WanTOV=O , TapeAcc=O , IKE=O ,I PSEC=O , KEY=O
portCfg . Sl0 . PO . FCIPTUNNELl : Idx=l , Remip=OxcOa817c4 , Locip=Oxc0a
81414 , RemWwn= 00 : 00 : 00 : 00 : 00 : 00 : 00 : 00 , LocWwn=lO : OO : OO : OS : le : 36
: 04 : 06 , Comp= l , FWrt= O, CommRt=l55000 , Sack= l , MinRetrTm=l00 , KpAlv
= 1 0 , MaxRetr=8 , PthMtu=O , WanTOV=O , TapeAcc=O , IKE=O , IPSEC=O , KEY=O
Footnote 1: If the Brocade FR4-18i or a Brocade 7500 is disabled, a

conf i g u p l oad would still succeed in pushing the FCIP configuration to the config
file.
Revision 0110 10 - 75
FCIP Configuration Parameters (cont.)

• A conf igdownload that includes FCIP settings requires the
switch to be disabled first
- I/Os in FCIP tunnels are disrupted
• To apply additive downloaded FCIP configuration to a Brocade
7500 or Brocade 48000 with a FR4-18i GbE port, you will
additionally need to
- Reboot the Brocade 7500
- slotpoweroff/on the Brocade FR4-18i or reboot the Brocade 48000
• Delete all IP interface and FCIP tunnel configurations by disabling
the GbE port, invoking portcfgdefaul t , and then re-enabling
the port1

Footnote 1: Invoke the following commands to delete the tunnel created between
the Brocade 7500 port 17 Brocade 48000 10/17:
• From Brocade 7500: portcfgdefaul t geO ; portcfgdefaul t 17
•From Brocade48000: portcfgdefault 10/geO. Slot 10port17 acts as a
Virtual E_Port, it does not have and VEX_Port parameters to delete.
Note that because geO was defaulted, the FCIP parameters associated with the
connection between the Brocade 7500 port 16 and Brocade 48000 10/16 created
earlier would also be deleted. If the portcfgdefaul t command were invoked on
the Brocade 7500 port 16, the VEX_Port parameters would also be deleted.
Revision 0110 10 - 76
FCIP Troubleshooting Basics

• Verify FCIP license and platform support; use compatibility information
• If possible, verify physical connectivity using LEDs and Web Tools views
• Ensure that you configure IP interfaces on the GbE ports (portcfg ipif,
not ipaddrset)
- Watch out with jumbo frames - all components in the data path must support
them, or frames will be dropped
• Verify IP communication between the IP interfaces (portcmd --ping)
- If IP Ping fails, configure an IP route and/or add entries to ARP table, then test
again
• You can also use standard networking commands in the same LAN
segment to isolate network problems
- Example: Run a trace route (traceroute, tracert) to determine data path
- Example: Run a ping test along various segments of the data path to isolate
segment that is causing packet loss; start with the local default gateway1

Additionally, check with your Brocade 7500/FR4-18i provider for the latest
compatibility information.
You can additionally use Web Tools FCIP statistics along with portshow
fc i ptunnel and ipif statistical information to examine network characteristics.
One way to try to eliminate certain parts of the network that may be the cause of
packet loss is to do the ping test along various segments along the path. The first
place to start testing is the local "default gateway". This is the first router that all your
data is transmitted to on the network. If there is high packet loss on this segment,
then the problem is localized to your service provider's network.
Footnote 1: Ping may be filtered in the network.
Example networking commands include: ping, netstat , t racerout e , trace r t ,
tcpdump, if con fig , route. Some of these commands are OS dependent.
Revision 0110 10 - 77
FCIP Troubleshooting Basics (cont.)

• Ensure that the TCP ports used by FCIP are permitted access
through LAN/WAN security
- Ports 3225 (class F), 3226 (class 2,3), and 3227 (IPPe r f )
• Verify end device connectivity (f cping)
• Verify port/fabric status and FCIP tunnel configuration with the
swi tchshow, fabricshow,portshow,por t showfcipt unnel ,
and portcmd -- ipperf commands
• Besides reviewing the *show commands, FCIP errors are reported
in the switch error log: errors appear as part of the FCI P error
module

See the latest version of Fabric OS System Error Message Reference Manual (Publication
Number: 53-1000046-0x) for the most up to date FCIP error messag ing.
Validate the output of the following commands:
• swi t chshow
• fabricshow
• topo l ogyshow
• portshow
• fcr fabri cshow
• fcrphydevshow
• fcrproxydevshow
• fcrproxyconfig
• fcrtrou teshow
• fcrxlateconfig
• f cr r esou r ceshow
• l sanzoneshow - s
• fcp i ng
• portcmd --pi ng
The supports ave command output includes information from the following CU
commands: fcrproxydevshow , fcrphydevshow , portcfgexport, fcrxlateconfig,
fcrrouteshow, lsanzoneshow and fcrfabr i cshow.
Revision 0110 10 - 78
FCIP Troubleshooting Basics (cont.)

• If a VE-VE connection does not merge fabrics because of a
configuration mismatch, then use standard FC segmentation
troubleshooting methodology: examine swi tchshow, and
e rrshow, and port show outputs
• If VEX-VE connected end devices are not able to communicate
check physical connectivity, network connectivity, and then zone
configuration (zoneshow, and lsanzoneshow - s )
• When escalating trouble calls, use methodology described in FCR
module
- Use the supportsave command to collect all key information
including FCIP

Validate the output of the following commands:

• swi tchshow
• fabricshow
• topologysh ow
• portshow
• fcrfabricshow
• fcrphydevshow
• f crprox ydevshow
• fcrproxyconfig
• fcrtro u teshow
• fcrxlatecon f ig
• f crresourceshow
• lsanzonesh ow - s
• fcping
• portcmd --ping
The supportsave command output includes information from the following CLI
commands: fcrproxydevshow, fcrphydevshow, portcfgexport,
fcrxlateconfig, fcrrouteshow, lsanzoneshow -sandfcrfabricshow.
Revision 0110 10 - 79
Summary
• In this module, we discussed:
- FCIP concepts and terminology
- FCIP infrastructure
- FCIP as a means of extending the SAN beyond the physical
boundaries of the Data Center

Revision 0110 10 - 80
Revision 0110 10 - 81
BROCADE
EDUCATION SOLUTIONS
Revision 0110 10 - 82
VLAN Tagging - Overview

• In Fabric OS v6.0 +, FCIP tunnels now support virtual LANs
(VLANs)
- Logical or virtual networks within a larger physical network or group of
networks
- Provides the segmentation service otherwise performed by a network
router
- Supported by L2 Ethernet switches
- Specified by the IEEE 802.1 Q standard

A Virtual LAN is a logical or virtual LAN network within a single physical network, or
logical network that can span several physical networks.
• A traditional LAN requires all devices within a network to be part of the same
broadcasUmulticast domain, and thus, the same LAN boundary. This may force
unrelated devices and applications to be part of the same domain. It may also
force related devices that are in physically separate domains to be routed from
one domain to the other.
• In a VLAN, devices that were traditionally part of different LAN boundaries can
now be a ,,member' of a single network. In contrast, larger physical networks
containing many devices can now be broken down into multiple smaller
networks.
• VLANs are enforced on layer 2 Ethernet switches. In comparison, the
Differentiated Services feature introduced in Fabric OS v5.3 is enforced on
layer 3 Ethernet routers.
VLANs are defined in the IEEE 802.1Q standard.
Revision 01 10 10 - 83
VLAN Tagging - Overview (cont.)

• To implement VLANs, the IEEE 802.1Q standard adds four bytes -
the VLAN Tag - to the Ethernet header (Ethernet II framing 1 )
• The first two bytes are the EtherType value
- For VLANs, this is always set to the Tag Protocol ID (TPID) value
(Ox8100), which identifies the frame as a 802 .1Q frame
• The last two bytes are the Tag Control Information (TCI), which
provides VLAN-specific details
EtherType = TCI
Ox8100
Bits 31-16 Bits 15-0

Footnote 1: The IEEE 802.1Q standard does not encapsulate the Ethernet frame in
another header. Instead, 802.1 Q uses Ethernet II framing, also known as DIX
Ethernet (named after the major participants in the framing of the protocol: Digital
Equipment Corporation, Intel, Xerox), to define an upper- layer protocol. Switches
that support Ethernet 11 framing interpret the two-byte field that follows the
destination and source addresses as an EtherType that immediately identifies an
upper-layer protocol.
Revision 01 10 10 - 84

• The TCI field includes three values, two of which are user-defined:
• The user-defined VLAN ID, which identifies the specific VLAN to
which a FCI P-generated packet belongs
- 12 bits; up to 4094 user-defined values1
EtherType = VLANID
Ox8100
Bits 31-16 Bits 11-0

Footnote 1: The VLAN ID (VID) is a 12-bit field that specifies the VLAN to which the
frame belongs. A value of 0 means that the frame doesn't belong to any VLAN; in
this case the 802.1Q tag specifies only a priority and is referred to as a priority tag.
A value of OxFFF is reserved for implementation use. All other values may be used
as VLAN identifiers, allowing up to 4094 VLANs. On bridges, VLAN 1 is often
reserved for management.
Revision 0110 10 - 85

• The user-defined Class of Service (CoS) value, which identifies
priority of delivery
- Class of Service is not the same as Differentiated Services or Quality
of Service - no service-level guarantees, merely "best effort"1
- 3 bits; 8 user-defined values: 0 (best-effort) to 7 (highest priority)
• The fixed-value, one-bit Canonical Format lndicator2 (CFI) value
EtherType = 0 VLANID
Ox8100
Bits 31-16 Bits Bits 11-0

15-13 Bit 12
Footnote 1: The Class of Service (CoS) value is a 3-bit field within a layer 2
Ethernet frame header when using VLANs. It specifies a priority value of between 0
(signifying best-effort) and 7 (signifying priority real-time data) that can be used by
Quality of Service disciplines to differentiate traffic. Unlike Quality of Service (QoS),
traffic management protocols like Differentiated Services, Class of Service
technologies do not guarantee a level of service in terms of bandwidth and delivery
time; they offer a "best-effort". On the other hand, Cos technology is simpler to
manage and more scalable as a network grows in structure and traffic volume. One
can think of CoS as "coarsely-grained" traffic control and QoS as "finely-grained"'
traffic control.
Footnote 2: The Canonical Format Indicator (CFI) is a 1-bit value that is always set
to zero for Ethernet switches. CFI is used for compatibility between Ethernet and
Token Ring networks. If a frame received at an Ethernet port has a CFI set to 1,
then that frame should not be bridged to an untagged port.
Revision 0110 10 - 86
VLAN Tagging - Details

• Each FCIP tunnel may have a different VLAN Tag
- VLAN ID and CoS values can be the same or different across tunnels
on the same GbE port
- May have different Cos values for control and data packets
• On outgoing traffic, all packets have the same VLAN ID
- Control and data packets may have different CoS levels
• On incoming traffic, VLAN IDs are enforced by comparing the
VLAN ID on the packet with the VLAN Tag Table
- If the VLAN ID in the packet is not in the VLAN Tag Table, the frame is
discarded
- Separate VLAN Tag Table for each GbE port
- Entries in the VLAN Tag Table are done manually

Revision 0110 10 - 87
VLAN Tagging - Details (cont.)

• Each FCIP tunnel may have both Differentiated Services Code
Point1 (DSCP) and CoS settings:
- VLAN tagging enabled: both CoS and DSCP settings are in effect
- No VLAN tag: only the DSCP setting is used

Footnote 1: Differentiated Services (or DiffServ) is a standard for IP networks that

classifies, manages, and provides quality of service (QoS) guarantees for network
traffic. The DiffServ class is specified by a six-bit Differentiated Services Code Point
(DSCP) value, and is managed by
layer 3 Ethernet routers. The Tunnel ID DSCP priorify/ bits L2CoS priorify/ bits
default mapping of DSCP
0 46 / 101110 7 I 111
priorities to Cos priorities for
1 7 / 000111 1 / 001
each FCIP tu nnel are:
2 11/ 001011 3 / 011
3 15/ 001111 3 / 011
4 19 / 010011 3 / 011
5 23 / 010111 3 / 011
6 27 I 011011 0 / 000
7 31 / 011111 0 / 000
8 35/ 100011 0 / 000
9 39/ 10011 1 0 / 000
10 43 / 101011 4 / 100
11 47 I 101111 4 / 100
12 51/ 110011 4 / 100
13 55/ 110111 4 / 100
14 59 / 11 1011 4 / 100
15 63 / 111111 0 / 000
Revision 0110 10 - 88
Configure IPSec on an FCIP Tunnel

• To enable data encryption (IPSec) on an FCIP tunnel , requires four
steps: 1
- Ensure that the IPSec license is installed on both switches
(licenseshow)
- Specify the IPSec parameters, including the encryption policy,
authorization algorithm, and SA lifetime, in an IPSec policy
- Specify the IKE 2 parameters, including the encryption policy,
authorization algorithm, PFS enable/disable, Diffie-Hellman group, and
SA lifetime, in an /KE policy
- Enable IPSec on the FCIP tunnel , specifying the IPSec policy, IKE
policy, and the IKE pre-shared key

Footnote 1: IPSec is not supported with jumbo frames or 1Pv6.

Footnote 2: IPSec and IKE terminology:
Internet Protocol Security (IPSec) is a framework of open standards that help
ensure private, secure communications via cryptographic security over IP-based
networks. IPSec supports network-level data integrity, data confidentiality, data
origin authentication, and replay protection. Because IPSec is integrated at the
Internet layer (layer 3), it provides security for almost all protocols in the TCP/IP
suite, and because IPSec is applied transparently to applications, there is no need
to configure separate security for each application that uses TCP/IP.
IPSec needs a secret key to do data encryption. Internet Key Exchange (IKE) is
used to securely exchange these secret keys.
Internet Key Exchange (/KE) is a multi-step process that exchanges secret keys
between two network entities (FCIP tunnel ports). Fabric OS v5.2 uses IKE main
mode (Phase 1) negotiation, so the FCIP tunnel ports determine a specific set of
cryptographic protection suites, exchange keying material to establish the shared
secret key, and authenticate computer identities.
Revision 0110 10 - 89
Configure IPSec on an FCIP Tunnel (cont.)

• You can create up to 32 separate IPSec policies per FC router
domain
- Allows different IPSec policies per tunnel/edge fabric
- IPSec policies are persistent, and held local to the switch
• Create an IPSec policy with the policy -- create ipsec
command
- Specify the policy ID (<policyID>), encryption policy (- e n c ),
authorization algorithm (- au t h ), and SA lifetime (- s e c li f e )
B48000 : adrnin> po l icy --crea te ipsec 10 -en c AES-256 -auth AES-XCBC -secli f e 4 3200
B7500 : adrni n> pol i c y --cre a t e ipsec 10 -enc AES-256 -au th AES-XCBC -seclife 43200
Disp l ay/ delete one/all IPSec policy : policy --s h ow/--del ete ipsec [<policyID>] lal l

The policy --create ipsec command creates an IPSec policy. The

command has one required argument: <policy ID>, a unique identifier for the
policy (values: 1 to 32). For IPSec policies, the command also has the following
optional parameters:
• -enc: The encryption algorithm used in this policy. Accepted values are NONE,
3DES , AES-128, and AES-2 5 6. Default (IPSec): AES- 1 28.
• -au th: The authentication algorithm used in this policy. Accepted values are
NONE, SHA-1, MDS, and AES-XCBC . Default: SHA-1.
• -seclife: The security association lifetime, in seconds; values may range
from 28800 to 250000000, or 0. Default: 28800 (8 hours).
• If no optional parameters are specified, the command displays the currently
In the command above, IPSec policy 10 is created on both the Brocade 7500 and
48000. The authentication algorithm is AES-XCBC, the encryption algorithm is AES-
256, and the SA lifetime is 43200 seconds (12 hours).
Revision 0110 10 - 90

Example
• Enable IPSec data encryption with the portcfg fcip t unnel
c reat e command and these arguments:
- IPSec policy: - ipsec < I PSec_poli cyID>
- IKE policy: - ike <IKE_policyID>
- IKE pre-shared key (12-32 characters): - key " key_value "
- Values must match at both ends of this tunnel
• A secure tunnel cannot be modified - you must delete it (port cf g
f c i ptunnel de l ete ), then recreate it (por tcfg fciptu n nel
create )

The three IPSec-related arguments for the existing portcmd fciptunnel

create command specify the needed IKE and IPSec policy values:
• -ipsec: The IPSec policy to be used with this tunnel. Value specify must
correspond t.o an existing IPSec policy.
• -i ke : The IKE policy to be used with this tunnel. Value specify must
correspond to an existing IKE policy.
• - key: The pre-shared key used during IKE authentication. Specified as a
double-quoted string of alphanumeric characters. The length of this key must
be in the range of 12 to 32 characters.
• These arguments may be introduced in any order.
Revision 01 10 10 - 91

Example (cont.)
• Configure the same FCIP tunnels, this time with IKE policy 11 ,
IPsec policy 10, and the IKE pre-shared-key "ipsec123456789"
B4 8000 : admin> portcfg fciptunnel 10/geO create 0 192.16:8.23 . 75
192 . 168 . 20 . 48 155000 -c -ike 11 -ipsec 10 -key " ipsec123456789 "
B7500 : admin> portcfg fciptunnel geO create 0 192 . 168 . 20 . 48

192 . 168.23 . 75 155000 -c - ike 11 -ipsec 10 -key " ipsecl23456789 "
Server Storage
Brocade 48000
with FR4-18i Blade
Brocade 7500
WAN
Fabric
B
VE_Port
192.168.20.48
Tunnel 0 (Port 10/16) -- VE_Port
192.168.23.75
Tunnel 0 (Port 16)

92
In the command above, FCIP tunnels are created on the Brocade 48000 and 7500,
using the same tunnel IDs, remote IP address, local IP address, and committee
rates are used. On both switches, IPSec policy 1 O and IKE policy 11 are selected,
and the IKE key is ipsec123456789.
Reminder: Create policies and configure security related parameters over a secure
link.
Revision 01 10 10 - 92

CFP380 StudentGuide

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

CFP380 StudentGuide

Загружено:

Авторское право:

Доступные форматы

Internal Use Only

Internal Use Only

Corporate Headquarters - San Jose, CA USA

European Headquarters - Geneva, Switzerland

Asia Pacific Headquarters - Singapore

© 2010 Brocade Communications Systems, Inc. All Rights Reserved.

Revision: May, 2010

C 2009 Brocade Commun ca1 ans Systems Inc 2

• Discuss Brocade Fibre Channel over Internet Protocol (FCIP)

• Configure Brocade Adaptive Networking

• Discuss virtual connectivity using NPIV and Brocade Access

• Implement Fabric OS security policies

• Manage advanced features using DCFM

Course Objectives (cont.)

• Identify Brocade switch and director hardware platforms (WBT)

• Describe Fabric OS long distance options (WBT)

• Describe Fabric OS interoperability options (WBT)

Instructor-Led Course Agenda

C 2009 Brocade Commun ca1 ans Systems Inc 5

This course also includes Web Based Training components.

Instructor-Led Course Agenda (cont.)

C 2009 Brocade Commun ca1 ans Systems Inc 6

This course also includes Web Based Training components.

Web-Based Training Agenda

C 2009 Brocade Commun ca1 ans Systems Inc 7

C 2009 Brocade Commun ca1 ans Systems Inc 8

What's Your BCFP Study Plan?

C 2009 Brocade Co-nrn.J"'I cat ans Sys!ems Inc 9

Brocade Certification and Accreditation Program

c 2009 Brocade Communica11ons Systems Inc 1O

Brocade Education website:

Registering for a Certification Exam

C 2009 Brocade Commun ca1 ans Systems Inc 11

Registering for an Accreditation Exam

• Kryterion's Webassessor tool is our chosen method for delivering

C 2009 Brocade Commun ca1 ans Systems Inc 12

Going Green and Taking Notes

C 2009 Brocade Commun ca1 ans Systems Inc 13

Footnote 1: Use "CTRL" and "6" as a shortcut to create PDF notes.

Training Facility and Training Policies

- Discuss Fibre Channel-to-Fibre Channel routing terminology, theory

- Identify additional FC-FC routing features

C 2009 Brocade Commun ca1 ans Systems Inc 2

Revision 0110 2-2

• Allows device access

C 2009 Brocade Co'Timun ca1 ans Systems nc 3

FC-FC Routing (cont.)

C 2009 Brocade Co'Timun ca1 ans Systems nc 4

Footnote 1: Includes implementing and configuring the underlying physical

Revision 0110 2-4

FC-FC Routing Terminology

C 2009 Brocade Co'Timun ca1 ans Systems nc 6

Revision 0110 2-6

FC-FC Routing Terminology (cont.)

Complex Backbone 1 Complex Backbone 2

C 2009 Brocade Commun ca1 ans Systems Inc 7

FC-FC Routing Terminology (cont.)

- A total of 128 FIDs

- Both backbone and

C 2009 Brocade Co'Timun ca1 ans Systems nc 8

Terminology Review (cont.)

The following output does not correspond to the diagram.

To set the FID for an edge fabric use port cf gexport :