Content

SAN-TS 300
Brocade SAN
Troubleshooting

Student Guide

Brocade Education Solutions
Revision 0213

Corporate Headquarters - San Jose, CA USA
T: (408) 333-8000
info@brocade.com

European Headquarters - Geneva, Switzerland
T: +41 22 799 56 40
emea-info@brocade.com

Asia Pacific Headquarters - Singapore
T: +65-6538-4700
apac-info@brocade.com

2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade, the Brocade B-weave logo, Fabric OS, File Lifecycle Manager, MyView, Secure Fabric
OS, SilkWorm, and StorageX are registered trademarks and the Brocade B-wing symbol and
Tapestry are trademarks of Brocade Communications Systems, Inc., in the United States and/or
in other countries. FICON is a registered trademark of IBM Corporation in the U.S. and other
countries. All other brands, products, or service names are or may be trademarks or service
marks of, and are used to identify, products or services of their respective owners.

Notice: This document is for informational purposes only and does not set forth any warranty,
expressed or implied, concerning any equipment, equipment feature, or service offered or to be
offered by Brocade. Brocade reserves the right to make changes to this document at any time,
without notice, and assumes no responsibility for its use. This informational document describes
features that may not be currently available. Contact a Brocade sales office for information on
feature and product availability. Export of technical data contained in this document may require
an export license from the United States government.

Revision: February, 2013
DCX 200
SAN-TS 300
Revision 0213
1 1
Course Introduction
SAN-TS 300
Revision 0213
1 2
Course Introduction
SAN-TS 300
Revision 0213
1 3
Course Introduction
Revision 0213
SAN-TS 300
1 4
Course Introduction
Revision 0213
SAN-TS 300
1 5
Course Introduction
Revision 0213
SAN-TS 300
1 6
Course Introduction
Revision 0213
SAN-TS 300
1 7
Course Introduction
SAN-TS 300
Learn more about the program at our website:
http://www.brocade.com/education/certification-accreditation

Revision 0213
1 8
Course Introduction
SAN-TS 300
Revision 0213
1 9
Course Introduction
SAN-TS 300
Registering for a certification exam:
Visit http://www.pearsonvue.com/brocade
Call 866-361-5817 toll-free in North America
Visit http://www.pearsonvue.com for other contact numbers worldwide (some
locations may not have toll-free numbers)
Registering for an accreditation exam:
https://www.webassessor.com/wa.do?page=publicHome&branding=BROCADE

Revision 0213
1 10
Course Introduction
Footnote 1: Brocade University releases nutshell guides for each certification exam. The
guides are named after the exam, i.e. BCFP in a Nutshell, and are available from the
Brocade University certification page: http://www.brocade.com/education/certification-
accreditation.
Revision 0213
SAN-TS 300
1 11
Course Introduction
Facebook Brocade Certified
http://www.facebook.com/pages/Brocade-Certified/161604617227755
LinkedIn Brocade Certified
http://www.linkedin.com/groups?gid=3752161&trk=hb_side_g
MyBrocade Brocade University Community
http://community.brocade.com/community/forums/education
MyBrocade Certification Community
http://community.brocade.com/community/forums/education/certification
MyBrocade Education Alumni Community
http://community.brocade.com/community/forums/education/alumni
SAN-TS 300
Revision 0213
1 12
Course Introduction
For a list of Brocade University courses please see our website:
http://www.brocade.com/education/product-training/index.page.
SAN-TS 300
Revision 0213
1 13
Course Introduction
Footnote 1: Use CTRL+6 as a shortcut to create PDF notes.

Revision 0213
SAN-TS 300
1 14
Course Introduction
Course Introduction
1 15
Revision 0213
SAN-TS 300
Course Introduction
1 16
Revision 0213
SAN-TS 300
Course Introduction
1 17
Revision 0213
SAN-TS 300
SAN-TS 300 Course Introduction
1 18
Revision 0213
Course Introduction
1 19
Revision 0213
SAN-TS 300
Course Introduction
1 20
Revision 0213
SAN-TS 300
Course Introduction
1 21
Revision 0213
SAN-TS 300
Revision 0213
SAN-TS 300
1 22
Course Introduction
Revision 0213
SAN-TS 300
1 23
Course Introduction
SAN-TS 300
Revision 0213
1 24
Course Introduction
Troubleshooting Overview
2 1
Revision 0213
SAN-TS 300
2 2
Revision 0213
SAN-TS 300
2 3
Revision 0213
SAN-TS 300
Footnote 1: Configuration problems can also be related to application specific
configuration requirements. For example, some applications or devices may not support
exchanged-based routing. These applications require that the fabric switches be
configured for port-based routing.

Here is a partial list of helpful commands associated with identifying these problems; all
problem determination steps include switchshow and errshow:
Timeout/sluggishness: urouteshow, topologyshow, porterrshow,
portshow, portstatsshow, portcfgshow, portbuffershow, and
aptpolicy (check routing configuration)
Segmented fabric: configshow, fabricshow, fabstatsshow, portshow,
portcfgshow, check zone related commands, and license configuration
Port/node configuration: portcfgshow, configshow, portlogdump,
portshow, fabricshow, trunkshow, portcfglongdistance,
licenseshow, and portshow
Missing device: Check physical connectivity using switchshow, portshow, and
fcping. Check fabric connectivity with nsallshow, nsshow, nscamshow,
zoning(zoneshow, etc.) and port configuration commands (portcfgshow,
portshow). Optionally use a diagnostic tests such as porttest or D_Port
diagnostics because this will test the port and link components.

2 4
Revision 0213
SAN-TS 300
For marginal links use D_Port tests or the porttest command to troubleshoot link
issues.

2 5
Revision 0213
SAN-TS 300
2 6
Revision 0213
SAN-TS 300
Footnote 1: Example if there is a performance issue with a server are other servers also
having problems? If so what severs, knowing this will help in the problem resolution.

2 7
Revision 0213
SAN-TS 300
Footnote 1: If the problem is a device that cannot log into the fabric capturing a
supportsave from the switch and HBA (if Brocade HBA), and server syslog will be
enough. If the problem is that the server cannot see storage, capturing a
supportsave from each switch in the path is required. If the issue is performance
then capturing a supportsave from each switch in the fabric is required.

2 8
Revision 0213
SAN-TS 300
Taking the supportsave after you have already started to troubleshoot the problem
can make resolution determination harder and may introduce false positives into the
supportsave data.
Brocade Network Advisor can be used to easily collect and store support save data from
multiple switches simultaneously.
During the supportsave process in the Fabric OS, the *.dump files get moved to
*.old.dump, the old file gets overwritten.
2 9
Revision 0213
SAN-TS 300
2 10
Revision 0213
SAN-TS 300
Footnote 1: The 80 means end of list, so there are no other devices that the server
currently has access to. If this were 00 instead of 80 that would mean there are
additional devices that the host has access too. Remember for a 24 bit address to be
included in this Name Server query, the device must be currently logged in and the
server must have access (zoned).

See appendix portlogdump module for more information on this output:

13:43:11.682 nsd ctout 10 fc 00038002,00010a00,80020b00
13:43:11.682 PORT Tx3 10 32 03010a00,00fffffc,80495d17,01000000

8002 is an
accept to
the CT
request
03 Response
010a00 is the DID
address (Server)

ffffffc SID which is
the Directory
(Name) Server

80 means end of
the list. 020b00 is
the address of the
storage device
connected to port
11
010a00 is the
address of the
server (Of course
the host will have
access to itself)
2 11
Revision 0213
SAN-TS 300
When working with the port counters it is important to remember that the numbers
displayed have been accumulating since the switch was last rebooted or the stats last
cleared. Because of this it is necessary to either clear the stats and wait or take a
baseline and note any increases.
2 12
Revision 0213
SAN-TS 300
2 13
Revision 0213
SAN-TS 300
2 14
Revision 0213
SAN-TS 300
The fabricshow command can be found in the SSHOW_FABRIC.txt file from the
supportsave capture.
Use this command to display information about switches in the fabric.
If the switch is initializing or is disabled, the message "no fabric" is displayed.
2 15
Revision 0213
SAN-TS 300
The islshow command can be found in the SSHOW_FABRIC.txt file from the
Use the islshow command to display the current connections and status of the
interswitch link (ISL) for each port on a switch. The command output includes the
following information:
Node world wide name (WWN)
Domain ID
Switch name
ISL connection speed, if applicable
Bandwidth
Trunking enabled, if applicable
QoS enabled, if applicable
Encryption enabled, if applicable
Compression enabled, if applicable

2 16
Revision 0213
SAN-TS 300
The trunkshow command can be found in the SSHOW_FABRIC.txt file from the
Use this command to display trunking information of both E_Ports and EX_Ports.
Port to port connections
Displays the port-to-port trunking connections.
WWN: Displays the world wide name of the connected port.
Domain: Displays the domain IDs of the switches directly connected to the physical
ports. In case of an FC Router backbone fabric interlinking several edge fabrics, the
domain ID displayed for an E_Port trunk refers to a domain of a switch within the
backbone fabric, whereas the domain ID displayed for an EX_Port trunk refers to the
domain ID of a switch in the edge fabric. Because they are independent fabrics, it is
possible that both the backbone and the edge fabric may have the same domain ID
assigned to switches. If this is the case, run switchshow to obtain information on the
port types of the local switch and the WWNs of the remote switches. Refer to the
Example section for an illustration.
Deskew: The difference between the time it takes for traffic to travel over each ISL
compared to the time it takes through the shortest ISL in the group plus the minimum
deskew value. The value is expressed in nanoseconds divided by 10. The firmware
automatically sets the minimum deskew value for the shortest ISL, which is 15.
Master: Displays whether this trunking port connection is the master port connection for
the trunking group.
2 17
Revision 0213
SAN-TS 300
2 18
Revision 0213
SAN-TS 300
Divide and Conquer is a troubleshooting methodology that involves taking a system and
breaking it up into smaller testable components. By moving through the system in a
systematic fashion you can, by thorough testing, identify and isolate parts of the system
that could potentially cause a problem.
The most important part is knowledge of the system you are trying to troubleshoot.
Knowing the technologies involved and how they interconnect and interact is essential
to know where to divide the system and how to eliminate potential problems.
A Brocade fabric can be separated into a number of individual components. The list
below is an example but is not all inclusive:
Storage devices
Hosts
Fabric switches
Cables / Patch panels
2 19
Revision 0213
SAN-TS 300
2 20
Revision 0213
SAN-TS 300
If a host does not see a particular storage device then check the following using CLI,
Web Tools or Brocade Network Advisor:
Is the device physically connected? If both devices do not appear as an F_Port,
FL_Port or an L_Port then it may not have a good physical connection. Look for a
marginal link or other initialization-related problem.
If the device has a good physical connection then ask yourself, is the device
logically connected? (Is it present in the Name Server? Use CLI commands such
as nsshow, nscamshow, and nsallshow or GUIs such as Webtool or Brocade
Network Advisor to determine if the fabric can see each device.)
In the case of one device that can not see another you may have to additionally
examine zoning configuration and link error counter information to make sure end
devices are in the same zone and one of them isnt bouncing (marginal) this
would clearly show up in the port log.

This goes back to the Divide and Conquer process: where did the breakdown occur? At
the link level or at the logical level?

2 21
Revision 0213
SAN-TS 300
The fabric in this example has five switches and devices attached, a deterministic path
exists and can be used to isolate this problem. The problem as described is that the
host on Switch3 cannot see one of the paths to the storage that is on the Switch2. A
path (in green) can be drawn that shows the connection the host and storage are
attempting to use. The other devices and switches in the fabric at this point should be
considered as non-existent until such time as they need to be existing again.

2 22
Revision 0213
SAN-TS 300
2 23
Revision 0213
SAN-TS 300
12
7
5
8
14
3
Switch1
Switch4
Switch2
Switch3
Host
Storage
Switch5
The G_Port being online indicates a problem. The device connected to that port has a
good link (it shows Online) but did not successfully get far enough into the process to
become either an E_Port or an F_Port (the port did not receive a FLOGI or ELS frame). If
the device did not come up as a G_Port and was still not physically connected, it would
come up with one of the following port states: No_Light (not receiving), No_Sync (not
synchronizing), In_Sync (receiving light and in synchronization but unable to go further in
initialization process), Laser_Flt, Port_Flt, Diag_Flt (diagnostics failed during bring up),
or Testing (which would explain why you do not see the device). You want to see Online.
switchshow port state Information:
No_Card no interface card present
No_Module no module (GBIC or other) present
No_Light module not receiving light
No_Sync module receiving light but out of synchronization
In_Sync module receiving light and in synchronization
Laser_Flt module signaling a laser fault
Port_Flt port marked faulty
Diag_Flt port failed diagnostics
Lock_Ref locking to the reference signal
Testing running diagnostics
Online port is up and running
2 24
Revision 0213
SAN-TS 300
Footnote 1: If moving the cable to another port and the storage device logs in, check the
original port configuration and try the SFP in the working port. If the device still will not
log in check the cable and the storage device. Also check the switch port for errors, such
as CRC errors (which generally indicates a physical problem). Also if there is a patch
panel involved check the connections on the patch panel.
2 25
Revision 0213
SAN-TS 300
2 26
Revision 0213
SAN-TS 300
Note: LLFD = Link, Login, Fabric, Devices

2 27
Revision 0213
SAN-TS 300
Establishing link is the first step in connecting to a fabric. To establish a link the device
and switch ports will start transmitting a signal. This signal is used to negotiate speed
and synchronize character and word boundaries in the transmission.

In the next few slides we will continue our overview of the LLFD concept. LLFD will be
discussed in much greater detail later in this course.
2 28
Revision 0213
SAN-TS 300
Footnote 1: If security is enabled there will also be an additional security policy check after the
FLOGI. The switch will check the Device Connection Control Policy (DCC) Access Control Lists
(ACL) to verify that the device requesting a login is permitted to attach to the fabric. This will
generate one of two responses:
Accept Assign fabric unique 24-bit address
Deny No response, do not assign fabric address
Footnote 2: Once logged into the Name Server, there is an implied login to all well known
address:
FFFFFF Broadcast Server
FFFFFE Fabric Login
FFFFFD Fabric Controller
FFFFFC Directory/Name Server
FFFFFB Time Server
FFFFFA Management Server
FFFFF8 Alias Server
FFFCxx Embedded Port (Domain Controller)
Footnote 3: Initiators should make a State Change Registration (SCR) prior to initiating a PLOGI
to a target. By issuing the SCR, they will ensure they are notified of any changes within their
zoning configuration prior to initiating communications with any targets. They may issue the SCR
after logging into a target, but the possibility exists that something may happen to the target
after they login and before they register to be notified of changes by the Name Server. For this
reason, the SCR usually occurs immediately after the PLOGI into the fabric.

2 29
Revision 0213
SAN-TS 300
To communicate with other end devices, the device must register with and query the
Name Server. Many Host Bus Adapters (drivers) and storage devices will send standard
SCSI Inquiry data to the switch for registration. This data can be very useful for
identifying a particular device. Depending on the vendor you may also get additional
data such as firmware and driver versions. Name server registration takes place after
the device performs a FLOGI to the Fabric Controller and then a PLOGI to the name
server port.
2 30
Revision 0213
SAN-TS 300
Footnote 1: This is not limited to initiators, some target devices will also query the name
server to see what devices has access to it, and will reject login requests from devices
that do not have access to it.
Footnote 2: There are several different query commands to get information about the
devices that an initiator has access to. Which query commands the server sends is
dependent on the driver for that device. Different initiators can send different query
commands.
Footnote 3: This is based upon the type of device that has registered. Type 8 is SCSI
FCP (Fibre Channel Protocol). Type 5 is IP/FCIP.
Footnote 4: Brocade Fabric OS switches log into each device in the fabric and probe for
additional information to populate into the Name Server. Device probing is on by default
but can be disabled using the configure command. Some initiators will reject this
probing which is OK. Target devices generally allow the probing. The SID from the switch
for this probe will be FFFCxx (where xx is the domain ID in hex of the switch).

2 31
Revision 0213
SAN-TS 300
2 32
Revision 0213
SAN-TS 300
If there are problems with end devices communicating with each other, start troubleshooting
from the switch and work toward one of the affected end devices
Common mistakes with LUN Masking include:
Initiator Node Wide Node Name (NWWN) defined when Port World Wide Name (PWWN) is
required (or both are required)
Wrong or no LUNs enabled for that particular initiator
Note: LUN Masking will sometimes be referred to using vendor specific terms such as
"LUN Security" or "LUN Mapping"
Common mistakes with persistent binding:
New device presented from storage, but not added to persistent binding list on host may
prevent device from being seen by the OS
Replaced device may need modification within persistent binding file
Note: Persistent binding could be done by HBA utility or within OS specific file
While these issues are beyond the scope of this course, verification of switch related
connectivity and availability will help isolate the problem to host OS driver, array LUN masking,
or persistent binding configuration file issues
Use host logs and utilities to verify whether device connectivity exists:
Can you gather inquiry data of a device from the host?
Can you access the device from the host?

2 33
Revision 0213
SAN-TS 300
Brocade Connect is the technical Web portal and online community for the Brocade
installed base. It empowers customers with self-service technical info, tools, and
community features that let them find answers to their questions, optimize their SAN
investment, and increase their productivity. Gain your customers' mind share, loyalty,
and appreciation by driving them to Brocade Connect on a daily basis. Best part it's
free!
2 34
Revision 0213
SAN-TS 300
2 35
Revision 0213
SAN-TS 300
2 36
Revision 0213
SAN-TS 300
Data Gathering
3 1
Revision 0213
SAN-TS 300
Data Gathering
3 2
Revision 0213
SAN-TS 300
Data Gathering
3 3
Revision 0213
SAN-TS 300
Footnote 1: Steps to capture command output
1. Connect to the switch through a Telnet or SSH utility.
2. Log in using an account assigned to the admin role.
3. Set the Telnet or SSH utility to capture output from the screen.
Some Telnet or SSH utilities require this step to be performed prior to opening up a
session. Check with your Telnet or SSH utility vendor for instructions.
Footnote 2: Additional information about supportshow output that is captured as part
of supportsave is described in another section of this module.
Footnote 3: Fabric OS v6.2 and later requires a console connection only when
troubleshooting boot problems (where the switch panics during POST). See Gather
Fabric OS console output and logs

in appendix material associated with this module for
additional information. Also, the Fabric OS v6.2 supportsave data capturing process
includes console output in a file that ends with *.RAS_POST. If auditing is
configured then you can also use the CLI to capture Audit messages; configure
capturing and then invoke the auditdump - s command.

Data Gathering
3 4
Revision 0213
SAN-TS 300
Data Gathering
3 5
Revision 0213
SAN-TS 300
Additional information about what supportsave captures is shared later in this section.
Footnote 1: Example of a B8510 switch: SW8510-S4cp-
201205021421.SSHOW_EX.txt.gz
For director class switches you will see files for both CPs (S4 and S5)
Footnote 2: This tool is not available to the general public.

Data Gathering
3 6
Revision 0213
SAN-TS 300
Additional information about what supportsave captures is shared throughout this
section. The number of files generated varies depending on switch type, Fabric OS level
and features (such as virtual fabrics). Example a B6510 switch running Fabric OS v7.1.0
will generate 50 files.
Example of a supportsave capture:

B8510:FID128:admin> supportsave
This command collects RASLOG, TRACE, supportShow, core file, FFDC data and
other support information from both active and standby CPs and then
transfer them to a FTP/SCP/SFTP server or a USB device. Local CP, remote
CP and BPs' information will be saved, but supportShow information is
available only on the Active CP. This operation can take several minutes.
NOTE: supportSave will transfer existing trace dump file first, then
automatically generate and transfer latest one. There will be two trace
dump files transferred after this command.
OK to proceed? (yes, y, no, n): [no] y

Host IP or Host Name: 10.255.252.50
User Name: dev
Password: <password>
Protocol (ftp | scp | sftp): ftp
Remote Directory: /8510ss

Saving support information for switch:B8510-4, module:RAS...
................................
Saving support information for switch:B8510-4, module:CTRACE_OLD...

Data Gathering
3 7
Revision 0213
SAN-TS 300
In this example the files would be B8510-4-Sx.RAS_POST.txt (where x is the slot
number. This is how you can determine which CP is the active and which is the
standby.
Data Gathering
3 8
Revision 0213
SAN-TS 300
This is an example supportsave
output from a B8510; it is truncated
because it can not all fit on the slide.
Notice the *-S4cp-* and *-S5cp-* files;
these represent trace dump, *.ss.gz
(engineering only) and *.CHKRPM.gz
(RedHat files from each CP (S4 is the
CP in slot 4 and S5 is the CP in slot 5).
The output also includes *.SSHOW*
files from each CP. These files
represent Fabric OS v7.1.0
supportshow output. The standby
CP supportshow will ONLY include
command output available on the
standby CP. You will see the following
message when you log into a standby
CP:
***************************
Logging into STANDBY CP,
not all commands are fully
supported !!
***************************
Invoke help from a standby CP to see
a list of available commands; Fabric OS
v7.1.0 standby CP commands include:

classconfig, cmsh, dbgshow,
Errclear, errdump,
Errmoduleshow, errshow,
fabriclog, fapwwn, fastboot,
Firmwarecommit,
firmwaredownload,
Firmwaredownloadstatus,
Firmwarekeyshow,
Firmwarerestore,
Firmwareshow, fosexec, grep,
H, hadump, hashow, help,
ifmodeset, ifmodeshow,
Killtelnet, login, logout,
Memshow, more, myid,
netstat, pdshow, ping,
ping6, reboot, roleconfig,
Setdbg, sleep, supportsave,
Switchviolation, top,
tracedump, uptime, version
Data Gathering
3 9
Revision 0213
SAN-TS 300
If Virtual Fabrics are enabled, commands are checked for context and switch type as follows:
Virtual Fabric context (VF) = Command applies to the current logical switch only, or to a
specified logical switch
Virtual Fabric commands are further constrained by one of the following switch types:
All Switches (All) = Command can be run in any switch context.
Base Switch (BS) = Command can be run only on the base switch
Default Switch ((DS) = Command can be run only in default switch
N/A = Switch Type is not applicable to the command
Chassis context (CH ) = Command applies to the chassis on which it is executed
Switch and Chassis context (VF/CH) = Command applies to the switch and the
chassis
Disallowed = Command can not be executed when Virtual Fabrics are enabled

Data Gathering
3 10
Revision 0213
SAN-TS 300
Command Name User Admin Oper Sw
Admin
Zone
Admin
Fabric
Admin
supportsave O OM OM OM O OM
supportshow O OM OM OM O OM
Command Name BS
Admin
Sec
Admin
Admin Domain Context
1
Switch
Type
supportsave O OM Disallowed CH N/A
supportshow O OM Disallowed VF All
O = observe; OM = observe-modify; CH = chassis context; VF = Virtual Fabric
Additional supportsave information:
supportsave [-n] [-c] [-k] [-u user_name -p password -h
host_ip -d remote_dir -l protocol]
supportsave [-R]
supportsave [-U -d remote_dir]
supportsave [-t timeout_multiplier]
When invoked without operands, this command goes into interactive mode. The
following operands are optional:
-n Does not prompt for confirmation. This operand is optional; if omitted, you are
prompted for confirmation.
-c Uses the FTP or SCP parameters saved by the supportftp command. This operand
is optional; if omitted, specify the FTP or SCP parameters through command line
options or interactively. To display the current FTP parameters, run supportftp (on
a dual-CP system, run supportftp on the active CP).
The -c option is mutually exclusive with -u, -p, -h, and -d.
-k Specifies that the supportftp auto file transfer configuration transfer only core and
FFDC files in non-interactive mode.
-u user_name - Specifies the user name for the FTP or SCP server. This operand is
optional; if omitted, anonymous FTP is used.
-p password - Specifies the password for the FTP or SCP server. This operand is
optional with FTP; if omitted, anonymous FTP is used.
-h host_ip - Specifies the IPv4 or IPv6 address for the remote server.
-d remote_dir- Specifies the remote directory to which the file is to be transferred.
When saving to a USB device, the predefined /support directory must be used.
-R Removes all core files on the CP and BP. This option cannot be used with any other
options.
-l protocol - Specifies the transfer protocol. Valid values are FTP or SCP.
-U Saves support data to an attached USB device. When using this option, a target
directory must be specified with the -d option.
-t (timeout_multiplier) Extends predefined SupportSave timeout values by the value
of the timeout multiplier. Use this option to repeat the supportSave operation
when supportSave completion indicates that one or more modules timed out
during the process. For example, -t 2 doubles the timeout values for each of the
SupportSave modules. Valid multiplier values are 2 to 5. The default is 1.
Data Gathering
3 11
Revision 0213
SAN-TS 300
To display the current parameters:
SW1:FID128:admin> supportftp -S
Host IP Addr: 10.255.252.50
User name: dev
Remote Dir: supportsave
Auto Upload protocol: ftp
Auto-FTP: Off

supportftp Usage:
-S
-s
[-h hostname or IP]
[-u username]
[-p password]
[-d remotedirectory]
[-l protocol]
-R | -t hours |-e | -d

Note: Passwords
are NOT displayed
Data Gathering
3 12
Revision 0213
SAN-TS 300
SW1:FID128:admin> supportsave -c

This command collects RASLOG, TRACE, supportShow, core file, FFDC data and
then transfer them to a FTP/SCP/SFTP server or a USB device. This
operation can take several minutes.
NOTE: supportSave will transfer existing trace dump file first, then
automatically generate and transfer latest one. There will be two trace
dump files transferred after this command.
OK to proceed? (yes, y, no, n): [no] y
Saving support information for switch:SW6510, module:RAS...
...................................
Saving support information for switch:SW6510, module:CTRACE_OLD...
Saving support information for switch:SW6510, module:CTRACE_NEW...
Saving support information for switch:SW6510, module:FABRIC.....
Saving support information for switch:SW6510, module:DIAG.....
Saving support information for switch:SW6510, module:RTE...
Saving support information for switch:SW6510, module:IF_TREE...
Saving support information for switch:SW6510, module:ISCSID_DBG...
Saving support information for switch:SW6510, module:AGDUMP...
Saving support information for switch:SW6510, module:AGWWNS...
Saving support information for switch:SW6510, module:AGWWN_CFG...
Saving support information for switch:SW6510, module:VPWWN_CFG.......
Saving support information for switch:SW6510, module:SSHOW_PLOG....

Output continued next slide

Data Gathering
3 13
Revision 0213
SAN-TS 300
The supportsave capture continued:
Saving support information for switch:SW6510, module:SSHOW_OS...
..................................
Saving support information for switch:SW6510, module:SSHOW_EX...
..
Saving support information for switch:SW6510, module:SSHOW_FABRIC...
.......................................
Saving support information for switch:SW6510, module:SSHOW_SERVICE...
..........
Saving support information for switch:SW6510, module:SSHOW_SEC...
................
Saving support information for switch:SW6510, module:SSHOW_NET...
...................
Saving support information for switch:SW6510, module:SSHOW_SYS...
...........................................
Saving support information for switch:SW6510, module:SSHOW_FICON...
...........
Saving support information for switch:SW6510, module:SSHOW_ISWITCH...
.
Saving support information for switch:SW6510, module:SSHOW_ISCSI...
Saving support information for switch:SW6510, module:SSHOW_ASICDB...
.........
Saving support information for switch:SW6510, module:SSHOW_AG...
Saving support information for switch:SW6510, module:SSHOW_APM...
Saving support information for switch:SW6510, module:SSHOW_CRYP...
Saving support information for switch:SW6510, module:SSHOW_FCIP...
Saving support information for switch:SW6510, module:SSHOW_PORT...
Saving support information for switch:SW6510, module:SSHOW_DCEHSL...
Saving support information for switch:SW6510, module:CEEDEBUG...
Saving support information for switch:SW6510, module:CEETECHSUPPORT...
Saving support information for switch:SW6510, module:FCOESUPPORT...
Saving support information for switch:SW6510, module:C2REGDUMP...
Saving support information for switch:SW6510, module:PBREGDUMP...
Saving support information for switch:SW6510, module:BLSREGDUMP...
Saving support information for switch:SW6510, module:AVREGDUMP...
Saving support information for switch:SW6510, module:CRYP...
................
Saving support information for switch:SW6510, module:FCIP...
Saving support information for switch:SW6510, module:VFABRIC...
.....
Saving support information for switch:SW6510, module:MAPS...
Saving support information for switch:SW6510, module:FABRIC_WATCH...
Saving support information for switch:SW6510, module:DM_FTR_FFDC...
Saving support information for switch:SW6510, module:PSDUMP...
Saving support information for switch:SW6510, module:CORE_FFDC...
No core or FFDC data files found!
Saving support information for switch:SW6510, module:ENC_LOGGER...
Saving support information for switch:SW6510, module:AN_DEBUG...
Saving support information for switch:SW6510, module:MP_LOG...
Saving support information for switch:SW6510, module:RAS_POST...

SupportSave completed.
Data Gathering
3 14
Revision 0213
SAN-TS 300
Data Gathering
3 15
Revision 0213
SAN-TS 300
Data Gathering
3 16
Revision 0213
SAN-TS 300
Data Gathering
3 17
Revision 0213
SAN-TS 300
Supportshow operands:
Slot On bladed systems only, specifies a slot number, followed by a slash (/).
port1[-por2] Specifies a port or a range of ports for which to display
supportShow information. This operand is optional; if omitted, the command
displays information for all ports.
Lines Specifies the number of lines for the portLogDump output. This
parameter is valid only with the slot/port parameters.
Data Gathering
3 18
Revision 0213
SAN-TS 300
Output generated by this command may vary by switch configuration, platform and
Fabric OS level.
Some of the more common logs are (Note: this does not cover every command in every
log, just the more common commands, also many of these commands can be found in
multiple files. The commands in bold are most commonly used for troubleshooting.:
SSHOW_EX (exception): Contains errdump, pdshow
SSHOW_OS: Contains Linux OS level commands
SSHOW_PLOG: Contains the portlogdump
SSHOW_FABRIC: fabricshow, islshow, lfcfg --showall cfg; lfcfg
--showall -lisl v, lfmlog dump, trunkshow, fabriclog show,
fabstatsshow, topologyshow, cfgshow, portzoneshow,
portcamshow, cfgsize, cfgshow, defzone -show, zone -show,
porttrunkarea -show all
SSHOW_NET: Contains network commands: ifconfig, route

Data Gathering
3 19
Revision 0213
SAN-TS 300
SSHOW_SEC: Contains security commands: secmodeshow, fddCfg showall,
secpolicydump, secstatsshow, fipscfg showall, aaaconfig

SSHOW_SERVICE: nsshow r, nsallshow, nszonemember n,
nscamshow t, nbrstateshow

SSHOW_FICON: Contains FICON commands: ficonshow, ficoncupshow,
ficucmd

SSHOW_SYS: Contains system commands: supportshowcfgshow, myid,
firmwareshow v, firmwareshow -history,
firmwaredownloadstatus, history, switchstatusshow,
switchshow, tempshow, sensorshow, psshow, fanshow,
licenseshow, portcfgshow, sfpshow all, porterrshow,
fwportdetailshow, slotshow, slotshow, chassisshow,
switchstatuspolicyshow, historyshow, portswapshow, hadump,
configshow -all

SSHOW_ISCSI: Contains iSCSI commands: isciscfg, iscisiportcfg,
iscsisessioncfg, iscsitargetname

SSHOW_ISWITCH: Contains FCR commands: portcfgexport,
portcfgvexport, lsanzoneshow, fcrproxydevshow,
fcrproxyconfig, fcrxlateconfig, fcrphydevshow, fcrrouteshow,
fcrfabricshow fcrresourceshow, fcrrouterportcost,
fcrlsanmatrix, fcrlsan, fcrdbgportshow, fcrdbgrouteshow

SSHOW_AG: Contains Access Gateway commands: ag, agshow

SSHOW_APM: Contains Advance Performance Monitor commands

SSHOW_ASICDB: Contains Engineering level commands containing information about
the ASICs

SSHOW_CRYP: Contains encryption level command outputs: cryptocfg
groupcfg, cryptocfg -groupmember all, cryptocfg
hacluster, cryptocfg container, cryptocfg rekey

SSHOW_FCIP: Contains commands for troubleshooting FCIP issues: portshow
fciptounnel, portshow ipif, portshow iproute, portstatsshow,
switchshow
Data Gathering
3 20
Revision 0213
SAN-TS 300
Data Gathering
3 21
Revision 0213
SAN-TS 300
Footnote 1: RAS Reliability, Availability, and Serviceability
Footnote 2: Forward RAS Log and Console log entries to a syslogd daemon on a host
computer (syslogdipadd)
Especially important on dual-CP systems as host computer logs maintain a single,
sequentially ordered, merged file for both CPs
Footnote 3: Use errdump/show -r to display error messages in reverse order: most-
recent to least-recent
Clear all internal and external messages from the error log with Admin level
errclear command

Data Gathering
3 22
Revision 0213
SAN-TS 300
Footnote 1: Message levels:
Critical level messages indicate that the software has detected serious problems
that will cause a partial or complete failure of a subsystem if not corrected
immediately; for example, a power supply failure or rise in temperature must receive
immediate attention.
Errorlevel messages represent an error condition that does not impact overall
system functionality significantly. For example, error-level messages might indicate
time-outs on certain operations, failures of certain operations after retries, invalid
parameters, or failure to perform a requested operation.
Warning level messages highlight a current operating condition that should be
checked or it might lead to a failure in the future. For example, a power supply
failure in a redundant system relays a warning that the system is no longer
operating in redundant mode unless the failed power supply is replaced or fixed.
Info level messages report the current non-error status of the system components:
for example, detecting online and offline status of a fabric port.
Data Gathering
3 23
Revision 0213
SAN-TS 300
Footnote 1: You can easily use this event code to search the Fabric OS Message
Reference Manual for more information.
Date and Time Stamp: The system time (UTC) when the message was generated on the
switch. The RASLog subsystem supports an internationalized time stamp format based
on the LOCAL setting.
Message Module and Message Number: The message module and number. These
values uniquely identify each message in the Fabric OS and reference the cause and
actions recommended in this manual. Note that not all message numbers are used;
there can be gaps in the numeric message sequence.
Sequence Number: The error message position in the log. When a new message is
added to the log, this number is incremented by 1. When this message reaches the last
position in the error log and becomes the oldest message in the log, it is deleted when a
new message is added. The message sequence number starts at 1 after a
firmwaredownload and will increase up to a value of 2,147,483,647 (0x7fffffff). The
sequence number will continue to increase beyond the storage limit of 1024 messages.
The sequence number can be reset to 1 using the errClear command. The sequence
number is persistent across power cycles and switch reboots.
Severity Level: The severity of the error:
1 = Critical
2 = Error
3 = Warning
4 = Info
Data Gathering
3 24
Revision 0213
SAN-TS 300
Data Gathering
3 25
Revision 0213
SAN-TS 300
Event class Description:
Zone: You can audit zone event configuration changes, but not the actual values that
were changed. For example, you may receive a message that states Zone configuration
has changed, but the message does not display the actual values that were changed.
Security: You can audit any user-initiated security event for all management interfaces.
For events that have an impact on the entire fabric, an audit is only generated for the
switch from which the event was initiated.
Configuration: You can audit configuration downloads of existing SNMP configuration
parameters. Configuration uploads are not audited.
Firmware: You can audit configuration downloads of existing SNMP configuration
parameters. Configuration uploads are not audited.
Fabric :You can audit Administration Domain-related changes.
Fabric Watch: You can audit Fabric Watch (FW) related changes.
Logical Switch: You can audit Virtual Fabric (Logical Switch) related changes.
Data Gathering
3 26
Revision 0213
SAN-TS 300
Data Gathering
3 27
Revision 0213
SAN-TS 300
Footnote 1: Cannot chain command, example: (The following example does not work.)
SW1:admin> auditcfg class 1,3,5 enable
Once audit logging is enabled classes can be change with out first disabling logging.

Footnote 2:
SW1:admin> auditcfg -show
Audit filter is enabled.
1-ZONE
2-SECURITY
3-CONFIGURATION
4-FIRMWARE
5-FABRIC
6-FW
7-LS
Severity level: INFO

Note: See next slide for information on the Severity levels and how to change them.

Data Gathering
3 28
Revision 0213
SAN-TS 300
There are four severity levels: INFO, WARNING, ERROR, CRITICAL To change severity
level (which by default is INFO which means all four levels will be included in the log)
run command: auditcfg -- severity

Example: To change the severity from info to warning (which would include error and
critical) run command:

SW1:admin> auditcfg --severity warning
SW1:admin> auditcfg --show
Audit filter is enabled.
1-ZONE
2-SECURITY
3-CONFIGURATION
4-FIRMWARE
5-FABRIC
6-FW
7-LS
Severity level: WARNING

Auditcfg command usage:
--show
--disable
--enable
--severity <INFO, WARNING, ERROR, CRITICAL>
--class <1,2,3,4,5,6,7>
1-ZONE, 2-SECURITY, 3-CONFIGURATION, 4-FIRMWARE,
5-FABRIC, 6-FW, 7-LS
Data Gathering
3 29
Revision 0213
SAN-TS 300
The generic message format seen on the syslog server:
AUDIT, <timestamp>, [<Event ID>], <Severity>, <Event Class>,
<User ID>/<Role>/<IP address>/<Interface>/<app name>. <Admin
Domain>/<Switch name>/Fabric ID (FID)#, <Reserved field for
future expansion>, <Event-specific information>

Note: Audit messages are also logged to the syslog server if configured.
Data Gathering
3 30
Revision 0213
SAN-TS 300
Data Gathering
3 31
Revision 0213
SAN-TS 300
AUDIT Messages (cont.)

Director considerations
Audit messages are generated independently by both the Active and Standby CPs.
Both CPs need an external management port connection. Both CPs need network
connectivity. A crossover cable attached to one CP card will prevent system logging
from the other CP card.

Syslog messages will always be delivered to the host syslog server from the Active CP.
The Audit configuration is propagated to the Standby CP during a CP card failover.
Syslog Server Considerations
To successfully deliver Audit messages to a syslog server, verify that:
External syslogd server is functional and the syslog facility is operational
IP network is functional
There will be some limitation for syslog on the frequency of events that can stream off
the switch. If too many events are generated by the switch, syslog will become a
bottleneck and audit events will be dropped by the software to prevent any issues with
the switch.
The Audit infrastructure is reliant on the event generating applications to provide the
audit-specific information. This means that if an application does not have the ability
to figure out the username/IP address/interface that an event came in, the Audit
infrastructure will not be able to transport that data and it will not be seen by the user.
i.e. events not generated by a user.
Audit messages are viewed from the console and, if syslog functionality is configured,
from the syslog server. Messages will continue to stream into the server. Methods to
sort, store, and clear these messages needs to be configured on the server. There is
no limit to the number of messages that a switch will send.

Data Gathering
3 32
Revision 0213
SAN-TS 300
Result: Audit messages are streamed chronologically to the configured syslog servers.

Data Gathering
3 33
Revision 0213
SAN-TS 300
Data Gathering
3 34
Revision 0213
SAN-TS 300
Each event that triggers an FFDC capture may result in more than one FFDC file being
created. The FFDC files are stored on the switch and transferred by supportsave;
once transferred they are automatically deleted from the switch.

Footnote 1: The specific events that trigger an FFDC capture are pre-selected by
Brocade engineering and cannot be changed by the user.

Footnote 2: When an FFDC capture occurs, the RAS Log error message includes FFDC is
the AUDIT flag field. Please check the latest revision of the Fabric OS Message
Reference manual or release notes for the latest details on which messages generate
an FFDC message.

Data Gathering
3 35
Revision 0213
SAN-TS 300
FFDC is as important as core files. It is an ERROR indicator, not an information or
warning indicator.
FFDC data capture indicators include:
1. RAS-LOG indicator
RAS-1001 INFO First failure data was captured
2. Console message: every hour

Data Gathering
3 36
Revision 0213
SAN-TS 300
When an FFDC defined event triggers a core dump then FFDC data is captured along
with panic data. The FFDC data is in readable format, the panic data is not.

Data Gathering
3 37
Revision 0213
SAN-TS 300
The pdshow is captured as part of the supportsave *.SSHOW_EX (exception group)
output. The pdshow command displays data in a panic dump file. The command has
one optional argument: the name of a specific panic dump file. If no file is specified,
output is displayed from the most recent panic dump file. In the example above, the
pdshow command output indicates that there were not any panic dump files were
available. Panic dumps are text files, core file contents are encrypted

Panic dumps and core files remain on the switch after the supportsave command is run.
Panic Dumps are caused by a reboot reason = panic. These occur when Linux
Kernel panics cause the Fabric OS to panic.
Core Files are Linux standard core files.
Footnote 1: It may take up to 60 seconds to detect the daemon failure. The interval
between daemon restart attempts is short seconds. If the daemon is successfully
restarted but fails again 10 minutes later, then 3 more daemon restart attempts will be
made. There is no permanent death; the 3 restart attempts every 15 minutes will
continue indefinitely.

Data Gathering
3 38
Revision 0213
SAN-TS 300
The trace dump file is meant to be like an airplane black-box recorder, tracking a brief
window of current values. This information can be an important aid to debugging system
crashes by provided an historical record of switch activity and behavior.
Only one trace dump file is retained on a switch at any time. If another trace dump is
triggered, the previous trace dump file is deleted.

Data Gathering
3 39
Revision 0213
SAN-TS 300
Footnote 1: Looking at the errdump output shows the creation of the dump file:
2012/05/24-13:27:36, [TRCE-1001], 208, CHASSIS, WARNING, SW1,
Trace dump available ! (reason: MANUAL)

You will also see one of the following two messaging depending of the auto FTP setting
(enable or disabled):
2012/05/24-12:49:18, [TRCE-1004], 203, CHASSIS, WARNING, SW1,
Trace dump was not transferred because trace auto-FTP
disabled.
Or
2012/05/24-13:27:47, [TRCE-1002], 209, CHASSIS, INFO, SW1,
Trace dump automatically transferred to address '
10.255.252.50 '.

Use the n option and include the s (slot) option on director switches to generate a
trace dump for a specific slot in the chassis
See Brocade Fabric OS Command Reference manual for more information on the
tracedump command.

Data Gathering
3 40
Revision 0213
SAN-TS 300
Footnote 1: The parameters set by the supportftp command are used by both the
supportsave and tracedump commands.

For more information on supportftp parameters see next page notes slide.

Footnote 2: The supportsave uses a different file name, its called
*.CTRACE_NEW.dmp.gz and *.CTRACE_OLD.dmp.gz, this uploads the last two trace files.
Data Gathering
3 41
Revision 0213
SAN-TS 300
Use the supportftp command to set, clear, or display support FTP parameters. This
command has the following optional arguments:
s: Set the FTP parameters. The following operands can be optionally specified on
the command line. If the -s option is specified without further operands, the
command interactively prompts for these parameters.

h <host>: Specifies the FTP host. Provide an IP address or a server name. IPv4
and IPv6 addresses are supported. To specify the host by name, a DNS
entry must exist for the server.
u <username>: Specifies the FTP account user name. The user name must be
less than 48 characters long.
p <password>: Specifies the FTP account password. The password must be
less than 48 characters long. When using anonymous FTP, a password is not
required.
d <remotedirectory>: Specifies the remote directory where the trace dump
files are stored. The directory name must be less than 48 characters long.
Specifying the root directory as the remote directory (/) is not allowed.
l protocol: Specifies the transfer protocol. Valid values are file transfer protocol
(FTP), secure copy protocol (SCP), or secure FTP (SFTP).

t <hours>: Specifies the time interval, in units of hours, at which the FTP server
connectivity is checked.

R: Clears all FTP parameters.

e: Enables auto file transfer. Trace dump files are automatically transferred to a
designated FTP server. The server parameters must be set before you can
enable auto file transfer.

d: Disables auto file transfer

In Fabric OS, you can administer limited parts of the trace dump functionality through
the Trace tab in the Switch Admin dialog in Web Tools.
Data Gathering
3 42
Revision 0213
SAN-TS 300
To access the Web Tools view on this slide click Switch Admin and then Show
Advanced Mode:
Data Gathering
3 43
Revision 0213
SAN-TS 300
Data Gathering
3 44
Revision 0213
SAN-TS 300
For more information on Brocade Network advisor see WBT BNA 200 Brocade Network
Advisor Training course.

Data Gathering
3 45
Revision 0213
SAN-TS 300
Footnote 1: The switch and the host (containing the Brocade HBA/Fabric Adapter) must
be discovered by Brocade Network Advisor.
Data Gathering
3 46
Revision 0213
SAN-TS 300
SAN-TS 300 Data Gathering
3 47
Revision 0213
Footnote 1: The fabric and the hosts must be discover by Brocade Network Advisor. To
get to the Technical SupportSave window click on: Monitor Technical Support
Product/Host SupportSave
Footnote 2: In would be the name of the fabric, in this example the name of the fabric is
Fabric.

Data Gathering
3 48
Revision 0213
SAN-TS 300
3 49
Revision 0213
3 50
Revision 0213
3 51
Revision 0213
3 52
Revision 0213
Footnote 1: To Generate reports select SAN and click on Reports in the menu. Select
Event Customer Reports, Generate or View:

Fabric Summary Report: List information for discovered fabrics. Creates a separate
report for each fabric. Includes a summary on: (See example next slide)
Fabric information
Switches
Device information
ISLs and trunks
Port Ports Report: Lists discovered ports including used and unused ports. Port data for
each fabric is divided into two parts: (See example in two more slides)
Director and switch utilization
Individual port details

Data Gathering
3 53
Revision 0213
SAN-TS 300
Example Fabric Summary report:

Data Gathering
3 54
Revision 0213
SAN-TS 300
Example Fabric Port report:
Data Gathering
3 55
Revision 0213
SAN-TS 300
Note: Requires Advanced Performance Monitor license on all switches.

Can display Rx and Tx Utilization or Mbps as well as the following error counters:
CRC Errors
Signal Losses
Sync Losses
Link Failures
Sequence Errors
Invalid Transmissions
Rx Link Resets
Tx Link Resets

Data Gathering
3 56
Revision 0213
SAN-TS 300
Footnote1: The freeze option freezes the log from on the fly updates. New events will
still be stored in the database but not the display will not be updated until the freeze is
unchecked.

Footnote 2: Event message can be user defined: Example the user can define pseudo
events (more on this later in this presentation) and assign a severity level to them. So a
user can assign an Emergency level to a pseudo event. This could be useful for
troubleshooting. To create a pseudo event: Monitor Event Processing Pseudo
Events

Data Gathering
3 57
Revision 0213
SAN-TS 300
3 58
Revision 0213
Data Gathering
3 59
Revision 0213
SAN-TS 300
HOW THE SAN HEALTH PROCESS WORKS
The SAN Health family gives you powerful tools that help you focus on optimizing your
SAN rather than manually tracking its components. A wide variety of useful features
make it easier for you to collect data, identify potential issues, and check your results
over time. As a result, you can greatly increase your productivity while enhancing your
SAN operations.
Time-saving Reports
To provide a comprehensive report about your SAN environment, the free SAN Health
Diagnostics Capture utility utilizes a data capture application and a back-end report
processing engine. After it captures switch diagnostic data, the utility automatically
generates a Visio topology diagram and a detailed "snapshot" report. This report
contains summary information as well as specific details about SAN fabrics, switches,
and individual ports. Other useful items include alerts, historical performance graphs,
and recommended best practices.
Enhanced Change Tracking
Because it provides a point-in-time snapshot of your SAN, SAN Health Diagnostics
Capture can be invaluable to your change-tracking process. For instance, you can use it
to track traffic pattern changes in weekly or monthly increments. And with a built-in
scheduler, you can run it after primary business hours for added safety and
convenience.

Data Gathering
3 60
Revision 0213
SAN-TS 300
Download the SAN Health Diagnostics Capture, and save to your hard drive.
SAN Health Diagnostics Capture minimum system requirements:
Intel Pentium processor 133 MHz or higher
Microsoft Windows 95 or higher
64 MB RAM / 10 MB available hard disk space

Data Gathering
3 61
Revision 0213
SAN-TS 300
Data Gathering
3 62
Revision 0213
SAN-TS 300
The last screen of the process gives you an option to send the diagnostic data to the
report generation queue via HTTPS or via email attachment to SHUpload@brocade.com
Either way you will get an email confirmation letting you know that the report was
received and a second email when the report is ready.

Data Gathering
3 63
Revision 0213
SAN-TS 300
3 64
Revision 0213
Values that merit attention are highlighted in red, orange and blue If a value is
highlighted in one of these colors, it is recommended that action be taken to assess the
impact to your SAN

Data Gathering
3 65
Revision 0213
SAN-TS 300
3 66
Revision 0213
Data Gathering
3 67
Revision 0213
SAN-TS 300
Data Gathering
3 68
Revision 0213
SAN-TS 300
Footnote 1: HCM under tools supportsave, however this is for the HCM application
only and does not capture information about the HBA. This supportsave is useful for
troubleshooting issues with the HCM application and management of an HBA. But is not
useful when troubleshooting issues with the HBA.

Data Gathering
3 69
Revision 0213
SAN-TS 300
Data Gathering
3 70
Revision 0213
SAN-TS 300
Data Gathering
3 71
Revision 0213
SAN-TS 300
HBA supportsave example:
bfa_ss.txt is the supportshow output
Data Gathering
3 72
Revision 0213
SAN-TS 300
Footnote 1: CNA Converged Network Adapter

Master Log: shows events such as when devices go online/offline.
Data Gathering
3 73
Revision 0213
SAN-TS 300
Data Gathering
3 74
Revision 0213
SAN-TS 300
Data Gathering
3 75
Revision 0213
SAN-TS 300
Data Gathering
3 76
Revision 0213
SAN-TS 300
Footnote 1: Some utilities require you to configure the utility for capturing prior to
opening up a session. Check with your utility vendor for instructions.

Steps to capture command output:
1. Connect to the switch through a Telnet or SSH utility.
2. Log in using an account assigned to the admin role.
3. Set the Telnet or SSH utility to capture output from the screen.
Some Telnet or SSH utilities require this step to be performed prior to opening up a
session. Check with your Telnet or SSH utility vendor for instructions.

Data Gathering
3 77
Revision 0213
SAN-TS 300
Best practice: Consider connecting a terminal server with network AND modem
capability for serial console access to switch. If you lose network access, you can
still dial in assuming that the terminal server has this capability. The serial console
is used to access a switch to configure network parameters, monitor switch console
messages, and sometimes to perform password recovery procedures. Not all
password recovery procedures require serial access.
Console messages that pop-up during CLI login sessions are not displayed in
errshow/Dump (log error message) outputs unless they contain a severity level.
Console messages are messages that go to the serial port. In Linux, messages
directed to standard error are mirrored on the console. Console messages that
contain severity levels will be displayed in the error log. Examples of console
messages that do not include severity codes include CP sync messages. These
CP sync messages let the console know about events that occur in the CP fail over
process. Console messages can be configured to go to syslog servers.
Management host needs:
An available serial port
A terminal emulation program configured with
the correct serial port parameters

Check the hardware reference manual for required
cable
The serial ports parameters are fixed at
9600 baud, 8 data bits, and no
parity, with flow control set to None
Brocade switch
Serial Console Server
Serial cable
The 1
st
point of contact with Brocade
switches is often the serial console
What can go wrong?
Consider connecting a terminal
server with network AND modem
capability for serial console
access to switch
Capture serial console output
during problem determination
processes, especially when a
switch reboot is required
Data Gathering
3 78
Revision 0213
SAN-TS 300
Data Gathering
3 79
Revision 0213
SAN-TS 300
Events can also be filtered by using the Reports Event Custom Reports utility.

Following columns will be displayed by default in Master Log :
Severity
Acknowledged
Area (SAN/IP/SAN + IP)
Source Name
Source Address
Origin

Category
Description
Last Event Server Time
Count
Module Name
Message ID

Data Gathering
3 80
Revision 0213
SAN-TS 300
Data Gathering
3 81
Revision 0213
SAN-TS 300
Data Gathering
3 82
Revision 0213
SAN-TS 300
Data Gathering
3 83
Revision 0213
SAN-TS 300
Data Gathering
3 84
Revision 0213
SAN-TS 300
Device Connectivity
Revision 0213
SAN-TS 300
4 1
Device Connectivity
Revision 0213
SAN-TS 300
4 2
Device Connectivity
Revision 0213
SAN-TS 300
4 3
Footnote 1: Is there light from the host or device?
A powered off or failed device may not provide light
Without light there will never be a login
Footnote 2: Does the switch port speed configuration match the attached device
speed configuration?
Devices and switch ports typically auto-negotiate
Verify that the switch port is not locked to a speed the device cannot handle.
Example if the switch is hard set for 16 Gbps and the HBA is an 8 Gbps the device
can not log into the fabric.
Footnote 3: Devices login to a fabric using Fabric Login (FLOGI), has this occurred
?

The end device will FLOGI to be assigned a 24-bit address
Until then, it has no fabric port ID (PID) with which to initiate communication in
the fabric
In most cases devices login using point-to-point but even if the device logs in as
loop, it should still proceed to the FLOGI stage to get a Public Loop Address (24-bit
address)
The device FLOGI response from switch will not have a 24-bit address if the device
is not part of an enabled DCC policy. In addition to DCC policies Active Gateway
Advanced Device Security can also prevent a device from logging into the fabric.

Device Connectivity
Revision 0213
SAN-TS 300
4 4
Troubleshooting is never an exact methodology. The path you take depends upon the
results of the command you typed in. It may depend on visual indicators within the
switch, the host, or the target.
No two people troubleshoot the same way, and this is only a summary of commands
available and symptoms to be aware of.
Think of a switchshow as a binary action you may be able to eliminate the systems
side of the picture if something looks wrong with the storage port. With the output of
your switchshow command, you may eliminate half of the configuration as suspect.
Try not to make it too complicated by keying in on one specific component until some
data points toward that component. Dont assume the information you have been given
is correct, always validate the information.

Footnote 1: You can also use command portlogshow which filters the portlogdump
for one specific port.

Device Connectivity
Revision 0213
SAN-TS 300
4 5
Device Connectivity
Revision 0213
SAN-TS 300
4 6
Note: Loop devices are not supported on Brocade 16 Gbps switches
Device Connectivity
Revision 0213
SAN-TS 300
4 7
Device Connectivity
Revision 0213
SAN-TS 300
4 8
Verify you are receiving light from the end device. Does the switch see light from the
device?
A disconnected or bad cable may be the problem. The HBA in the host may have failed.
OS configuration file parameters, driver parameters, and HBA firmware parameters
could also be a reason that the switch is not receiving light from the end device.
Start with the switchshow command to get an overall view of the ports.
For port state, the following would be related to Light:
No_Card - no interface card present
No_Module - no module (SFP or other) present
Mod_Val - module validation in process
Mod_Inv - invalid module
No_Light - the module is not receiving light
Use portflagsshow to verify whether Light had previously been seen.
Your SFP within that port on the switch could be faulty. Use the sfpshow <port>
command to verify that the SFP is functioning properly.
Footnote 1: D_Port is an advanced diagnostics used to diagnose issues with: SFPs,
cables, Condor 3 ASICs, and Connections. Does require the switch port ASIC to be
Condor3. D_Port test is cover in more detail in switch to switch connectivity module.

Device Connectivity
Revision 0213
SAN-TS 300
4 9
Make sure we are receiving light from the end device. Does the switch see light from the
device?

A disconnected or bad cable may be the problem. The HBA in the host may have failed.
OS configuration file parameters, driver parameters, and HBA firmware parameters
could also be a reason that the switch is not receiving light from the end device.

Start with the switchshow command to get an overall view of the ports.
For port state, the following would be related to Light:
No_Module - no module (SFP or other) present
Mod_Val - module validation in process
Mod_Inv - invalid module

Your SFP within that port on the switch could be faulty. Use the sfpshow <port>
command to verify that the SFP is functioning properly.

Device Connectivity
Revision 0213
SAN-TS 300
4 10
The full sfpshow output is in the notes on the following page.
Device Connectivity
Revision 0213
SAN-TS 300
4 11
R8-ST01-DCX-8510-4:admin> sfpshow 1/10
Identifier: 3 SFP
Connector: 7 LC
Transceiver: 540c404000000000 2,4,8_Gbps M5,M6 sw Short_dist
Encoding: 1 8B10B
Baud Rate: 85 (units 100 megabaud)
Length 9u: 0 (units km)
Length 9u: 0 (units 100 meters)
Length 50u (OM2): 5 (units 10 meters)
Length 50u (OM3): 0 (units 10 meters)
Length 62.5u:2 (units 10 meters)
Length Cu: 0 (units 1 meter)
Vendor Name: BROCADE
Vendor OUI: 00:05:1e
Vendor PN: 57-1000012-01
Vendor Rev: A
Wavelength: 850 (units nm)
Options: 003a Loss_of_Sig,Tx_Fault,Tx_Disable
BR Max: 0
BR Min: 0
Serial No: UAF108520000LA3
Date Code: 081226
DD Type: 0x68
Enh Options: 0xfa
Status/Ctrl: 0xb0
Alarm flags[0,1] = 0x0, 0x0
Warn Flags[0,1] = 0x0, 0x0
Alarm Warn
low high low high
Temperature: 27 Centigrade -10 90 -5 85
Current: 6.718 mAmps 1.000 17.000 2.000 14.000
Voltage: 3294.3 mVolts 2900.0 3700.0 3000.0 3600.0
RX Power: -40.0 dBm (0.1 uW) 10.0 uW 1258.9 uW 15.8 uW 1000.0 uW
TX Power: -3.4 dBm (459.7 uW) 125.9 uW 631.0 uW 158.5 uW 562.3 uW
Cable specs
shortwave length SFP
Alarm
thresholds and
current sensor
readings
Device Connectivity
Revision 0213
SAN-TS 300
4 12
Footnote 1: Remember to check both ends of the link for light/signal. One end may be showing no sync because it is
receiving light but not transmitting light.
Device Connectivity
Revision 0213
SAN-TS 300
4 13
Footnote 1: These errors are always detected on the ingress port.

Footnote 2: There are two types of encoding errors:
enc_in:
Increments when 8b/10b encoding errors are detected within a frame
enc_in errors are always detected on the ingress port
enc_out:
8b/10b encoding errors not associated with frames (IDLE, R_RDY, and various other
primitives)
This counter increments during speed negotiation prior to login
Locking a port to a speed supported by the end device can be used to isolate
issues
Possible bad media (SFP, cable, patch panel)
Can cause performance problems due to corruption of R_RDY primitives leading to
buffer credit starvation
This will be covered in greater detail in the performance module

Device Connectivity
Revision 0213
SAN-TS 300
4 14
Device Connectivity
Revision 0213
SAN-TS 300
4 15
The lines of the display show:
frames tx/rx Counters representing the number of frames transmitted and received.
enc_in 8bit/10bit encoding errors inside frame. Words inside of frames are encoded,
if this encoding is corrupted or an error is detected enc_in is generated.
crc_err counter are frames with CRC errors. If this counter goes up, then the physical
path should be inspected. Check the cables to and from the switch, patch panel, and
other devices. Check the SFP by swapping it with a known good working SFP. If you see
this issue on an 8 Gbps blade, use the portCfgfillword command to reduce EMI.
Suggested actions would be to replace the cable or SFP, move cable to another port, or
run porttest or portdporttest.

crc g_eof The crc_g_eof counter are frames with CRC errors and a good EOF. The
first port detecting a CRC error marks the frame with a bad EOF and passes the frame
on to its destination. Subsequent ports in the path also detect the CRC error and the
crc_err counter increments on these ports. However, since the first port marked the
frame with a bad EOF, the good EOF counter on the subsequent ports does not
increment. The marginal link associated with the port with an increasing good EOF
counter is the marginal link and the source of the errors.
too_short The too_short counter is incremented whenever a frame, bounded by an
SOF and EOF, is received and the number of words between the SOF and EOF is less
than 7 words (6 word header plus 1 word CRC). This would be 38 bytes including the
Device Connectivity
Revision 0213
SAN-TS 300
4 16
SOF and EOF. This could be caused by the transmitter, or an unreliable link.
SAN-TS 300 Device Connectivity
4 16
Revision 0213

too_long Fibre Channel frames are 2148 byes maximum. If an eof is corrupted or
data generation is incorrect a too_long error is generated.

bad_eof After a loss of synchronization error continuous mode alignment allows the
receiver to reestablish word alignment at any point in the incoming bit stream while the
receiver is operational. Such realignment is likely (but not guaranteed) to result in code
violations and subsequent loss of synchronization. Under certain conditions, it may be
possible to realign an incoming bit stream without loss of synchronization. If such a
realignment occurs within a received frame, detection of the resulting error condition is
dependent upon higher-level function (e.g., invalid CRC, missing EOF Delimiter).
enc_out 8bit/10bit encoding errors occurred in words (ordered sets) outside the
Fibre Channel frame and usually indicating a bad primitive. Words outside of frames
are encoded, if this encoding is corrupted or an error is detected enc_out is
generated. This is a sign of a hardware problem, take snapshots of the port errors by
using the porterrshow command in increments of 5 to 10 minutes. If you notice the
crc_err counter go up, you have a bad or damaged cable, or a bad or damaged
device in the path. Suggested actions would be to replace the cable or SFP, move cable
to another port, or run porttest or portdporttest to verify. NOTE: ICLs will see
enc_out errors when ports on other side of the link are disabled, this is normal and
OK.
Disc c3 Discard class 3 errors could be generated by a switch when devices send
frames without performing a FLOGI first or send frames to an invalid destination. It also
is an indication of a possible performance problem, when a switch port cant send a
frame due to congestion and must discard the frame when the hold time expires. More
information on this in the performance module of this course.
Link fail If a port remains in the LR Receive State for a period of time greater than a
timeout period (R_T_TOV), a Link Reset Protocol Timeout shall be detected which
results in a Link Failure condition (enter the NOS Transmit State). The link failure also
indicates that loss of signal or loss of sync lasting longer than the R_T_TOV value was
detected while not in the Offline state.
Loss sync Synchronization failures on either bit or transmission word boundaries are
not separately identifiable and cause loss-of synchronization errors.
Loss sig Occurs when a signal is transmitted but none is being received on the same
port.
Frjt If the fabric cannot process a Class 2 frame a F_RJT is returned. The F_RJT
response to a frame indicates that delivery of that frame is being rejected. Rejection
indicates that the frame contents are intact (i.e. no transmission errors) but the frame
cannot be received for some protocol-related reasons, such as non-support of a service
or inconsistent frame header fields.

Device Connectivity
Revision 0213
SAN-TS 300
4 17
Fbsy If the fabric cannot deliver a class 2 frame within E_D_TOV frame will be
discarded and an F_BSY returned. The F_BSY indicates that the frame cant be
delivered, because either the fabric or the destination N_Port is temporarily busy. On
receipt of an F_BSY in response to a frame transmitted, the source N_Port is expected
to attempt Frame retransmission, up to some number of retries. Recovery after retry is
exhausted is dependent on the FC-4 ULP and the Exchange Error Policy.

Port counters can be cleared on an octet or per-port basis with:
Switch1:admin> portstatsclear 4
wait a few minutes
Switch1:admin> portstatsshow 4

For 8 Gbps switches: use the porttest command along with porterrshow to verify
physical near-end components
Switch1:admin> porttest ports 1 iteration 100
For 16 Gbps switches with 16 Gbps SFPs: use portdporttest, this to verify hysical
near-end components. Note: If you have a 16 Gbps with 8 Gpbs SFPs must use
porttest.

portstatsshow command gives better granularity on counters when counters are
high (example: 1.2m vs. 1.3m there is a large gap between these two values).
porterrshow is a good overall command to identify suspect ports.

frames enc crc crc too too bad enc disc link loss loss frjt fbsy
tx rx in err g_eof shrt long eof out c3 fail sync sig
==========================================================================
<truncated output>
5: 3.3g 3.8g 0 0 0 0 0 0 45 1 0 15 30 0 0
6: 3.7g 3.1g 0 2.4k 0 2.4k 2 2.4k 2 0 0 13 26 0 0
7: 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0
8: 26m 23m 0 0 0 0 0 0 180k 2.4k 1 10 17 0 0
9: 30m 40m 3 1 0 1 0 3 3.4k 6.4k 0 14 25 0 0
<truncated output>
Device Connectivity
Revision 0213
SAN-TS 300
4 18
Once you identify suspect ports with porterrshow, use portshow <port> or
portstatsshow <port> to look at actual port counters. Fields within the
portstatsshow output are larger than porterrshow. Look at the enc_out errors.
The difference between 3.8g and 3.9g is larger than the difference between 38 and
39. For 3.8g to increment to 3.9g, 1,000,000 more errors must occur. The exact values
can be seen with portstatsshow <port> or portshow <port>. Alternatively,
you could clear the counters for the port with portstatsclear <port>, and then
continue to monitor.

Footnote 1: It is not uncommon for enc_out values to increment by millions on
E_Ports that auto negotiate at one end and have their speed locked to 2 or 4 Gbit/sec
at the other end. During the speed negotiation process these errors can increment
dramatically. To monitor enc_out values, either establish a baseline or issue a
portstatsclear AFTER speed negotiation has taken place.
Device Connectivity
Revision 0213
SAN-TS 300
4 19
Fiber cable needs to be matched to the SFP in use and cables on both sides of a patch
need to be the same type.
Footnote 1: If the marginal link was caused by Switch3 port 5 the CRC and ENC errors
would only be seen on switch4 port 7 and Switch2 port 12.
crc_err counter are frames with CRC errors. If this counter goes up, then the physical
path should be inspected. Check the cables to and from the switch, patch panel, and
other devices. Check the SFP by swapping it with a known good working SFP. If you see
this issue on an 8 Gbps blade, use the portcfgfillword command to reduce EMI.
Suggested actions would be to replace the cable or SFP, move cable to another port, or
run porttest or portdporttest.

Footnote 2: crc g_eof The crc_g_eof counter are frames with CRC errors and a good
EOF. The first port detecting a CRC error marks the frame with a bad EOF and passes
the frame on to its destination. Subsequent ports in the path also detect the CRC error
and the crc_err counter increments on these ports. However, since the first port marked
the frame with a bad EOF, the good EOF counter on the subsequent ports does not
increment. The marginal link associated with the port with an increasing good EOF
counter is the marginal link and the source of the errors.

Device Connectivity
Revision 0213
SAN-TS 300
4 20
Device Connectivity
Revision 0213
SAN-TS 300
4 21
porttest: Use this command to isolate problems in a single replaceable element and
to trace problems to near-end terminal equipment, far-end terminal equipment, or the
transmission line. This command verifies the functional operation of the switch by
sending frames from a port's transmitter, and looping the frames back through an
external fiber cable into the port's receiver. The test exercises all switch components
from the main board, to the fibre cable, to the media (of the devices and the switch),
and back to the main board.

portdporttest: (Requires the link to be running at 16 Gbps) Use this command to
manually terminate or re-initiate testing on a diagnostic port (D_Port). The port must be
configured as a D_Port and physically connected to a second D_Port on a remote
switch. The D_Port test performs the following diagnostics:
An electrical loopback test (supported only on 16G SFPs capable of electrical
loopback)
An optical loopback test (supported only on 16G SFPs capable of optical loopback)
A link traffic test
A link distance measurement

See Fabric OS command reference manual for more information on these tests.
Device Connectivity
Revision 0213
SAN-TS 300
4 22
Footnote 1: Port speeds are configured using the portcfgspeed command. Syntax
is:
Usage: portCfgSpeed PortNumber Speed_Level
Speed_Level: 0 - Auto Negotiate (Hardware)
1 - 1Gbps
2 - 2Gbps
4 - 4Gbps
8 - 8Gbps
10 - 10Gbps
16 - 16Gbps
ax - Auto Negotiate (Hardware) + retries
s - Auto Negotiate (Software)
Both the sender and receiver attempt to clock bits as they receive them. When they
agree on the frequency of the bits, speed has been negotiated and established. At this
point they can start bit synchronization. If they cannot achieve this synchronization, the
port remains in a No_Sync state. This is part of the Port State Machine T-11 FC-FS
Device Connectivity
Revision 0213
SAN-TS 300
4 23
Standard.
4 23
Revision 0213
The output from portshow or portflagsshow can be used to get a high level
overview of the login process for a port. In addition to login information other port level
information is sometimes shown in the port flags. The flags output for both commands
is the same.
The flags are read from right to left. The possible flags that can be displayed in Fabric
OS are:
PRESENT Port present (card plugged in)
ACTIVE Port is in the active state
VIRTUAL This is a virtual port
E_PORT Port type is an E_Port (ISL port)
T_FPORT F_Port is a trunk port
T_FMASTER F_Port is a trunk master
T_PORT Port is a trunk port
T_MASTER Port is a trunk master
F_PORT Port type is an edge port connecting to fabric capable devices
G_PORT Port type is a Generic port Acts as a transition for non-loop fabric
capable devices
L_/FL_PORT Port type is a Fabric Loop port
U_PORT Port type can be unidentified port
Device Connectivity
Revision 0213
SAN-TS 300
4 24
LE_PORT Port type can be Logical E_Port
EX_PORT Port type can be EX_Port connects to a Fibre Channel router
SEGMENTED Port is segmented (incompatible)
NPIV Port is N_Port Virtualization F_Port
LOGICAL_ONLINE Port is logical online (used by HA)
RRDY_MODE Port has receiver ready (R_RDY) mode
DISABLED Port is disabled
INT_LB - Port is in internal loopback mode
SLW_LB Port is in serial link wrapback (serdes) mode
EXT_LB Port is in external loopback (serdes) mode
CBL_LB Port is in cabled loopback mode
SSC_LB Port is in silk screen loopback mode
FL_INT_LB Port is in FL_Port internal loopback mode
DEB Debounce period started
LOGIN Port is logged in (FLOGI or ELP)
NOELP ELP failed don't try again
LED Enable automatic LED control
NSREG Port has devices to be registered
PROBE Port has devices to be probed
FAULT Port is faulty
FAULT_RETRY Faulty port is being re-inited
ACCEPT FLOGI ACC can be sent if set
BYPASSED -- Bypassed
LOGIHELD Port disabled to disallow FLOGI
FLOGI one or more FLOGIs accepted
PROBING FCP probing in progress
WAS_MLOG Was a port capable of multiple logins before disable (could be loop
or NPIV port)
BUFFER_LIMITED Number of buffers available to a port is reduced
AOQ Port is F_Port and in extended VC mode
LISL_PORT Port is a Logical port (LISL/VSAN/TAport)

Device Connectivity
Revision 0213
SAN-TS 300
4 25
Device Connectivity
Revision 0213
SAN-TS 300
Is
something
plugged
in?
Is it a
loop
device?
G_Port
U_Port
L_/FL_Por
t
F_Port
E_Port
Is it a
fabric
point to
point
device
No
Yes
Yes
No
No
Yes
Note: Condor3 ASICs do not
support loop ports
4 26
The example on this slide shows an instance where both the device and switch ports are
hard set to different speeds. Since auto-negotiation does not occur the switch and
attached device are unable to complete the speed negotiation process.

Here we see that the port has been locked to 4 Gbit/sec with the command:
Switch1:admin> portcfgspeed 1 4
This can be confirmed with a portcfgshow or portshow <port>, but
switchshow has already shown this above.
Device Connectivity
Revision 0213
SAN-TS 300
4 27
Device Connectivity
Revision 0213
SAN-TS 300
4 28
Device Connectivity
Revision 0213
SAN-TS 300
4 29
Footnote 1:
Link Fail - If a port remains in the Link Reset (LR) Receive State for a period of time
greater than a timeout period (R_T_TOV), a Link Reset Protocol Timeout shall be
detected which results in a Link Failure condition (enter the NOS Transmit State). Also
indicates loss of sync or loss of signal lasting longer than Receiver Transmitter Timeout
Value (R_T_TOV) while port was not in the Offline State; both will cause only the Link
Failure counter to increase. For loss of sync lasting shorter then R_T_TOV the port will
remain in the active state and the Loss of Sync counter will increase.

Per Fibre Channel standards, the default R_T_TOV value is 100 milliseconds but can be
set as low as 100 microseconds.
Device Connectivity
Revision 0213
SAN-TS 300
4 30
Device Connectivity
Revision 0213
SAN-TS 300
4 31
Device Connectivity
Revision 0213
SAN-TS 300
4 32
Fabric OS switches also support a port fencing option with Fabric Watch which will
disable a port when a threshold is reached.
Switch1:admin> errdump
2009/03/17-22:21:07, [FW-1170], 10,, WARNING, Switch1, , Port#1,Loss of
Signal, is above high boundary (High=1, Low=0). Current value is 3
Error(s)/second.

2009/03/17-22:21:19, [FW-1168], 15,, INFO, Switch1, , Port#1,Loss of
Signal, value has changed(High=1, Low=0). Current value is 2
Error(s)/second.

2009/03/17-22:21:19, [FW-1170], 16,, WARNING, Switch1, , Port#1,Loss of
Signal, is above high boundary (High=1, Low=0). Current value is 2
Error(s)/second.

Signal, value has changed(High=1, Low=0). Current value is 1
Error(s)/second.

Signal, is between high and low boundaries (High=1, Low=0). Current value
is 1 Error(s)/second.

2009/03/17-22:21:57, [FW-1192], 24,, INFO, Switch1, , FOP Port#1,State
Changes, value has changed(High=5, Low=0). Current value is 2
Change(s)/minute.
Device Connectivity
Revision 0213
SAN-TS 300
4 33
Port LEDs are flashing
Probable cause and recommended action: Depending on the rate of the flash and
the color of the port LED this could mean several things. To determine what is
happening on either your port status LED or power status LED, refer to that
switchs model hardware reference manual. There is a table that describes the
LEDs purpose and explains the current behavior as well as provides suggested
resolutions.
Port LEDs are steady
Probable cause and recommended action: The color of the port LED is important in
this instance. To determine what is happening on either your port status LED or
power status LED, refer to that switchs model hardware reference manual. There
is a table that describes the LEDs purpose and explains the current behavior as
well as provides suggested resolutions.
No light from the port LEDs
Probable cause and recommended action If there is no light coming from the port
LED, then no signal is being detected. Check your cable and SFP to determine the
physical fault.

Device Connectivity
Revision 0213
SAN-TS 300
4 34
Device Connectivity
Revision 0213
SAN-TS 300
4 35
Speed Displays AN for auto speed negotiation mode, or a specific speed of 1, 2, 4, or 8
Gbits/sec. This value is set by the portcfgspeed command.
AL_PA Offset 13 Displays (...) or OFF when the arbitrated loop physical address (AL_PA) on the
port is configured to use a 0x0 AL_PA address (default). Displays ON when the address
configuration is 0x13 AL_PA. This value is set by the portcfgalpa command.
Trunk Port Displays ON when port is set for trunking. Displays (..) or OFF when trunking is
disabled on the port. This value is set by the portcfgtrunkPort command.
Long Distance Displays (..) or OFF when long distance mode is off; otherwise, displays long
distance levels as shown below. This value is set by the portcfglongdistance command.
LE The link is up to 10 km
LD The distance is determined dynamically
LS The distance is determined statically by user input
Locked L_Port Displays ON when the port is locked to L_Port only. Displays (..) or OFF when
L_Port lock mode is disabled and the port behaves as a U_Port). This value is set by the
portcfglport command.
Locked G_Port Displays ON when the port is locked to G_Port only. Displays (..) or OFF when
G_Port lock mode is disabled and the port behaves as a U_Port. This value is set by the
portcfggport command.
Disabled E_Port Displays ON when the port is not allowed to be an E_Port. Displays (..) or OFF
when the port is allowed to function as an E_Port. This value is set by the portcfgeport
command.

Device Connectivity
Revision 0213
SAN-TS 300
4 36
ISL R_RDY Mode Displays ON when ISL R_RDY mode is enabled on the port. Displays
(..) or OFF when ISL R_RDY mode is disabled. This value is set by the
portcfgislmode command.
RSCN Suppression Displays ON when RSCN suppression is enabled on the port.
Displays (..) or OFF when RSCN suppression is disabled. This value is set by the
portcfg rscnsupr command.
Persistent Disable Displays ON when the port is persistently disabled; otherwise
displays (..) or OFF. This value is set by the portcfgpersistentdisable command.
NPIV capability Displays ON when N_Port ID Virtualization (NPIV) is enabled on the
port (default). Displays (..) or OFF when NPIV capability is disabled. This value is set by
the portcfgnpivport command.
QOS E_Port Displays ON when Quality of Service (QoS) is enabled on the port. Displays
(..) or OFF when QoS is disabled. By default, QoS is enabled by best effort based on
availability of buffers. This value is set by the portcfgqos command.
EX_Port Displays ON when the port is configured as an EX_Port. Otherwise displays (..)
or OFF. This value is set by the portcfgexport command.
Mirror Port Displays ON when Mirror Port is enabled on the port. Displays (..) or OFF
when Mirror Port is disabled. This value is set by the portcfg mirrorport
command.
Device Connectivity
Revision 0213
SAN-TS 300
4 37
FC Fastwrite Displays ON when FC Fastwrite is enabled on the port or (..) or OFF when
disabled. Fastwrite is disabled by default. This value is set by the portcfg
fastwrite command.
Rate Limit Displays ON when ingress rate limit is set on the port or (..) or OFF when
the ingress rate limiting feature is disabled. This value is set by the portcfgqos --
setratelimit command. The default value is OFF.
Credit Recovery Displays ON when Credit Recovery is enabled on the port or (..) or
OFF when disabled. This value is set by the portcfgcreditrecovery command.
The credit recovery feature is enabled by default, but only ports configured as long
distance ports can utilize this feature.
Port Auto Disable This is the Port Fencing feature. Displays On when the Auto
Disable feature is enabled on a port or (..) when disabled. This feature causes ports to
become disabled when they encounter an event that would cause them to reinitialize.
This feature is enabled by the portcfgautodisable command. The feature is
disabled by default.

Device Connectivity
Revision 0213
SAN-TS 300
4 38
Switch1:admin> portcfglport 1
Usage: portcfglport PortNumber <0 | 1> [0 | 1] [0 | 1 | 2]

Switch1:admin> portcfglport 1 1
Port 1 is already locked as a G-Port

Need to unlock port from one mode to lock it in another mode:
Switch1:admin> portcfggport 1 0
Device Connectivity
Revision 0213
SAN-TS 300
4 39
Switch1:admin> portcfglport 1
Usage: portcfglport PortNumber <0 | 1> [0 | 1] [0 | 1 | 2]

Port 1 is already locked as a G-Port

Need to unlock port from one mode to lock it in another mode:
Switch1:admin> portcfggport 1 0
Device Connectivity
Revision 0213
SAN-TS 300
4 40
The portswap command was originally introduced to assist in troubleshooting FICON
issues. The portswap command is intended to be used as a temporary
troubleshooting measure and should not be implemented as a permanent solution.
Device Connectivity
Revision 0213
SAN-TS 300
4 41
Device Connectivity
Revision 0213
SAN-TS 300
4 42
Device Connectivity
Revision 0213
SAN-TS 300
4 43
Once a link is established a device must login with the fabric and request a 24-bit Fibre
Channel address. During this time the device will register the number of buffer-to-buffer
credits it has available, its max receive frame size, and the Class of Service (CoS)
supported.

Footnote 1: Can also use portlogdumpport port_index command with filters
the portlogdump for a specific port. The portlogdump output will be covered in detail in
the Appendix portlogdump module.

Device Connectivity
Revision 0213
SAN-TS 300
4 44
The portshow command displays the following output:
portFlags Bitmap and English translation of the ports login process
portState
Online - up and running
Offline - not online, portPhys gives details
Testing - running diagnostics
Faulty - failed diagnostics
portPhys
No_Module - no module (GBIC or other) present
No_Sync - receiving light but out of sync
In_Sync - receiving light and in sync
Laser_Flt - module is signaling a laser fault
Port_Flt - port marked faulty
Diag_Flt - port failed diagnostics
Lock_Ref - locking to the reference signal
portId 24-bit Fabric Address, port identifier (PID) of device
portScn F_Port, from the fabrics point of view all end devices that successfully logged in are
F_Ports
Port WWN of connected device an F_Port will have one WWN; an FL_Port can have multiple
WWNs
Distance and speed configuration of the port

Device Connectivity
Revision 0213
SAN-TS 300
4 45
Login Device to switch connectivity
FLOGI to Fabric Port (FFFFFE)
Security Policy Check Device Connection Control POLICY (DCC_POLICY) Access
Control List (ACL)
Accept: Assign fabric unique 24-bit address
No response: Do not assign fabric address
PLOGI to Name Server (FFFFFC)
The fabric could also have a fabric-wide consistency policy that enforces the DCC policy
across all the switches in the fabric.

Switch1:admin> fddcfg -show
Local Switch Configuration for all Databases:-
DATABASE - Accept/Reject
---------------------------------
SCC - accept
DCC - accept
PWD - accept
FCS - accept
AUTH - accept
IPFILTER - accept
Fabric Wide Consistency Policy:- "DCC"

This DCC output is
indicative of a fabric-wide
DCC consistency policy of
tolerant. A strict
policy would be displayed
as DCC:S.
Device Connectivity
Revision 0213
SAN-TS 300
4 46
Device Connectivity
Revision 0213
SAN-TS 300
4 47
Device Connectivity
Revision 0213
SAN-TS 300
4 48
Once the fabric login is completed the next step is registering with the name server. The
device registers its attributes with the name server. In addition to the device registration
the name server also probes the device to attempt to gather additional information. To
see information about the device probing run the fcpprobeshow command.

Device Connectivity
Revision 0213
SAN-TS 300
4 49
After the FLOGI has completed, the HBA will send a PLOGI request to FFFFFC asking for
permission to log into the Name Server.
If a device does not perform a PLOGI or the device does not receive an ACCEPT from the
switch, the device does not complete login into the Name Server. This can be verified
through nsshow (no entry in Name Server). On occasion a device may only be partially
registered with the Name Server. In a case like this it is necessary to first know how a
device is supposed to appear in the Name Server in order to spot the differences.
Device Connectivity
Revision 0213
SAN-TS 300
4 50
State Change Register (SCR) Nx_Port request to receive notification when something
in the fabric changes. FC devices that choose to receive RSCNs must register for this
service.
Devices send a SCR to FFFFFD
Registration indicates that the device wants to be notified of changes
Devices register after PLOGI to Name Server
Registered State Change Notification (RSCN) Issued by the Fabric Controller Service or
an Nx_Port to devices that registered
Only sent to devices within an affected zone
Initiators should register for RSCNs using SCR. This is commonly a function within the
driver and may not be changed with any configuration files. Targets do not register for
SCNs.
SCR 0 No SCR registration
SCR 1 Fabric detected registration
Device registered to receive all RSCNs issued by Fabric Controller for events detected by
fabric
SCR 2 Nx_Port detected registration
Device registered to receive all RSCN requests issued for events detected by that affected
Nx_Port
Device Connectivity
Revision 0213
SAN-TS 300
4 51
SCR 3 Register to receive all RSCNs within the zone
4 51
Revision 0213
The Fabric Controller Service (FFFFFD) alerts device that changes have occurred in
the fabric by sending a Registered State Change Notification (RSCN) if:
Device registered to receive RSCN using an SCR
A new device has been added (within the same zone)
An existing device has been removed (within the same zone)
A zone has been changed
A switch name or IP address changed
The fabric reconfigured
Registration is optional
SCSI initiators normally register
SCSI targets may not register

The Fabric Controller (FFFFFD) is responsible for routing changes, topology changes
and the SCR/SCN/RSCN processes.
Fabric Controller (FFFFFD) service is a required logical entity within a fabric that
controls the general operation of the fabric. It is the fabric owner as well as the traffic
controller. Functions include fabric initialization, frame routing management,
generation of link responses, and setup and tear down of dedicated connections. Since
Fabric Controller is such an important service, Fibre Channel deploys a fully distributed
environment for this service. The Fabric Controller exists in every single switch in a
fabric, therefore, there is no single point of failure.
Major fabric management responsibilities:
Execution of the fabric initialization procedure
Advertise RSCN (Registered State Change Notification)
Major traffic management responsibilities:
F_Ports are interconnected by a routing function which is managed by Fabric
Controller and allowing frames to flow from one F_Port to another N_Port connect
to a F_Port in the fabric.
Setup and tear down of dedicated connections
Perform general frame routing
Parse and routes of frames directed to well-known addresses
Generation of class 2 F_BSY (Fabric Busy) and F_RJT (Fabric Rejected) link
responses

Device Connectivity
Revision 0213
SAN-TS 300
4 52
Device Connectivity
Revision 0213
SAN-TS 300
4 53
There are several commands used to identify and locate devices within the fabric:
switchshow displays devices and whether they are logged into the local switch
nsshow displays devices logged in the local Name Server
nscamshow displays devices logged in a remote Name Server (other switch within the
fabric)
nsallshow Lists 24-bit PID addresses of all devices logged into the fabric
nodefind specify with ALIAS,WWN, or PID to locate Name Server information (local
or remote) within the fabric.

Footnote1: The switchshow is contained in the SSHOW_SYS output. The nsshow,
nscamshow and nsallshow is contaned in the SSHOW_SERVICE output from the
active CP. The nodefind command is not part of supportsave.

Device Connectivity
Revision 0213
SAN-TS 300
4 54
Use nsshow to verify that a device logged into the Name Server. We can further verify
information about that device, below we see the following information in the PortSymb
field:

Vendor: Emulex
Model: LP1150-F4
Firmware Version: V2.10A7
Driver Version: V5.20A9

Switch1:admin> nsshow
{
Type Pid COS PortName NodeName TTL(sec)
N 0a0000; 2,3;10:00:00:00:c9:51:35:96;20:00:00:00:c9:51:35:96; na
FC4s: FCP
NodeSymb: [52] "Emulex LP1150-F4 FV2.10A7 DV5-5.20A9 RSL1-ST15-W2K-1"
Fabric Port Name: 20:00:00:05:1e:02:0c:77
Permanent Port Name: 10:00:00:00:c9:51:35:96
Port Index: 0
Share Area: No
Device Shared in Other AD: No

Device Connectivity
Revision 0213
SAN-TS 300
4 55
Device Connectivity
Revision 0213
SAN-TS 300
4 56
Device Connectivity
Revision 0213
SAN-TS 300
4 57
Initiators perform the PLOGI and PRLI handshake from initiator Nx_Port to target
(storage) Nx_Port. After this occurs, the initiator issues SCSI commands such as Report
LUNs, Test Unit Ready, Start, and Read/Write to targets. This is all a function of the HBA,
firmware, system drivers, and configuration files.

Device Connectivity
Revision 0213
SAN-TS 300
4 58
fcping is a command that can verify connectivity and verify zoning. fcping performs
two operations:
Sends five Fibre Channel Extended Link Service (ELS) Echo request to a pair of
ports or to a single destination, or executes a SuperPing.
Does a Zone Database check to make sure devices are zoned together
See Fabric OS command reference manual for more information and usage
options.

Devices that do not support the ELS ECHO will time out, but that does not mean there is
no physical connectivity. Verify connectivity through previously supported methods of
switchshow, nsshow, and nsallshow.

Device Connectivity
Revision 0213
SAN-TS 300
4 59
Device Connectivity
Revision 0213
SAN-TS 300
4 60
Other command line options for nszonemember are PID or WWN:
Switch1:admin> nszonemember -a
Port: 0 Pid: 0x0a0000 Aliases: Windows_HBA
Zoned Members: 3 devices
Pid: 0x0a0000 Aliases: Windows_HBA
Pid: 0x1400e4 Aliases: B41_JBOD2
Pid: 0x1400ef Aliases: B41_JBOD4

Port: 1 Pid: 0x0a0100 Aliases: Sun_HBA
Zoned Members: 2 devices
Pid: 0x0a0100 Aliases: Sun_HBA
Pid: 0x1400e8 Aliases: B41_JBOD3

Note: This verifies that devices are zoned together within the Effective Configuration. It
does not display whether devices are logged in and online to the switch.

Device Connectivity
Revision 0213
SAN-TS 300
4 61
You can use the nszonemember u command to look for unzoned devices in the fabric.

Switch1:admin> nszonemember -u
Pid: 0x020400; Aliases:
Total of 3 unzoned device(s) in the fabric.

Device Connectivity
Revision 0213
SAN-TS 300
4 62
Footnote 1: Though it is theoretically possible for GoldenEye, GoldenEye2, Condor and
Condor2 ASICs to run RAS Logentries, it is very improbable as the number of CAM
entries for these ASICs is significantly higher.
Use portcamshow to see available hard enforcement resources:
rsl1_st15_41_1:admin> portcamshow 5
--------------------------------------------------
Port SID used DID used SID entries DID entries
5 0 0 000000 000000
--------------------------------------------------
SID free, DID free: (49152, 49152)
If all resources are used, then out of CAM entries error is reported in RAS Log. At this
point a port will use session based enforcement. This has not been an issue since
Bloom ASICs, starting with Condor the number of CAM entries was increased
significantly.
If a mix of WWN and <D,PI> are defined within a zone, it will use session based
enforcement.
Preference is all WWN definitions and hard enforcement.
Device Connectivity
Revision 0213
SAN-TS 300
4 63

State Change Notifications (SCN) are used for internal state change notifications. This is
the switch logging that the port is online or is an Fx_port. SCNs are not sent from the
switch to the Nx_ports and should not be confused with RSCNs.

Note: Devices can send RSCNs to the fabric if they change their Name Server attributes.

Device Connectivity
Revision 0213
SAN-TS 300
4 64
Footnote 1: This command is the same as fabstateshow which will be obsolete at
some point in the future.

Note: The *Removing all nodes from port entry is listed when a port goes
offline and after a port online occurs when a port can no longer be an E_Port. In this
case the port has come online as an F_Port.

An asterisk (*) in the message indicates an action.
Device Connectivity
Revision 0213
SAN-TS 300
4 65
Possible values for S, Sn are:
F0 Non-Disruptive Fabric Reconfiguration request, Build Fabric BF, transition
state
F1 Reconfigure Fabric (RCF) is not supported
F2 EFP idle state
F3 Flood EFPs
D0 Setup: At the completion of Principal Switch Selection, the Principal Switch
shall assume the role of Domain Address Manager
D1 Send DIA: The Principal Switch shall then transmit a DIA SW_ILS request
Sequence on all E_Ports
D2 Idle: The Principal Switch shall remain in this state until it receives an RDI
SW_ILS request Sequence. Reception of RDIs and or EFPs shall be queued
in this state
D3 The principal switch is processing the request Domain ID
A0 Get Domain ID: At the completion of Principal Switch Selection, the Switch
receives the DIA SW_ILS request Sequence via the upstream Principal ISL.
A1 Send DIA: After the Switch is granted a Domain ID, it shall then transmit a
DIA SW_ILS request Sequence via all ISLs other than the Principal ISL
A2 Idle: The Switch shall remain in this state until it receives an RDI SW_ILS
request Sequence. Reception of RDIs and or EFPs shall be queued in this
state
A3 A non-principal switch is processing the request Domain ID
S0 The switch is in offline state
Possible values for P, Pn are:
P0 Port offline
P1 Port online
P2 Exchange Link Parameter (ELP) Accept Frame (ACC) received.
P3 Link reset occurred on master or E_Port
I0 Trunk Initiator: Exchange Mark Timestamp (EMT) sent.
I1 Trunk Initiator: Exchange Trunking Parameters (ETP) Accept Frame (ACC)
received.
I2 Trunk Initiator: ETP sent.
I3 W Trunk Initiator: Link reset occurred.
I4 Trunk Initiator: Link reset done on slave.
T0 Trunk Target: EMT received.
T1 Trunk Target: ETP received.
T2 Trunk Target: Link reset.
T3 Trunk Target: Link reset done on slave.
LD Dynamic long distance ECP sent or received.
ESC Exchange Switch Capabilities (ESC) state between P2 and P3.

Device Connectivity
Revision 0213
SAN-TS 300
4 66
Device Connectivity
Revision 0213
SAN-TS 300
4 67
Device Connectivity
Revision 0213
SAN-TS 300
4 68
HCM can be used to see devices the sever has access too. There are also counters for
the HBA ports as well as detailed information on discovered targets and LUNs.
Device Connectivity
Revision 0213
SAN-TS 300
4 69
Device Connectivity
Revision 0213
SAN-TS 300
4 70
Device Connectivity
Revision 0213
SAN-TS 300
4 71
To clear the statistics for a port use the bcu fabric --statsclr <portID>
command.
Device Connectivity
Revision 0213
SAN-TS 300
4 72
Device Connectivity
Revision 0213
SAN-TS 300
4 73
Device Connectivity
Revision 0213
SAN-TS 300
4 74
Device Connectivity
Revision 0213
SAN-TS 300
4 75
Device Connectivity
Revision 0213
SAN-TS 300
4 76
Device Connectivity
Revision 0213
SAN-TS 300
4 77
Device Connectivity
Revision 0213
SAN-TS 300
4 78
Device Connectivity
Revision 0213
SAN-TS 300
4 79
Device Connectivity
Revision 0213
SAN-TS 300
4 80
Device Connectivity
Revision 0213
SAN-TS 300
4 81
Today, a customer does not have to bring down the SAN or interrupt production traffic to
install an analyzer and collect data to aid in troubleshooting Fibre Channel end-to-end
link communication. The port mirroring feature allows a customer configure a switch
port as an analyzer port to mirror a specific source port and destination port traffic
passing though a switch port. The port mirroring feature will not completely replace
inline analyzers due to some minor limitations. Port mirroring is only supported on
Condor, Condor2, and GoldenEye2 based platforms. Port mirroring cannot be
implemented on any 1 or 2 Gbit/sec based platforms or on GoldenEye based platforms
(Brocade 200E or embedded products).
The port mirroring feature will mirror the traffic in both directions between the source
identifier and the destination identifier to a single mirror port. It will create and delete
mirror connections between two identifiers. All traffic between the two identifiers will be
mirrored to the specified mirror port. The user should connect a FC analyzer to this
mirror port to capture all the mirrored traffic. The analyzer will only need one connection
between the port and the analyzer, this one connection will capture traffic in both
directions.
In the ingress directions, traffic originating from the source identifier and destined to the
destination identifier are mirrored to the mirrored port. In the egress direction, traffic
originating from the destination identifier and destined to the source identifier are
mirrored to the mirror port.
Device Connectivity
Revision 0213
SAN-TS 300
4 82
The idea of port mirroring is to capture traffic between two devices. We chose not to
mirror all the traffic from one device received and transmitted because it is not required,
i.e. if there is an issue between two devices mirroring that SID/DID pair is enough. A
complete port mirror would require two mirror ports to provide enough bandwidth to
support full line rate traffic. In addition, two ports would be consumed by the mirror
connection to support each direction of traffic. A user would then need to connect a FC
analyzer to each mirror port.
Examples of communication between an end device and a switch include Fabric Logins
(FLOGIs), FLOGI ACC, Name Server Fibre Channel Common Transport (FC_CT) Requests
and Responses, State Change Registrations (SCRs), and Registered State Change
Notifications (RSCNs).

Port Mirroring Is Port Mirroring Can Not
Capable of mirroring end-to-end traffic
Mirror port can be any non-shared port
located on the same switch as the
source identifier (SID)
Can be uses to detect missing frames
(zoning issues/hold timeout)
Can be used to capture protocol errors
Can be used to capture ULP traffic
(SCSI/FICON)
Supported on Condor, Condor2, and
GoldenEye2 platforms
Debug frames with invalid SID or DID
Debug link issues
Debug embedded switch traffic
Debug frames to well-known
addresses
Be implemented on any 1 or
2Gbit/sec based platforms or
GoldenEye ASIC based platforms
(Brocade 200E or embedded
products)
Mirror E_Ports
Device Connectivity
Revision 0213
SAN-TS 300
4 83
Device Connectivity
Revision 0213
SAN-TS 300
4 84
Footnote 1: Disable of the mirrorport connection will cause frames to be received out of
order. If IOSET is enabled, a frame will be dropped during this step. All other steps are
nondistuptive to I/O.
If the mirror port was not online you will get the following message:
ST01-B48:AD255:admin> portmirror --add 1/0 0x010c00 0x0163e8
Port Mirror: mirror port is offline.
Configure a mirror port using the CLI command portcfg mirrorport.
Switch> portcfg mirrorport 1/0 --enable
Connect a FC analyzer to the configured mirror port.
Setup a port mirror connection between the two F_PORT devices using the CLI
command portmirror --add.
Switch> portmirror --add 1/0 0x0a0500 0x0a0800
Start FC Analyzer capture, reproduce problem, stop FC analyzer capture, review FC
Analyzer trace.
Remove port mirror connection using the CLI command portmirror --delete.
Switch> portmirror --delete 1/0 0x0a0500 0x0a0800
Remove the mirror port using the CLI command portcfg mirrorport.
Switch> portcfg mirrorport 1/0 --disable
Device Connectivity
Revision 0213
SAN-TS 300
4 85
Device Connectivity
Revision 0213
SAN-TS 300
4 86
Footnote 1: ZONE-1058
Message <timestamp>, [ZONE-1058], <sequence-number>,, WARNING, <system-name>, Domain
<Domain ID of the switch that becomes unreachable> present in TI zone <TI zone name> became
unreachable due to failover disabled mode.
Probable Cause Indicates that the domain present in the Traffic Isolation (TI) zone path is unreachable.
This occurs if the TI zone paths are unavailable or the TI zone is set up incorrectly.
Recommended Action: Verify that the paths defined by TI zones are online or remove the domain from the
TI zone.
Severity WARNING
ZONE-1059
Message <timestamp>, [ZONE-1059], <sequence-number>,, WARNING, <system-name>, Unexpected TI
routing behavior or a potentially un-routable TI configuration has been detected on local domain <Domain
ID of the local Logical Switch where the error
was detected>.
Probable Cause Indicates that the current fabric topology and TI Zone configuration may result in an
unroutable condition or unexpected routing behavior.
Recommended Action: Execute the zone --showTIerrors command on the specified switch to report the
conflicting configuration details.
Severity WARNING
ZONE-1060
Message <timestamp>, [ZONE-1060], <sequence-number>,, WARNING, <system-name>, Non-TI and TI
failover-enabled traffic restricted to domain <Domain ID> due to TI failover-disabled zoning.
Probable Cause Indicates that only TI failover-disabled paths remain to reach the given domain causing
non-TI and TI failover traffic disruption.
Recommended Action: Add or restore the non-TI ISLs and/or TI failover-enabled ISLs to the specified
domain.
Severity WARNING

Device Connectivity
Revision 0213
SAN-TS 300
4 87
Zone --showTIerrors
Analyzes real and potential routing problems with the activated TI zoning set and prints
a report. This command must be executed in the local domain and analyzes only that
domain.
Error Types: Error and Warning:
Error type records indicate that a problem is present within the fabric given the current
set of online devices and activated TI zone configuration. In this case, if traffic between
the involved devices has already been started, frames are likely to be dropped within the
fabric.
Warnings are not currently a problem given the current set of online devices and ports
and reachable domains. Traffic may not be getting dropped in the fabric at the moment.
However, given the activated TI zone configuration, parallel exclusive paths between a
shared device and a remote domain have been detected which might cause a issue for
devices that join the fabric later and attempt to start communicating.

Device Connectivity
Revision 0213
SAN-TS 300
4 88
Device Connectivity
Revision 0213
SAN-TS 300
4 89
Device Connectivity
Revision 0213
SAN-TS 300
4 90
Device Connectivity
Revision 0213
SAN-TS 300
4 91
Looking at the switchshow output: The device is present however the it has been removed
from the Name Server
Sw1:admiin> switchshow
<truncated output>
Index Slot Port Address Media Speed State Proto
===================================================
<truncated output>
2 1 2 010200 id N8 Online FC F-Port 10:00:00:05:1e:db:69:d9
<truncated output>

Run nsshow command shows no entries
Sw1:admin> nsshow
There is no entry in the Local Name

Device Connectivity
Revision 0213
SAN-TS 300
4 92
The option is under the F-Port login parameters:
Sw1:admin> switchdisable
Sw1:admin> configure
Configure...
Fabric parameters (yes, y, no, n): [no]
Virtual Channel parameters (yes, y, no, n): [no]
F-Port login parameters (yes, y, no, n): [no] y
Maximum logins per switch: (1..25200) [3200]
Logins per second: (0..200) [0]
Login stage interval (milli-seconds): (0..10000) [0]
Stage FDISC logins with busy reject:
[1-255] - Number of logins without staging
0 - No staging: (0..255) [0]
Enforce FLOGI/FDISC login: (0..1) [0]

Default
is 0
Device Connectivity
Revision 0213
SAN-TS 300
4 93
Device Connectivity
Revision 0213
SAN-TS 300
4 94
Device Connectivity
Revision 0213
SAN-TS 300
4 95
Footnote 1: Switch will use a GE_PT (Get Entries by Port Type) command to verify device.
Each of the node devices typically determine the properties of the other node devices
with which it communicates. Upon connecting to the network, the node devices send a
request addressed to the name server, which is then received by the resident name
server on the entry switch. Typically, where such request forms are supported, the
request takes the form of GE_PT (get entries of a given Port Type) or GE_FT (get entries
of a given FC-4 Type). Where such forms are not supported, the request may take the
form of GID_PT (get identifiers for ports of a given Port Type) or GID_FT (get identifiers
for ports of a given FC-4 Type). Once the identifiers have been obtained, a series of
GE_ID (get entry for a given identifier) requests may be used to obtain the corresponding
entries. In either case, the effect is to cause the entry switch to request each of the
other switches to send all name server database entries that satisfy the given criteria to
the entry switch, which then forwards the entries to the requesting device. As the
number of entries is generally proportional to the number of node devices, and each
device typically generates such a request, the amount of traffic increases as the square
of the number of node devices.

Device Connectivity
Revision 0213
SAN-TS 300
4 96
Footnote 1: If duplicate is verified a RASLOG message is generated.

Device Connectivity
Revision 0213
SAN-TS 300
4 97
Device Connectivity
Revision 0213
SAN-TS 300
4 98
Device Connectivity
Revision 0213
SAN-TS 300
4 99
Footnote 1: GE_PT Get Entries by Port Type
Device Connectivity
Revision 0213
SAN-TS 300
4 100
Why not allow all duplicate devices to remain in the fabric?
Device WWN is intended to be a unique value. Many of the FC-GS query responses only
support a single-entry response when device WWN is used as the qualifier. If two
devices exist in the fabric with the same device WWN, then response to such queries will
not be consistent; likewise for zoning enforcement where the device WWN is used to
define connectivity. If Fabric OS simply logged the condition and allowed the devices to
exist, disruption is still a possibility. This situation is worse than removing the devices
completely, as the configuration might give the appearance of working properly (since all
conflicting devices are still online), but the intended I/O may differ from the actual
results.

Device Connectivity
Revision 0213
SAN-TS 300
4 101
Why not retain just one of the duplicate devices instead of removing them all?
In order to retain a single device that has duplicates, a decision has to be made
regarding which device is the right device. It is often suggested that the most current
login or SCN represents the correct device, and that the older device does not require
verification prior to removal. The weakness of this assumption is that we cannot ensure
that all SCNs are delivered and processed in an identical order for every switch. With
remote duplicates, making the decision based solely on the arrival order of an SCN
would lead to an inconsistent Name Server database across all switches in the fabric.
Other issues:
Fabric Build Scenario A - In the case where a switch is joining a fabric, and the fabric has
existing duplicates, the login order cannot be deduced by the joining switch. The joining
switch would simply have to guess which device is the correct one. Similar issues exist
when considering HA failover.
Fabric Build Scenario B - Another case where order cannot be determined is where a
switch is joining a fabric and this introduces a duplicate condition. It is not possible to
determine either the login order or correctness of the joining device versus the existing
device. Again, FOS would simply have to guess which device is the correct one. Similar
issues exist when considering HA failover.
With any of the above scenarios the method for choosing the correct device would be
imperfect. If we choose to favor local devices over remote devices (or vice versa) it would
lead to an inconsistent Name Server database across the fabric. If we use a numeric
selection process and, for example, have all switches favor the device with the highest
(numerically) PID we would ensure a consistent Name Server database, but could offer
no guarantee that the selection would be the best choice.
The only case where order can be used to determine which device should be retained is
the one where all duplicates occur on the same (local) switch. The local switch has
control over all logins and can determine the login order for every device. As such, our
approach does retain a single device, rather than removing all devices, in this particular
case. Beyond this case, our approach removes all conflicting devices from the Name
Server database uniformly across the fabric. The Name Server is left in a predictable
state and the user is notified of the condition. The decision regarding which device
should exist in the fabric is left to the fabric administrator, an approach that is aligned
with customer feedback on this issue.

Device Connectivity
Revision 0213
SAN-TS 300
4 102
Device Connectivity
Revision 0213
SAN-TS 300
4 103
Device Connectivity
Revision 0213
SAN-TS 300
4 104
Portlog Analysis
Revision 0213
SAN-TS 300
5 1
Portlog Analysis
Revision 0213
SAN-TS 300
5 2
Footnote 1: The portlog captures information for the entire physical switch even if
virtual switches are created.
Footnote 2: The portlog captures portions of Fibre Channel frames used by devices
and switches to communicate. It also captures other events that do not use FC
frames. The caution here is not to assume every line displayed as part of a portlog
dump is a FC frame. This module will point out the differences.

Portlog Analysis
Revision 0213
SAN-TS 300
5 3
Portlog Analysis
Revision 0213
SAN-TS 300
5 4
Footnote 1: Portlogs are circular files. They display the activity for all switch ports in
a running log using a First in First out (FIFO) format.
Footnote 2: Though increasing the portlog size is safe on a switch functioning
normally it could cause problems on a switch that is low on memory. For this reason
it is recommend to only increase the portlog size when directed to do so by Brocade
Support/Engineering.

Portlog Analysis
Revision 0213
SAN-TS 300
5 5
The supported portlog size for switches depends on the firmware version running.
Use the portlogconfigshow command to get the current value and
portlogresize command with no value set to verify the supported range.

Switches running FOS v7.1, 7.0, v6.4:
32,768 (default) to 131,072 entries (8510, DCX, DCX-4S, BES)

16,384 (default) to 32,768 entries (All others)
Switches running FOS v6.2:
32,768 (default) to 65,536 entries (DCX, DCX-4S, 48000)

8,192 (default) to 16,384 entries (All others)

Portlog Analysis
Revision 0213
SAN-TS 300
5 6
Portlog Analysis
Revision 0213
SAN-TS 300
5 7
Portlog Analysis
Revision 0213
SAN-TS 300
5 8
Footnote 1: Power on Switch, Power On Self Test (POST), Light Signal, and
Character/Word sync will not be covered.

Speed Negotiation Wait Signal (WS) & Negotiation Complete (NC)

FLOGI / ACC Device will login to Fabric, Fabric accepts

PLOGI / ACC Device will login to Name Server

SCR / ACC Device will register to receive RSCNs

Register & Query/ ACC Initiator will register & probe for SCSI devices

PLOGI/ ACC - Fabric will login to Device, Device accepts

PRLI/ ACC Fabric probes for SCSI information

LOGO/ ACC Fabric logs out of device

FLOGI ACC
FLOGI
Switch
FC
Node
Portlog Analysis
Revision 0213
SAN-TS 300
5 9
An Ordered Set is a four-character combination of data and special transmission
characters. Ordered Sets provide the ability to obtain bit and word synchronization that
also establishes word boundary alignment.

Portlog Analysis
Revision 0213
SAN-TS 300
5 10
Frame delimiters:
SOF identifies the start of a frame and conditions the receiver to begin frame reception.
EOF identifies the end of a frame and deconditions the frame reception logic.
Primitive signals:
IDLE or ARB are transmitted on a link whenever a port is operational and has no other specific
information to send. The transmitter side of a port is always sending words to maintain
synchronization with the receiver at the other end of the link.
Receiver Ready (R_RDY) indicates that the receiver has emptied a receive buffer and is ready to
receive another frame.
Virtual Channel Ready (VC_RDY) are used for buffer-to-buffer flow control on ISLs that support
Virtual Channels.
Primitive Sequences:
Not Operational (NOS) is transmitted by port to indicate that the transmitting port has detected a
link failure or is in an offline condition, waiting for the OLS sequence to be received.
Offline (OLS) is transmitted by a port to indicate the port is beginning the link initialization
protocol, has received and recognized the NOS sequence or is entering the Offline state.
Link Reset (LR) is used to initiate a link reset.
Link Reset Response (LRR) is transmitted by a port to indicate that is has recognized a LR
sequence and has performed the appropriate link reset actions.
Portlog Analysis
Revision 0213
SAN-TS 300
5 11
Portlog interpretation requires FC theory and frame format understanding.
The portlog will capture device-to-switch, switch-to-switch and switch machine code
information.
When device-to-switch communication is captured, Fibre Channel (FC) Words 0, 1, 4 will
be pulled from the FC header and FC Word 6 will be pulled from the first word in the
payload.

Portlog Analysis
Revision 0213
SAN-TS 300
5 12
R_CTL - Routing Control byte is one of the first items to check.
D_ID - Destination ID (port address or Well-Known Address)
CS_CTL - Class specific Control Field. This field is always zero for Classes 2 and
3 per the standards but may change in the future.
S_ID - Source ID (port address or Well-Known Address)
OX_ID - Originator ID - Exchange ID assigned by the originator
RX_ID - Responder ID - Exchange ID assigned by the responder
Data Field/Payload - This is the payload of the frame and can be from 0 to 2112
bytes in length.

Portlog Analysis
Revision 0213
SAN-TS 300
5 13
Footnote 1: For Tx and Rx events, there are 4 valid arguments:
Arg 1: (FC0) is word 0 in the FC frame (Destination PID)
Arg 2: (FC1) is word 1 in the FC frame (Source PID)
Arg 3: (FC4) is word 2 in the FC frame (Originator/Responder exchange IDs)
Arg 4: (FC6) is word 6 in the FC frame (first word in the payload)

Portlog Analysis
Revision 0213
SAN-TS 300
5 14
Portlog Analysis
Revision 0213
SAN-TS 300
5 15
Footnote 1: Light Signal and Character/Word Synchronization isnt recorded in the
portlog because it is performed by the ASIC and happens too quickly to be logged.

The FC standards do define the error recovery process to use when sync is lost
which can be seen in the portlog and is covered in the notes under Not Operational
(NOS) Link Initialization Protocol later in the module.

Footnote 2: 1 Gbps speed is no longer support on Condor3 based switches. 10
Gbps speed is supported on the Condor3 ASIC but uses a specific 10 Gbps SFP
and does not use speed negotiation.
Portlog Analysis
Revision 0213
SAN-TS 300
5 16
Footnote 1: Each device starts speed negotiation at its highest supported speed and
works down until a common supported speed is found.
Portlog Analysis
Revision 0213
SAN-TS 300
5 17
There are four possible commands used by the speed negotiation process:
WS (Wait for Signal) - wait until a signal is detected.
NM (Negotiate Master) - Tx starts at maximum speed and progressively and
cyclically reduces speed. It dwells at each speed t_txcycl to allow the device to
follow. Meanwhile tunes to incoming speeds. It changes Rx speed from maximum
downwards at t_rxcycl periods.
NF (Negotiate Follow) - tests the stability of the Rx speed.
NC (Negotiate Complete) indicates a negotiated speed has been reached
successfully.
Argument 1 WS possible values:
00 - Start speed negotiation
01 - Wait for signal
f0 - Loss of Rx_Sync
ee - Signal (light) received
e0 - Lost light
ff - Sync gained
f0 - Sync lost
Argument 1 NC possible values:
01 = 1 Gbps
02 = 2 Gbps
04 = 4 Gbps
08 = 8 Gbps
10 = 16 Gbps

Portlog Analysis
Revision 0213
SAN-TS 300
5 18
Portlog Analysis
Revision 0213
SAN-TS 300
5 19
Footnote 1: NOS Link Failure Reasons:
Loss-of-Synchronization for more than a timeout period (R_T_TOV) while in the
Word Synchronization Acquired State
Loss-of-Signal while in the Word Synchronization Acquired State
Timeout (R_T_TOV) during the Link Reset Protocol

PORT STATE MACHINE
AC Active State
IDLE Idle
LR1 Link Reset: LR transmit state
LR2 Link Reset: LR receive state
LR3 Link Reset: LRR receive state
LF1 Link Failure: NOS transmit state
LF2 Link Failure: NOS receive state
OL1 Offline: OLS transmit state
OL2 Offline: OLS receive state
OL3 Offline: Wait for OLS state
NOS Not Operational
Portlog Analysis
Revision 0213
SAN-TS 300
5 20
NOS is sent when a device that was previously in the active state goes offline (OL3)
or if a Link Failure is detected (LF2).
OLS is sent when a device comes online for the first time, Link Initialize, or when
receiving NOS.
LR is sent when a port in the active state performing a Link Reset (for example
buffer credit recovery) or when receiving OLS. A port in the Active state that
issues a successful Link Reset doesn't need to login to the fabric (FLOGI) if it had
previously done so.
Portlog Analysis
Revision 0213
SAN-TS 300
5 21
Sample output from 2 Gbit/sec switch
06:31:43.539 INTR pstate 1 LF2
06:31:43.541 INTR pstate 1 OL2
06:31:43.547 INTR pstate 1 LR3
06:31:43.547 INTR pstate 1 AC
Portlog Analysis
Revision 0213
SAN-TS 300
5 22
Extended Link Services will be the most common type of frame to become familiar
with and decode in the portlog. FC-4 Data frames use the Common Transport
protocol and is used for Name Server registrations and queries.

Extended Link Services Requests (R_CTL = 22):
N_Port wants to login to the fabric - FLOGI
N_Port wants to login to the Name Server to register a Symbolic Name - PLOGI
Name Server probes FCP device for its details (type of disk, firmware rev.) - PRLI
N_Port wants to register for State Change Notification - SCR
Fabric Controller sends out a Registered State Change Notification - RSCN

Extended Link Services Replies (R_CTL = 23):
Accepts the request - ACC
Rejects the request - LS_RJT

Portlog Analysis
Revision 0213
SAN-TS 300
5 23
Portlog Analysis
Revision 0213
SAN-TS 300
5 24
Portlog Analysis
Revision 0213
SAN-TS 300
5 25
Footnote 1:
The three most-common Well-Known Addresses are:
FFFFFE is the address for Fabric F_Port Service.
FFFFFD is the address for Fabric Controller Service.
FFFFFC is the address for Name Server Service.
Less common Well-Known Addresses are:
FFFFFF is address for Broadcast
FFFFFA is address for Management Server
FFFFFB is address for Time Server
FFFFF8 is address for Alias Server
FFFCxx is address for Domain Controller (embedded port / switch ID). The xx will
be the Domain ID of the switch.

Portlog Analysis
Revision 0213
SAN-TS 300
5 26
Some ELS commands, such as RSCN, include Page Length and Payload Length
but not all. This slide illustrates an ELS Fabric Login (FLOGI) which doesnt use
Page Length or Payload Length.

Portlog Analysis
Revision 0213
SAN-TS 300
5 27
ELS command code is in word 6 of the FC frame (first word of the frame payload).

Portlog Analysis
Revision 0213
SAN-TS 300
5 28
Footnote 1: Resource Allocation Timeout Value (RA_TOV) and Error Detect
Timeout Value (ED_TOV)
Portlog Analysis
Revision 0213
SAN-TS 300
5 29
ELS command code 04 = FLOGI
ELS command code 02 = Accept
Portlog Analysis
Revision 0213
SAN-TS 300
5 30
Part of the FLOGI request includes common service parameters and class of
service parameters for each class of service 1, 2 and 3. These parameters must
match what the switch supports in order to successfully login to the Fabric.

Common Service Parameters: These parameters apply to all classes of service and
include the FC_PH version supported, BB Credit, max receive frame size and
timeout values. This field represents the basic capabilities of the N_Port.

Portlog Analysis
Revision 0213
SAN-TS 300
5 31
Part of the ELS Accept used to respond to the FLOGI includes common service
parameters and class of service parameters for each class of service 1, 2 and 3.

Common Service Parameters: These parameters apply to all classes of service and
include the FC_PH version supported, BB Credit, max receive frame size and
timeout values. This field represents the basic capabilities of the F_Port.

Portlog Analysis
Revision 0213
SAN-TS 300
5 32
ELS command 03 = PLOGI
ELS command 02 = Accept
Portlog Analysis
Revision 0213
SAN-TS 300
5 33
Footnote 1: SCR can occur before or after the PLOGI.

Portlog Analysis
Revision 0213
SAN-TS 300
5 34
ELS command 62 = SCR
ELS command 02 = Accept
Portlog Analysis
Revision 0213
SAN-TS 300
5 35
State Change Registration (SCR) options:
00 Reserved
01 (Fabric Detected Registration) Register to receive all RSCN requests issued
by the Fabric Controller for events detected by the fabric.
02 (N_Port Detected Registration) Register to receive all RSCN requests issued
by the Fabric Controller for events detected by the affected N_Port or NL_Port.
03 (Full Registration (1 and 2)) Register to receive all RSCN requests issued by
the Fabric Controller for events detected by both.
Portlog Analysis
Revision 0213
SAN-TS 300
5 36
RSL8-ST01-B51:admin> nsshow -r
Type Pid COS PortName NodeName SCR
N 020400; 3;20:02:00:11:0d:e7:50:00;20:02:00:11:0d:e7:50:00; 0x01000003
FC4s: FCP
PortSymb: [36] "Brocade University Virtual FC Target"
Fabric Port Name: 20:04:00:05:1e:0c:ad:e5
Permanent Port Name: 20:02:00:11:0d:e7:50:00
Port Index: 4
Share Area: No
Device Shared in Other AD: No
Redirect: No
Partial: No
The Local Name Server has 1 entry }

Portlog Analysis
Revision 0213
SAN-TS 300
5 37
Portlog Analysis
Revision 0213
SAN-TS 300
5 38
Footnote 1: SCR can occur before or after the PLOGI.
Portlog Analysis
Revision 0213
SAN-TS 300
5 39
Portlog Analysis
Revision 0213
SAN-TS 300
5 40
For a frame, the portlog only captures words 0, 1, 4 and 6. For an ELS frame we
learned the first word of the payload (word 6) is the command code. But for an FC-4
Data frame the command code is in word 8. Another entry in the portlog will identify
the command code from this frame.

Portlog Analysis
Revision 0213
SAN-TS 300
5 41
CT command code is the first two bytes in word 8 of the FC frame.

Portlog Analysis
Revision 0213
SAN-TS 300
5 42
Portlog Analysis
Revision 0213
SAN-TS 300
5 43
Footnote 1: The ctin will display zero, one or two words of additional information,
depending on the request CT command code. The additional information is the
number of words displayed after the command code.
It uses the bit map as follows:
Hex 0000 = Binary 0000000000000000 = 0 words
Hex 0001 = Binary 0000000000000001 = 1 word
Hex 0003 = Binary 0000000000000011 = 2 words

Portlog Analysis
Revision 0213
SAN-TS 300
5 44
Portlog Analysis
Revision 0213
SAN-TS 300
5 45
Footnote 1: The ctout will display zero, one or two words of additional
information, depending on the requested CT command code. The additional
information is the number of words displayed after the command code.
It uses the bit map as follows:
Hex 0000 = Binary 0000000000000000 = 0 words
Hex 0001 = Binary 0000000000000001 = 1 word
Hex 0003 = Binary 0000000000000011 = 2 words

If you see 8001 in the Reply Command code this means the registration was rejected.
Portlog Analysis
Revision 0213
SAN-TS 300
5 46
Portlog Analysis
Revision 0213
SAN-TS 300
5 47
Portlog Analysis
Revision 0213
SAN-TS 300
5 48
Portlog Analysis
Revision 0213
SAN-TS 300
5 49
Portlog Analysis
Revision 0213
SAN-TS 300
5 50
Portlog Analysis
Revision 0213
SAN-TS 300
5 51
Portlog Analysis
Revision 0213
SAN-TS 300
5 52
A Brocade switch is enabled by default to probe devices for type information. This
probing can be disabled using the configure command on a disabled switch, then
changing the Fabric parameter FCP probe disable to a 1 (default is 0 which means
enabled).

A storage device will accept a PLOGI from the switch. Then the switch will do a
Process Login (PRLI). The reason for this is to get / query the storage device about
its FCP information (type of disk i.e. Seagate, driver version, etc.). The Name
Server will store this information in its database for other devices (hosts) to get /
query and build device tables.

Portlog Analysis
Revision 0213
SAN-TS 300
5 53

Portlog Analysis
Revision 0213
SAN-TS 300
5 54
The switch is done with its probing and logs out of the device. The device accepts
the log out. Note: There is NO fabric logout, just N_Port log outs.

Portlog Analysis
Revision 0213
SAN-TS 300
5 55

Portlog Analysis
Revision 0213
SAN-TS 300
5 56
Portlog Analysis
Revision 0213
SAN-TS 300
5 57
Portlog Analysis
Revision 0213
SAN-TS 300
5 58
Footnote 1: The Condor ASIC (4 Gbps) and Condor2 ASIC (8 Gbps) support
FL_Ports. The Condor3 ASIC (16 Gbps) does not.
Portlog Analysis
Revision 0213
SAN-TS 300
5 59
In the example above, the port 0 starts loop initialization (LIP 8002), LIP times out
(TMO), port retries (LIP 801e), times out again, port retries a third time followed by the
port dropping back down to speed negotiation (not shown in truncated output). This
most likely was caused by the NL_Port not ready to perform loop init and is normal
behavior until both ends of the link are ready to start loop init. After the SN is
completed, port 0 again starts loop (LIP 8002) followed by L_Port acquiring an AL_PA
(LIP F7,F7) and the switch port becoming loop master (LIM). Not shown is the ELS loop
init process covered shortly.
Arbitrated Loop Physical Address (AL_PA): A unique one-byte valid value assigned
during Loop Initialization to each NL_Port or FL_Port on a Loop.
Arbitrated Loop Destination Address (AL_PD): The Arbitrated Loop Physical Address of
the L_Port on the Loop that should receive the Primitive Signal.
Arbitrated Loop Source Address (AL_PS): The Arbitrated Loop Physical Address of the
L_Port on the Loop that sent the Primitive Signal.
L_Port: Either an FL_Port or an NL_Port as defined in ANSI X3.230, FC-PH, 3.1.
Without the qualifier "Public or "Private," an NL_Port is assumed to be a Public
NL_Port.
Public Loop: A Loop that includes a participating FL_Port and may contain both Public
and Private NL_Ports.
Public NL_Port: An NL_Port that does a Fabric Login (FLOGI).
Portlog Analysis
Revision 0213
SAN-TS 300
5 60
Loop Primitive Sequence
LIP (8001) Retry loop initialization.
LIP (8002) Start loop after gaining sync
LIP (8003) Restart loop after port reset.
LIP (8004) LIP the loop after loop timeout.
LIP (800d) LIP due to loop rdx buffer overflow.
LIP (800e) Start loop because of l;oop diagnostic.
LIP (801e) In loop initialization and need to retry.
LIP (801f) LIP received from remote port.
LIP (F7xx) Port AL_PA xx requests loop initialization (I.E. loop master
AL_PA 00 sends LIP(F700).
LIP (F7,F7) Used by the originating L_Port to acquire an AL_PA. The loop
port in the initializing state is requesting loop initialization but
does not currently have a valid AL_PA.
LIP (F8,F7) Used to indicate that a Loop failure has been detected. The
L_Port has not completed initialization, therefore, the hex 'F7' is
used instead of a valid AL_PA.
LIP (F7,AL_PS) The loop port identified by the AL_PS value is requesting loop
initialization.
LIP (F8,AL_PS) Used to indicate that a Loop failure has been detected .
LIP
(AL_PD,AL_PS)
The Selective Reset LIP is used to perform a vendor specific
reset at the loop port specified by the AL_PD value. AL_PD=FF
as a destination
indicating all ports.
LIM LISM completed, FL_Port became the loop master
OLP Offline
TMO LIP time out. The loop initialization timed out
Portlog Analysis
Revision 0213
SAN-TS 300
5 61
Portlog Analysis
Revision 0213
SAN-TS 300
5 62
Portlog Analysis
Revision 0213
SAN-TS 300
5 63
Portlog Analysis
Revision 0213
SAN-TS 300
5 64
Switch to Switch Connectivity
1
TS300
Revision 0512
TS300 Switch to Switch Connectivity
<Mod #> 2
Revision 0512
3
TS300
Revision 0512
Footnote 1: Generally but not limited too.

You can typically see the reason for the segmentation in three places: switchshow,
fabstatsshow, errshow (errdump). In the slides that follow, we will review each of
these conditions and associated outputs.
Revision 0512
TS300
6 4
Revision 0512
TS300
6 5
Use Product Information from an M-EOS switch to see similar information.
Revision 0512
TS300
6 6
In the first error message on this slide the other switch rejected this switches exchange
link parameter (ELP) request because the fabric.ops parameters do not match.
Parameters exchanged include: Port_Name and Switch_Name, Class F service
parameters, R_A_TOV and E_D_TOV (part of fabric.ops parameters), and Virtual Channel
(VC) information. The Fabric OS Error Message Guides have the same error message
Probable Cause indicates that the specified switch port is isolated because of a
segmentation due to mismatched configuration parameters. Probable Action is based
on the segmentation reason displayed within the message, look for a possible mismatch
of relevant configuration parameters in the switches at both ends of the link. Run the
configure command to modify the appropriate switch parameters on both the local and
remote switch.
Flow control parameters and a subset of Class-n parameters. If the parameters are
incompatible, the E_Port link will segment.
When switches connect they go through the following initialization process:
Negotiate link speed, if supported
Determine the switch port operating mode
If an F_Port or FL_Port, wait for node to initiate login
If an E_Port, exchange link parameters (ELP) and switch capabilities with neighbor
Select a principal switch during an Exchange Fabric Parameters (EFP) process
Request/assign Domain IDs
Revision 0512
TS300
6 7
Build routing tables and select paths
Keep links alive with HELLO frames every HELLO interval (64 bytes every 10
seconds).
<Mod #> 7
Revision 0512
The fabstatsshow command is brief and concise but be cautious - the counters/"<"s
are not cleared when the segmentation is fixed.
The information displayed is as follows:
Number of times a switch domain ID has been forcibly changed
Number of E_Port offline transitions
Number of fabric reconfigurations
Number of fabric segmentations due to:
Loopback - Number of times this switch segmented port due to port being placed
into mcastloopback mode used to prevent IP/FC broadcast problems.
Incompatibility - Fabric.ops parameters are different.
Overlap - there are duplicate domains in attaching fabrics and fabrics are being hot
plugged
Zoning - cfgmismatch (only one enabled cfg allowed), type mismatch (fabric A has
a zone called eng27, fabric B has an Alias called eng27), content mismatch
(fabric A has an eng27 zone with 2,4; 2,6; 4,6, fabric B has an eng27 zone
with 2,4; 4,6. When a switch is taken out of a fabric for maintenance persistently
disable ports (Fabric OS v3.1/4.1 and higher), clean the zoning out (cfgdisable,
cfgclear, cfgsave), re-attach switch to fabric establish E_Port connection and
allow the existing fabrics zone to propagate to the new switch and then persistently
enable or enable end device ports.

Revision 0512
TS300
6 8
Licensing - Some licenses are required to activate specific features. In some cases
a license may b e required on every switch in the fabric.
Disabled E_Port - a segmentation occurs because a ports E_Port capability was
disabled using the portcfgeport command.
Incompatible management server Platform DBs - Management server platform
DBs when enabled need to be compatible. If you have incompatible platform db
segmentation use the msplcleardb command on merging fabric to clear data
base, incorporate merging fabric members into the existing fabrics access list.
Management server platform databases are enabled (msplmtactivate) and
configured (msconfigure) when applications that use the management server
are desired in a fabric. The management server enables a management
application to access and configure switches in the fabric. It is located at the Fibre
Channel address, FFFFFA. If the access control list (ACL) is empty (default value),
the management server is available to all systems connected in-band to the fabric.
To restrict access, specify the World Wide Name (WWN) for one or more
management applications using the msconfigure command; access is then
restricted to those WWNs. Up to 16 maximum WWNs are supported in the ACL. The
ACL is implemented on a per-switch basis and should be configured on the switch
to which the management application station is directly connected.
Security incompatibility - A security incompatibility could occur for the following
reasons: Unknown incompatibility, Security parameters incompatibility, Exchange
FCS failed, Data incompatibility, MS Platform config incompatibility
Revision 0512
TS300
6 9
Security violations - A security violation could occur if The port used to connect
switches has a DCC policy that does not allow the switch into the fabric. The
fabric has an SCC policy that does not allow switch into the fabric. ECP Error:
Exchange Credit parameter.
<Mod #> 9
Revision 0512
10
TS300
Revision 0512
Note: The Fabric, Zoning, and Extended Fabric licenses are required on each switch. If a
license is not present on a switch it will segment when it tries to join the fabric.
Revision 0512
TS300
6 11
12
TS300
Revision 0512
Configuration mismatches: In the example, the cfg4 zoning configuration has different
definitions in the two fabrics. When you attempt to merge the two fabrics, there will be a
fabric segmentation.

Revision 0512
TS300
6 13
Type mismatches: In the example, the Device1 object is defined as a zone alias in Fabric
A, and as a zone in Fabric B. Because this object is a different type of object in the two
fabrics, an attempt to merge the fabrics will result in a fabric segmentation.

Revision 0512
TS300
6 14
Content mismatches: In the example, the Green_Zone object is a zone in both fabrics;
however, since zoning definitions depend on both the zone members and the order in
which the members are defined, the definitions are thus different between the two
fabrics. As a result, an attempted fabric merge will result in a fabric segmentation.
Revision 0512
TS300
6 15
Footnote 1: The defzone command has two settings: All Access and No Access, if No
access is enabled, when you do a cfgdisable, no devices will be able to access any
other devices in that SAN.

Revision 0512
TS300
6 16
Always begin by running the switchshow command, as this will identify the ISL
connection that reports the fabric segmentation, and thus the remote switch or fabric
that is conflicting with the local switch/fabric. You may want to capture the command
output in a text file to aid making the comparison between the fabrics.
Revision 0512
TS300
6 17
Revision 0512
TS300
6 18
Network Advisor allows you to easily compare two zone databases using a unique
interface. To access the Compare/Merge Zone DBs utility click Zone DB Operation
Compare. The left side of the screen is the reference zone DB and the right side is the
editable zone DB. Changes can be merged into the editable zone DB from the reference
zone DB.

Use the Tree Level and pull-down to view all entries, only zones, or only zone
configurations. Use the Add, Merge, and Merge All buttons to make changes.

The Sync Scroll checkbox can be used to lock the two zone views together so that when
one side scrolls the other side scrolls at the same time. Unchecking the Sync Scroll box
will allow you to move through the two views independently.
SAN Feature Updates
6 19
Revision 1210
PR205
20
TS300
Revision 0512
The configshow method discussed above does not require the switch to be disabled,
resulting in a non-disruptive review. All of the fabric.ops parameters must be identical on
both switches/fabrics. If you have a segmentation due to incompatibility, the output of
switchshow may (newer versions of firmware) tell you exactly which parameter is
mismatched.
Revision 0512
TS300
6 21
Revision 0512
TS300
6 22
You could also change these values by uploading the switch configuration file
(configupload), editing the file manually, then downloading the edited file
(switchdisable; configdownload).
You will see the reason for the segmentation in the following two command outputs and
also in the error log: fabstatsshow and switchshow
Revision 0512
TS300
6 23
Revision 0512
TS300
6 24
In a two-switch fabric, a Brocade 5100 was configured with the following long distance
parameters: portcfglongdistance 8 LS 1 40; the attached Brocade 300E was
configured with the following parameters: portcfglongdistance 8 LD 1 40
The Brocade 5100 displayed the following truncated outputs (it was the first to ELP):
switchshow: 8 8 id N4 Online LS E-Port segmented,(incompatible)
Switch1:admin> errshow -r
Fabric OS: v6.2.0c
2009/03/30-21:37:40, [FABR-1001], 959, FID 128, WARNING, Switch1, port 8, ELP
rejected by the other switch.
The Brocade 300E displayed the following truncated outputs:
switchshow: 8 8 id N4 Online LD E-Port segmented,((Long distance mode
incompat)
Switch1:admin> errshow -r
Fabric OS: v6.2.0c
2009/03/30-21:37:40, [FABR-1001], 2533, FID 128, WARNING, Switch1, port 8,
incompatible Long distance mode.
Both switches had the same truncated fabstatsshow output:
Incompatibility: 2 <
Revision 0512
TS300
6 25
Run portcfgshow on both switches and make sure all the
portcfglongdistance parameters are configured identically.
Starting with Fabric OS v4.4.0, VC translation link initialization (an option of the
portcfglongdistance command) is enabled by default for long-distance links.
For previous Fabric OS versions that support this option, it was disabled by default.
To avoid inconsistency in the fabric, make sure that this value is enabled on both
ends of the link. To connect to switches running Fabric OS versions earlier than
v4.0.2 and v3.0.2c, make sure that VC translation link initialization is disabled
because these versions do not support it.
<Mod #> 25
Revision 0512
Use the commands listed above to change the port parameters.
The portcfgshow command shows detailed port configuration of every port on the
switch. Use this command to analyze port configurations and to compare extended
fabric and ISL R_RDY mode configurations. Both extended fabric and ISL R_RDY mode
configurations must be identical on both sides of the ISL.
Revision 0512
TS300
6 26
Switch1:admin> portcfgshow
Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-----------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+--
Speed AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN
Fill Word 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AL_PA Offset 13 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Trunk Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON
Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
VC Link Init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked L_Port .. .. .. .. .. .. ON .. .. .. .. .. .. .. .. ..
Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
RSCN Suppressed .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Persistent Disable.. ON ON .. .. .. .. .. .. .. .. .. .. .. .. ..
NPIV capability ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON
Revision 0512
TS300
6 27
QOS E_Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON
EX Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Mirror Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Rate Limit .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Credit Recovery ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON
Fport Buffers .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Port Auto Disable .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

<Mod #> 27
Revision 0512
28
TS300
Revision 0512
Revision 0512
TS300
6 29
In a three-switch fabric, here is the fabricshow command output from one of the switches. The "<"
indicates which switch is the principal switch in the fabric.
sw5:admin> fabricshow
Switch ID Worldwide Name Enet IP Addr FC IP Addr Name
--------------------------------------------------------------------------
34: fffc22 10:00:00:60:69:00:06:56 192.168.64.59 192.168.65.59 >"sw5"
38: fffc26 10:00:00:60:69:00:02:0b 192.168.64.180 192.168.65.180 "sw180"
As there are only two switches shown, there could be a domain ID conflict.
Note: It is recommended that you set a unique domain ID for every switch in the fabric. On newer
switches you may not get a fabric segmentation error if two switches have the same domain ID, one of the
switches will take the next available domain ID in the fabric. However, for planning and documentation
accuracy, you may want to set the domain IDs.
Switch1:admin> switchshow
switchName: Switch1
Area Port Media Speed State
==============================
0 0 id N4 Online F-Port 10:00:00:00:c9:53:c6:c5
1 1 id N2 Online E-Port segmented, (domain overlap) Trunk
master)
Insistent Domain ID mode Required to be enabled for FICON environments, recommended for HP-UX
and AIX environments. Consider enabling this as a best practice when defining <domain, port>, <domain,
area> or <domain, port index> zoning. This mode enables a flag for the domain ID, so that the current
domain setting for the switch is insistent: that is, remains the same over switch reboots, power cycles, CP
failovers, firmware downloads, and fabric reconfigurations. If a switch does not get the selected insistent
domain ID during a fabric reconfiguration, it segments itself out of the fabric.

Revision 0512
TS300
6 30
In a three-switch fabric, here is the fabricshow command output from one of the switches. The "<"
indicates which switch is the principal switch in the fabric.
sw5:admin> fabricshow
Switch ID Worldwide Name Enet IP Addr FC IP Addr Name
--------------------------------------------------------------------------
34: fffc22 10:00:00:60:69:00:06:56 192.168.64.59 192.168.65.59 >"sw5"
38: fffc26 10:00:00:60:69:00:02:0b 192.168.64.180 192.168.65.180 "sw180"
As there are only two switches shown, there could be a domain ID conflict.
Note: It is recommended that you set a unique domain ID for every switch in the fabric. On newer
switches you may not get a fabric segmentation error if two switches have the same domain ID, one of the
switches will take the next available domain ID in the fabric. However, for planning and documentation
accuracy, you may want to set the domain IDs.
Switch1:admin> switchshow
switchName: Switch1
Area Port Media Speed State
==============================
0 0 id N4 Online F-Port 10:00:00:00:c9:53:c6:c5
1 1 id N2 Online E-Port segmented, (domain overlap) Trunk
master)
Insistent Domain ID mode Required to be enabled for FICON environments, recommended for HP-UX
and AIX environments. Consider enabling this as a best practice when defining <domain, port>, <domain,
area> or <domain, port index> zoning. This mode enables a flag for the domain ID, so that the current
domain setting for the switch is insistent: that is, remains the same over switch reboots, power cycles, CP
failovers, firmware downloads, and fabric reconfigurations. If a switch does not get the selected insistent
domain ID during a fabric reconfiguration, it segments itself out of the fabric.

Revision 0512
TS300
6 31
32
TS300
Revision 0512
Revision 0512
TS300
6 33
See the Fabric OS Command Reference Guide for additional information about
commands that will help resolve the segmentation issue:
fddcfg
secpolicyabort
secpolicyadd
secpolicycreate
secpolicydelete
secpolicydump
secpolicyremove
secpolicysave
secpolicyshow
There are RASLog error messages (errshow) that indicate ACL conflicts.

Revision 0512
TS300
6 34
secpolicycreate "name"[, "member[;member...]"]; the name for an SCC
policy must be SCC_POLICY.
SCC_POLICY requires members to be specified as WWN strings, domains, or switch
names. If domain or switch names are used, the switches associated must be present in
the fabric or the command fails. To add all switches in the current fabric as members of
the SCC_POLICY, enter an asterisk (*) as the member value.

Revision 0512
TS300
6 35
Footnote 1: The B5100 shows no light because the B300 has detected the security
violation and disabled its transmitter.
Revision 0512
TS300
6 36
Now each switch's SCC policy contains the other switch's WWN; the Brocade 300
additionally contains its own WWN:
Switch1:admin> secpolicyshow
____________________________________________________
ACTIVE POLICY SET
SCC_POLICY
WWN DId swName
--------------------------------------------------
10:00:00:05:1e:0a:85:05 2 Switch1
10:00:00:05:1e:0c:dc:72 98 Switch2
____________________________________________________
DEFINED POLICY SET
SCC_POLICY
WWN DId swName
--------------------------------------------------
10:00:00:05:1e:0a:85:05 2 Switch1
10:00:00:05:1e:0c:dc:72 98 Switch2
<Truncated Output>
Revision 0512
TS300
6 37
The Fabric Data Distribution (FDD) ACL policy is viewed and configured with the fddcfg
command. Use the fddcfg --showall command to determine fabric-wide
consistency policy. Use the fddcfg --fabwideset and the secpolicycreate
and secpolicyadd commands to resolve the conflict.
Switch1:admin> fddcfg --showall
Local Switch Configuration for all Databases:-
DATABASE - Accept/Reject
---------------------------------
SCC - accept
DCC - accept
PWD - accept
FCS - accept
AUTH - accept
IPFILTER - accept
Fabric Wide Consistency Policy:- ""
When the fabric-wide consistency policy is set to Tolerant and the ACL SCC or DCC
databases are different, an error log entry made and ACL distribution must be done
manually. This scenario still requires the SCC policies to contain connecting switch
WWNs, domain ID, and/or switch names.
Use the secpolicyshow command in connecting fabrics to check the contents of SCC
Revision 0512
TS300
6 38
and DCC policies. Use the secpolicycreate/add commands followed by the
secpolicyactivate command in connecting fabrics to resolve an SCC or DCC
conflicts. Use the distribute command to manually get the contents of one
switch's SCC and DCC policies to another switch.
<Mod #> 38
Revision 0512
39
TS300
Revision 0512
Revision 0512
TS300
6 40
Revision 0512
TS300
6 41
Revision 0512
TS300
6 42
Switch1:admin> trunkdebug
trunkdebug: area_number1 area_number2
Revision 0512
TS300
6 43
Additional command output examples:

Trunking license not installed or deleted
Switch1:admin> trunkdebug 8 9
Trunking license required

If the port speed is locked to 1 Gbit/sec the trunk will not come online
Switch1:admin> trunkdebug 8 9
port 8 and port 9 speed is not 2G, 4G or 8G

Revision 0512
TS300
6 44
45
TS300
Revision 0512
WDM Wavelength Division Multiplexing
TDM Time Division Multiplexing
Revision 0512
6 46
Revision 0512
6 47
Footnote 1:
r10-st16-b51:admin> portbuffershow
User Port Lx Max/Resv Buffer Needed Link Remaining
Port Type Mode Buffers Usage Buffers Distance Buffers
---- ---- ---- ------- ------ ------- --------- ----------
<truncated output>
38 E LE 46 40 40 10km
39 E LE 46 40 40 10km 1616

Revision 0512
6 48
Revision 0512
6 49
Revision 0512
6 50
51
TS300
Revision 0512
Footnote 1: The D_Port is not part of the fabric:
No routes set
No switch control frames flow through
No device data traffic flows through
No impact on the fabric operations or traffic though other ports
Complete isolation from the fabrics at both ends of link

Basic Troubleshooting
10 52
Revision 0512
CFA 200
10 53
Revision 0512
CFA 200
Footnote 1: Distance measurement accuracy:
5 meters for 16G SFP

10 54
Revision 0512
CFA 200
Footnote 1: Distance measurement accuracy:

10 55
Revision 0512
CFA 200
Here is an example of a D_Port test running from port 0 to port 1:

From the switchshow output:

<truncated output>
Index Port Address Media Speed State Proto
==============================================
0 0 020000 id N16 Online FC D-Port Loopback->Port 0
1 1 020100 id N16 In_Sync FC D-Port
<truncated output>

From the portdporttest --show 0 output:

<truncated output>
Status: PASSED
==========================================================================
Test Start time Result EST(secs) Comments
==========================================================================
Electrical loopback 09:23:09 PASSED -- --------
Optical loopback -------- SKIPPED -- --------
Link traffic test 09:24:14 PASSED -- --------
==========================================================================
Roundtrip link latency: 990 nano-seconds
Estimated cable distance: 7 meters

10 56
Revision 0512
CFA 200
10 57
Revision 0512
CFA 200
Footnote 1: Not supported on 8 Gbps SFPs. If a port with a 16 Gbps SFP is hard set to 8
Gbps the test will run at 8 Gbps speed.

10 58
Revision 0512
CFA 200
No High Availability (HA) support for D_Ports, the D_Port test may be restarted manually
after HA failover. Use bcu on HBA to configure D-port.

10 59
Revision 0512
CFA 200
Footnote 1: When a port is enabled as a D_Port, the caution message below is
displayed:
SwitchA:admin> portcfgdport --enable 1
Caution: D-port functionality is only available on 16G-
capable platforms with Brocade branded 16G SFPs and 10G FC
SFPs. Electrical and Optical loopback tests are supported
only for Brocade branded 16G SFPs.
Syntax:
portcfgdport --enable <port_range>
portcfgdport --disable <port_range>
portcfgdport help
Specifying port ranges: port_range -- Specifies a set of
ports as a list, range or wildcard: examples: "10/6-9" or
"10/6, 9" or "0-31" or "0 1 2 3" or "*")
Actions:
--enable -- Configure the port as D_Port
--disable -- Disable the D_Port configuration
--help -- Displays usage information
10 60
Revision 0512
CFA 200
Test can be manually restarted with CLI
portdporttest --start <slot/port>

10 61
Revision 0512
CFA 200
Looking at switch B shows the following results:

SwitchB:admin> portdporttest --show 2

D_Port Information:
===================
Port: 1
Remote WWNN: 10:00:00:05:33:13:2f:b4
Remote port: 2
Mode: Automatic
Start time: Fri Mar 11 01:41:55 2011
End time: Fri Mar 11 01:43:21 2011
Status: PASS
========================================================================
Test Start time Result EST(secs) Comments
========================================================================
Electrical loopback 01:42:11 PASS -- --------
Optical loopback 01:42:14 RESPONDER -- See remote
Link traffic test 01:43:10 RESPONDER -- See remote
========================================================================
Roundtrip link latency: 1108 nano-seconds
Estimated cable distance: 20 meters

10 62
Revision 0512
CFA 200
10 63
Revision 0512
CFA 200
The portcfgshow command can be used to see if a port is configured as a D_Port

SwitchA:admin> portcfgshow

Ports of Slot 0 24 25 26
----------------------+---+---+---
Octet Speed Combo 1 1 1
Speed AN AN AN
AL_PA Offset 13 .. .. ..
Trunk Port ON ON ON
Long Distance .. .. ..
.....
.....
Mirror Port .. .. ..
Rate Limit .. .. ..
Credit Recovery ON ON ON
Fport Buffers .. .. ..
Port Auto Disable .. .. ..
CSCTL mode .. .. ..
D_Port mode ON OFF ON

<truncated output>
10 64
Revision 0512
CFA 200
10 65
Revision 0512
CFA 200
10 66
Revision 0512
CFA 200
If the port being tested is an ISL that is currently functional, Network Advisor performs
the following:
Disable the ports
Put the ports into D_Port mode
Enable the ports
Run the tests
Disable the ports
De-configure the ports as a D_Ports
Enable the ports which should restore the ISL connection

Network Advisor does not display the cable distance or round trip latency.

10 67
Revision 0512
CFA 200
The master log is located at the bottom of the main Network Advisor screen:

Master log
10 68
Revision 0512
CFA 200
69
TS300
Revision 0512
Revision 0512
TS300
6 70
Revision 0512
TS300
6 71
Revision 0512
TS300
6 72
73
TS300
Revision 0512
74
TS300
Revision 0512
In case of a FCIP tunnel configuration mismatch, an error log message is written.
To configure fastwrite, add the -f option to the portcfg fciptunnel create command
switch1:admin> portcfg fciptunnel ge0 create 1 192.168.0.20
192.168.0.30 -f
To configure tape pipelining, add the -f -t options to the portcfg fciptunnel
create command
switch2:admin> portcfg fciptunnel ge0 create 2 192.168.0.21
192.168.0.31 f -t
The portshow command output includes the fastwrite and tape pipelining settings.

Footnote 1: These features only allow one FCIP tunnel per GbE port
Revision 0512
TS300
6 75
Confirm that the IP tunnel is configured correctly
Use the portshow fciptunnel command to check FCIP configuration
switch1:admin> portshow fciptunnel ge1 all
Port: ge1
-------------------------------------------
Tunnel ID 0
Tunnel Description Not Configured
Remote IP Addr 20.24.60.164
Local IP Addr 20.23.70.177
Remote WWN Not Configured
Local WWN 10:00:00:05:1e:37:0d:59
Compression off
Fastwrite off
Tape Pipelining off
Committed Rate 1000000 Kbps (1.000000 Gbps)
SACK on
Min Retransmit Time 100
Keepalive Timeout 10
Max Retransmissions 8
VC QoS Mapping off
DSCP Marking (Control): 0, DSCP Marking (Data): 0
VLAN Tagging Not Configured
TCP Byte Streaming off
Status : Active
Connected Count: 2
Uptime 31 seconds

Confirm the GE port is online
switch1:admin> portshow ge1
Eth Mac Address: 00.05.1e.37.93.06
Port State: 1 Online
Revision 0512
7 76
Port Phys: 6 In_Sync
Port Flags: 0x3 PRESENT ACTIVE
Port Speed: 1G

<Mod #> 76
Revision 0512

Confirm that the IP configuration is correct on both endpoints
switch1:admin> portshow ipif ge1
Port: ge1
Interface IP Address NetMask MTU
----------------------------------------------------------
0 11.1.1.1 255.255.255.0 1500

Revision 0512
7 77
Revision 0512
TS300
6 78
Footnote1: The default route is the default gateway used by the FCIP tunnel to get
outside of the local IP subnet.
Revision 0512
TS300
6 79
Footnote1: Use the portcmd --ipperf command to determine the link bandwidth
available for the tunnel
On local side:
portcmd --ipperf <slot/GBPort> -s <local IP> -d
<remote IP> -R
On Remote side:
portcmd --ipperf <slot/GBPort> -s <local IP> -d
<remote IP> -S
Command must be run from both endpoints simultaneously
Allow ipperf to run for at least three minutes
The last 30 seconds of data will indicate good recommended commit rates from the
S (remote) side
Repeat the test in the opposite direction to get throughput

In the ipperf command used above the s option represents the source IP and the d
option represents the destination IP of the remote switch.
The S switch is used to specify source mode for the ipperf connection. The source end-
point generates a traffic stream and reports the end-to-end bandwidth from this end-
Revision 0512
TS300
6 80
point toward the receiver sink point.
The R switch specifies sink mode to accept the new connection from the end-point
with the S flag specified. No end-to-end data is reported for the sink (S) endpoint.

<Mod #> 80
Revision 0512
Revision 0512
TS300
6 81
82
TS300
Revision 0512
Revision 0512
6 83
Revision 0512
6 84
Footnote1: For a complete list of interop mode requirements refer to the Fabric OS 6.2
Admin Guide and Release Note.
Revision 0512
6 85
Footnote1: If only i10K M-EOS switches are used domain ID 1-239 can be used.
Revision 0512
6 86
Revision 0512
6 87
Revision 0512
7 88
Performance TS300
Revision Alpha
7 1
TS300 Performance
Revision Alpha
7 2
TS300 Performance
Revision Alpha
7 3
Performance TS300
Revision Alpha
7 4
Note: When dealing with FCIP congestion issues the concept is the same with regards
to too much data being sent over the link. On these IP links, look for re-transmission
and slow starts. Suggest slowing down the amount of data being put on the IP pipe until
the re-transmissions and slow starts go away and adjust and monitor the IP port speeds
to find maximum performance. Other things to check would be window size, MTU size,
and compression.

TS300 Performance
Revision Alpha
7 5
Footnote 1: ISL VC_RDY flow control mode is not used when M-EOS interoperability is enabled on a Fabric OS switch.
M-EOS switches do not support VC_RDY flow control mode. VC_RDY flow control is also not used for some long
distance links depending on long distance settings.
Footnote 2: The number of VCs used and amount of credit assigned will change depending on long distance and QoS
settings.
The default ISL setting is 8 VCs sharing 26 buffer credits with VC0 used for class F traffic receiving 4 credits, VC 2-5
used for data frames each receiving 5 credits each and VC 6-7 used for broadcast/multicast each receiving1 credit
each. For 8 Gbit/sec switches using QoS 16 VCs are used with VC 0 used for class F traffic receiving 3 credits, VC 2-5
medium priority data traffic receiving 2 each, VC 8-14
low/high priority traffic receiving 2 each and VC 6-7
broadcast/multicast traffic each receiving 1.

TS300 Performance
Revision Alpha
7 6
Footnote 1: Each switch port always uses a set PID based on the switch port number
unless portswap has been used.
Footnote 2: When QoS is enabled the VC selected is based on the traffic priority. See
notes on prior slide.

Footnote 3: The graphic below illustrates traffic distributed across all VCs to remove the
VC congestion.

TS300 Performance
Revision Alpha
67_congestion-VCs.png
7 7
TS300 Performance
Revision Alpha
7 8
Footnote 1: In this example the hosts are pushing their frames to an ISL at a greater
bandwidth then the ISL can handle. Thus the ISL VC credit pool is continuously backed
up from the central credit pool that is feeding it. Eventually the frames are TX through
ISL, but the link is congested never the less, causing some of the host behind it to hold
frame beyond 500 ms and dropping, if credits are not available to receive from behind
because its oversubscribed. This is know as high I/O credit starvation.
Performance
Revision 0609
TS300
7 9
TS300 Performance
Revision Alpha
7 10
Footnote 1: APM Advanced Performance Monitor
Performance
Revision Alpha
TS300
Congestion/Slow Drain Commands and Tools
portperf
show
Use to monitor I/O (per port)
portstat
sshow
Use to monitor buffer credit to zero
counter and class 3 frame discards
Fabric
Watch
Use to monitor I/O (per port)
APM Use to monitor I/O (end-to-end)
SAN Health
Provide network diagram and
monitor I/O over time
DCFM
Provide network diagram, event log
and monitor I/O over time
7 11
Performance TS300
Revision Alpha
7 12
Footnote 1: See next page.
Footnote 2: A long distance link may have the correct number of credits for the distance
but due to small frames sizes used may still be unable to keep the link full. Mainframe
environments require more buffer credits to keep the pipe full. Mainframe uses 800
byte frames thus requiring approximately 3 times the number of buffer credits as a
device using a frame size of 2112 bytes.

Note: The above table is an approximation and assumes 2k payload size. For smaller
payload size, increase the number accordingly.
Footnote 3: See appendix for more information on this feature
TS300 Performance
Revision Alpha
Speed Credits/km Credits/50km Credits/100km
1 Gbps .5 25 50
2 Gbps 1 50 100
4 Gbps 2 100 200
8 Gbps 4 200 400
16 Gbps 8 400 800
7 13
Footnote 1: Buffer-to-Buffer Flow Control is flow control between adjacent ports in the
I/O path, A separate, independent pool of credits are used to manage Buffer-to-Buffer
Flow Control. Buffer-to-Buffer Flow Control works by a sending port using its available
credit supply and waiting to have the credits replenished by the port on the opposite
end of the link. These Buffer-to-Buffer credits (BB credits) are used by Class 2 and
Class 3 service and rely on the Fibre Channel Receiver-Ready (R_RDY) control word to
be sent by the receiving link port to the sender. An end node attached to a switch will
establish its BB credit during login to the fabric. A communicating partner attached
elsewhere on the switch will establish its own and most likely different BB credit value
to the director during its login process. Hence, BB credit has no end-to-end component.
The sender then decrements the BB credit by 1 for each R_RDY received. The initial
value of BB credit must be non-zero. The rate of frame transmission is regulated by the
receiving port based on the availability of buffers to hold received frames.
At first glance, it is readily apparent that this system may leave something to be
desired in terms of overall performance and efficiency. This is due to the time required
for frames to travel from the sending port to the receiving port and responses to return
from the receiving port back to the sending port. Now, consider that it takes light
approximately 5 nsec to propagate through 1 meter of optical fiber, or 50
microseconds to travel 10 km. This behavior becomes even less efficient and more of a
performance drag on faster links, longer-distance links, or when traveling through
complex topologies that contribute significant delivery latencies. So, to achieve the
higher performance while preventing the overrun of receive buffers, we need to use BB
credit values greater than one. If a sending port is allowed to send more than one
frame without having to wait for a response to each, performance can be improved.
This is referred to as frame streaming. As more credits (beyond one) are made
available, link utilization (and performance) will increase until link utilization reaches
100 percent. When the link is thus fully utilized, frames can be sent as rapidly as
allowed but additional credits will not help matters.
Revision Alpha
TS300 Performance
7 14
In this example insufficient buffer credits were assigned for the long distance link. This
same type of behavior will also take place had credits been lost due to R_RDYs or
VC_RDYs being lost.
TS300 Performance
Revision Alpha
7 15
TS300 Performance
Revision Alpha
7 16
Performance
Revision Alpha
TS300
7 17
Footnote 1: Brocade Fabric OS switches can detect link distance and assign the correct
number of buffer credits (LD long distance mode) or a specific distance can be selected
which will assign the correct buffer credits need.
If you double the speed or double the distance, you need to double the credits
available on the port
If the speed doubles, the maximum distance is cut in half

Note: The above table is an approximation and assumes 2k payload size. For smaller
payload size, increase the number accordingly.

Footnote 2: Requires condor 3 ASICs at both ends of the link running at 10 or 16 Gbps.
TS300 Performance
Revision Alpha
Speed Credits/km Credits/50km Credits/100km
1 Gbps .5 25 50
2 Gbps 1 50 100
4 Gbps 2 100 200
8 Gbps 4 200 400
16 Gbps 8 400 800
7 18
Performance TS300
Revision Alpha
7 19
Footnote 1: If you are using QoS or ISL trunking these will be VC_RDY.
Footnote 2: This is generally caused by a misbehaving or mis-configured device.
Footnote 3: In the graphic example above, if storage 1 port 0 was not returning R_RDYs
or was returning them slower than it should, frames would backup on port 0 causing
frames to backup on VC 2 which in turn would slow all traffic using that VC, in this case
Hosts 1, 5 and 6. Also, expect I/O to be lower then expected on storage 2 port 4 but with
no frame drop or buffer credit starvation as frames are not backed up but rather never
reaching this port.

If Storage 1 was returning R_RDYs too slowly for all four ports then frame backup would
occur on all VCs slowing traffic for all 7 hosts in this example.
TS300 Performance
Revision Alpha
7 20
TS300 Performance
Revision Alpha
7 21
Footnote 1. Even though slow performance may be seen on devices using the same VC
as the problem device, only the end device causing the problem will have class 3 frame
drop counter increasing. In this example the storage on the bottom is seeing low
utilization but not credit starvation.
Performance
Revision Alpha
TS300
7 22
Footnote 1: See appendix for more information
TS300 Performance
Revision Alpha
7 23
Performance TS300
Revision Alpha
7 24
Footnote 1: As stated earlier, encoding outside the frame errors can also cause R_RDYs
to be lost causing buffer credit starvation.

Footnote 2: Forward Error Correction can automatically fix some of these corrupted
frames. More information later in this module as well as the appendix of this module.

TS300 Performance
Revision Alpha
7 25
TS300 Performance
Revision Alpha
7 26
The benefits of bottleneck detection are they allow you to:
Prevent degradation of throughput in the fabric.
The bottleneck detection feature alerts you to the existence and locations of devices
that are causing latency. If you receive alerts for one or more F_Ports, use the CLI to
check whether these F_Ports have a history of bottlenecks.
Reduce the time it takes to troubleshoot network problems.
If you notice one or more applications slowing down, you can determine whether any
latency devices are attached to the fabric and where. You can use the CLI to display
a history of bottleneck conditions on a port. If the CLI shows above-threshold
bottleneck severity, you can narrow the problem down to device latency rather than
problems in the fabric.

Adaptive Networking: Fabric Profiling
Revision 0312
CFP 300
7 27
You configure bottleneck detection on a per-switch basis, with optional per-port
exclusions. Bottleneck detection is disabled by default. Best practice is to enable
bottleneck detection on all switches in the fabric, and leave it on to continuously gather
statistics.
High availability considerations for bottleneck detection
The bottleneck detection configuration is maintained across a failover or reboot;
however, bottleneck statistics collected are lost.
Upgrade and downgrade considerations for bottleneck detection
The bottleneck detection configuration is persistent across firmware upgrades and
downgrades. The sub-second latency criterion parameter settings are not preserved on
downgrade to firmware versions earlier than Fabric OS 7.0.0. If you downgrade and then
upgrade back to Fabric OS 7.0.0, the settings revert to their default values.
Trunking considerations for bottleneck detection
A trunk behaves like a single port. Both latency and congestion bottlenecks are reported
on the master port only, but apply to the entire trunk. For masterless trunking, if the
master port goes offline, the new master acquires all the configurations and bottleneck
history of the old master and continues with bottleneck detection on the trunk.
Revision 0312
CFP 300
7 28
Bottlenecks are also reported through RASlog alerts and SNMP traps. These two alerting
mechanisms are intertwined and cannot be independently turned on and off. You can
use the bottleneckmon command to specify alerting parameters for the following:
Whether alerts are to be sent when a bottleneck condition is detected
The size of the time window to look at when determining whether to alert
How many affected seconds are needed to generate the alert.
How long to stay quiet after an alert
Changing alerting parameters affects RASlog alerting as well as SNMP traps
For more information: Fabric OS Administrators Guide 53-1002148-01

Revision 0312
CFP 300
7 29
Bottleneck detection is enabled on a per switch basis. It is recommended that you
enable bottleneck detection on every switch in the fabric. If you later add additional
switches, including logical switches, to the fabric be sure to enable bottleneck detection
on those switches as well. When you enable bottleneck detection on a switch, the
settings are applied to all eligible ports on that switch. If ineligible ports later become
eligible or, in the case of a logical switch, if ports are moved to the logical switch,
bottleneck detection is automatically applied to those ports.

You can use the bottleneckmon --show command to display a history of
bottleneck conditions, for up to three hours. This command has several display options:
Display only latency bottlenecks, only congestion bottlenecks, or both combined.
Display bottleneck statistics for a single port, bottleneck statistics for all ports on
the switch, or a list of ports affected by bottleneck conditions.
Continuously update the displayed data with fresh data.
Revision 0312
CFP 300
7 30
Example of displaying the bottleneck history in 5-second windows over a period of 30
seconds.
In this example, the definition of bottlenecked ports is any port that had a bottleneck
occur during any second in the corresponding interval.

switch:admin> bottleneckmon --show -interval 5 -span 30
===========================================================
Wed Jan 13 18:54:35 UTC 2010
===========================================================
List of bottlenecked ports in most recent interval:
23
===========================================================
Number of From To bottlenecked ports
===========================================================
Jan 13 18:54:05 Jan 13 18:54:10 1
Jan 13 18:54:10 Jan 13 18:54:15 2
Jan 13 18:54:15 Jan 13 18:54:20 1
Jan 13 18:54:20 Jan 13 18:54:25 1
Revision 0312
CFP 300
7 31
Setting a threshold of 0.1 and a time window of 30 seconds specifies that an alert
should be sent when 10% of the one-second samples over any period of 30 seconds
were affected by bottleneck conditions. The -qtime option can be used to throttle alerts
by specifying the minimum number of seconds between consecutive alerts.

Syntax:
bottleneckmon --enable [-alert][-thresh threshold]
[-time window] [-qtime quiet_time]
[slot/]port_list [[slot/]port_list] ...
bottleneckmon --disable [slot/]port_list
[[slot/]port_list] ...
bottleneckmon --show [-interval interval_size]
[-span span_size] [slot/]port
bottleneckmon status
bottleneckmon --help

Revision 0312
CFP 300
7 32
Enter the bottleneckmon --status command to display the details of bottleneck detection
configuration for the switch, which includes the following:
Whether the feature is enabled
Switch-wide parameters
Per-port overrides, if any
Excluded ports
Example
switch:admin> bottleneckmon --status
Bottleneck detection - Enabled
==============================
Switch-wide sub-second latency bottleneck criterion:
====================================================
Time threshold - 0.800
Severity threshold - 50.000
Switch-wide alerting parameters:
============================
Revision 0312
CFP 300
7 33
Alerts - Yes
Latency threshold for alert - 0.100
Congestion threshold for alert - 0.800
Averaging time for alert - 300 seconds
Quiet time for alert - 300 seconds
Per-port overrides for sub-second latency bottleneck criterion:
==========================================================
Slot Port TimeThresh SevThresh
=========================================
0 3 0.500 100.000
0 4 0.600 50.000
0 5 0.700 20.000
Per-port overrides for alert parameters:
========================================
Slot Port Alerts? LatencyThresh CongestionThresh Time (s) QTime (s)
==========================================================
0 1 Y 0.990 0.900 3000 600
0 2 Y 0.990 0.900 4000 600
0 3 Y 0.990 0.900 4000 600
Excluded ports:
===============
Slot Port
============
0 2
0 3
0 4
Revision 0312
CFP 300
7 34
Enter the following command to set the alerting and sub-second latency criterion
parameters
switch:admin> bottleneckmon --config
Enter the following command to remove any port-specific alerting and sub-second
latency criterion parameters and revert to the switch-wide parameters
switch:admin> bottleneckmon --configclear
When you enable bottleneck detection, you can configure switch-wide alerting and sub-
second latency criterion parameters that apply to every port on the switch. After you
enable bottleneck detection, you can change the alerting parameters on the entire
switch or on individual ports. You can change the sub-second latency criterion
parameters on individual ports only. You can also
change the parameters on ports that have been excluded from bottleneck detection.
The alerting parameters indicate whether alerts are sent, and the threshold, time, and
quiet time options.
For a trunk, you can change the parameters only on the master port.
1. Connect to the switch and log in as admin.
2. Enter the bottleneckmon --config command to set the alerting and sub-
second latency criterion parameters.
Enter the bottleneckmon --configclear command to remove any port-specific
alerting and sub-second latency criterion parameters and revert to the switch-wide
parameters.

Revision 0312
CFP 300
7 35
The following example changes alerting parameters for the entire logical switch.
switch:admin> bottleneckmon --config -alert -lthresh .97 -cthresh .8 time 5000
switch:admin> bottleneckmon --status
Bottleneck detection - Enabled
==============================
Switch-wide sub-second latency bottleneck criterion:
====================================================
Revision 0312
CFP 300
7 36
Time threshold - 0.800
Severity threshold - 0.100
Switch-wide alerting parameters:
================================
Alerts - Yes
Latency threshold for alert - 0.970
Congestion threshold for alert - 0.800
Averaging time for alert - 5000 seconds
Quiet time for alert - 300 seconds
Per-port overrides for alert parameters:
========================================
Port Alerts? LatencyThresh CongestionThresh Time(s) QTime(s)
================================================================
1 N -- -- -- --
2 Y 0.990 0.900 4000 600
3 Y 0.990 0.900 4000 600
Excluded ports:
===============
Port
====
2
3
4
Revision 0312
CFP 300
7 37
Revision 0312
CFP 300
7 38
Revision 0312
CFP 300
7 39
Sw1:admin> portcfgfec --show 1
Forward Error Correction capable: YES
Forward Error Correction configured: ON
This command has the following operands:
Slot: On bladed systems only, specifies the slot number of the ports to be configured,
followed by a slash (/).
port [-Range]: Specifies a port or a port range, relative to the slot number on bladed
systems, for example, 5/17-29. Multiple port ranges are not supported with this
command. .
--enable: Enables the FEC flag on the specified ports.
--disable: Disables the FEC flag on the specified ports.
--show: Displays the FEC port configuration. The output displays the following
parameters:

Layer 2 latency of 800ns w/o FEC
2
(F_Port to F_Port)
Layer 2 latency of 1.2s with FEC (E_Port to E_Port)

PR208 Support Enhancements
Revision 0511
7 40
Provides a mechanism that allows configuring different thresholds for different type of
frames. Report events whenever a count for particular frame type crosses its respective
threshold

Revision 0511
7 41
Footnote 1: APM Advanced Performance Monitor
Revision 0511
7 42
Performance TS300
Revision Alpha
7 43
Footnote 1: In the example above, insufficient ISL bandwidth is causing congestion on
the ISLs slowing fabric performance. Had this been a slow draining device expect to see
normal to low utilization. For insufficient buffer credit expect to see low utilization. Also
note that congestion can occur in a single direction so link utilization may look normal
running as low as 50% utilization.

TS300 Performance
Revision Alpha
7 44
Footnote 1: The VC specific counters are not shown for Condor and GoldenEye ASICs.
This feature requires Condor2 and GoldenEye2 or later ASICs.
TS300 Performance
Revision Alpha
7 45
Footnote 1: In the example above, the buffer credit to zero counter are incrementing on
the ISL while the performance problem is occurring. This along with the high ISL
utilization on the prior slide helps confirms the ISLs are congested in both directions. To
further confirm this, the next step is to check for class 3 frame discards on the end
device ports.
Also note that Tx and Rx counters can be used to check for one way congestion. If Tx is
increasing at a significantly higher rate then Rx, this may be the direction of the
congestion.

Footnote 2: For VC congestion use the tim_txcrd_z_vc to help identify which VCs may be
congested. In the example above, all credit to zero instances are on VC 0 indicating this
port is probably long distance using R_RDY mode rather then VC_RDY.

TS300 Performance
Revision Alpha
7 46
Class 3 frame discard counters:
The er_c3_timeout counter is the number of class 3 frames discarded at the receiving
port due to timeout (platform and port specific). A timeout occurs when a frame is held
at an ASIC port longer then the switch hold time (default hold time 500ms).

The er_c3_dest_unreach counter is the number of class 3 frames discarded at the
receiving port due to destination unreachable (platform and port specific). A
destination becomes unreachable when it no longer exists in the fabric such as a
switch or port going offline while the frame is in-flight.

The er_other_discard counter is the number of other discards (platform and port
specific) on a receiving port. This counter is used to track class 3 frames dropped that
are not due to hold time or destination unreachable. For example a frame that is
dropped due to a zoning violation.
Buffer Credit counters:
The tim_txcrd_z counter is the number of times that the port was unable to transmit
frames because the transmit BB credit was zero. The purpose of this statistic is to
detect congestion or a slow drain device. This parameter is sampled at intervals of
2.5Us (microseconds), and the counter is incremented if the condition is true.

The tim_txcrd_z_vc counter is the number of times that the port was unable to
transmit frames because the transmit BB credit was zero for each of the port's 16
Virtual Channels (VC 0-15). The purpose of this statistic is to detect congestion or a
slow drain device. This parameter is sam-pled at intervals of 2.5Us (microseconds),
and the counter is incremented if the condition is true. Each sample represents 2.5Us
of time with zero Tx BB Credit. An increment of this counter means that the frames
could not be send to the attached device for 2.5Us, indicating degraded performance
(platform-and port-specific).

Revision Alpha
TS300 Performance
Congestion
Multiple
8 Gbit/sec
Hosts
8 Gbit/sec
Connection
4 Gbit/sec
Storage
8 Gbit/sec
Storage
ISL Congestion
High I/O
Credit Starvation
Frame Drop
VC Congestion
Low Utilization
Credit Starvation
7 47
r10-st16-b51:admin> portstatsshow 38
stat_wtx 3337133291 4-byte words transmitted
stat_wrx 3152940356 4-byte words received
stat_ftx 26927315 Frames transmitted
stat_frx 348368552 Frames received
stat_c2_frx 0 Class 2 frames received
stat_c3_frx 348368552 Class 3 frames received
stat_lc_rx 0 Link control frames received
stat_mc_rx 0 Multicast frames received
stat_mc_to 0 Multicast timeouts
stat_mc_tx 0 Multicast frames transmitted
tim_rdy_pri 0 Time R_RDY high priority
tim_txcrd_z 7806470 Time TX Credit Zero (2.5Us ticks)
tim_txcrd_z_vc 0- 3: 7806470 0 0 0
tim_txcrd_z_vc 4- 7: 0 0 0 0
tim_txcrd_z_vc 8-11: 0 0 0 0
tim_txcrd_z_vc 12-15: 0 0 0 0
er_enc_in 0 Encoding errors inside of frames
er_crc 0 Frames with CRC errors
er_trunc 0 Frames shorter than minimum
er_toolong 0 Frames longer than maximum
er_bad_eof 0 Frames with bad end-of-frame
er_enc_out 0 Encoding error outside of frames
er_bad_os 0 Invalid ordered set
er_c3_timeout 785 Class 3 frames discarded due to
timeout
er_c3_dest_unreach 0 Class 3 frames discarded due to
destination unreachable
er_other_discard 0 Other discards
<Truncated Output>
Revision Alpha
TS300 Performance
7 48
See notes on next page
TS300 Performance
Revision Alpha
7 49
portbuffershow [[slotnumber/]portnumber]

DESCRIPTION
Use this command to display the current long distance buffer information for the
ports in a port group. The port group can be specified by giving any port number in
that group. If no port is specified, then the long distance buffer information for all of
the port groups of the switch is displayed.

This command displays the following fields of the long distance buffer information:
Lx Mode Long distance mode.
L0 - link is not in long distance mode, LE - link is up to 10Km, LD distance
determined dynamically
LS - distance determined statically via user input
Max/Resv Buffers
The maximum or reserved number of buffers that are allocated to the port based on
the estimated distance (as defined by the desired_distance operand of the
portCfgLongDistance command). If the port is not configured in long distance mode,
certain systems may reserve buffers for the port. This field then shows the number
of buffers reserved for the port
Buffer Usage
The actual number of buffers allocated to the port. In LD mode, the number is
determined by two factors: the actual distance and the user specified desired
distance (as defined by the desired_distance operand of the portCfgLongDistance
command).
Needed Buffers
The number of buffers that are needed to utilize the port at full bandwidth
(depending on port speed configuration). If the number of Buffer Usage is less than
the number of Needed Buffers, the port is operating in the buffer limited mode.
Link Distance
For L0 (not in long distance mode), the command displays the fixed distance
based on port speed, for instance: 10 Km (1 Gbps), 5 Km (2 Gbps), 2 Km (4 Gbps),
or 1 Km (8 Gbps). For static long distance mode (LE), the fixed distance displays10
Km. For LD mode, the distance in kilometers displays as measured by timing the
return trip of a MARK primitive that is sent and then echoed back to the switch. LD
mode supports distances up to 500 Km. Distance measurement on a link longer
than 500 Km might not be accurate. If the connecting port does not support LD
mode, is shows "N/A".
Remaining Buffers
The remaining (unallocated and unreserved) buffers in a port group.

Revision Alpha
TS300 Performance
7 50
Footnote1: To export the performance reports, run a SAN Export and include the
performance reports as part of the exported data.
Performance
Revision Alpha
TS300
7 51
fcping [--number frames] [--length size] [--interval wait] [--
pattern pattern] [--bypasszone] [--quiet] [--help] source
destination
--number frames Number of frames to send.
--length size Size of data to send.
--interval wait Number of seconds between frames.
--pattern pattern Pattern data to fill frame.
--bypasszone Bypass zone check.
--quiet Quell diagnostic output.
pathinfo destination_switch [source_port [destination_port]] [-r]
[-t]
switch:admin> pathinfo 91
Target port is Embedded
Hop In Port Domain ID (Name) Out Port BW Cost
---------------------------------------------------------
0 E 9 (web226) 2 1G 1000
1 3 10 (web229) 8 1G 1000
TS300 Performance
Revision Alpha
7 52
Performance TS300
Revision Alpha
7 53
TS300 Performance
Revision Alpha
7 54
TS300 Performance
Revision Alpha
7 55
Footnote 1: An analysis of the network performance taken at the time of the problem,
such as a portperfshow, would be very helpful but this information is not provided in a
supportsave.

Footnote 2: The supportsave/supportshow uses the older fabstateshow
rather then fabriclog s but both are the same command output.
TS300 Performance
Revision Alpha
7 56
Based on the data we have, we can rule out :
Marginal links as porterrshow shows no physical layer errors such as CRCs or
encoding errors.
Insufficient buffer credits as there are no long distance links and no encoding errors
to cause lost buffer credit.

The remaining possibilities are:
Congestions
ISL or device port over-subscription
VC congestion
Slow draining device

TS300 Performance
Revision Alpha
7 57
TS300 Performance
Revision Alpha
7 58
TS300 Performance
Revision Alpha
7 59

All device and ISL port traffic is low to normal. If this was ISL or device port over-
subscription we should see ISL or device port utilization peaked at max or near max
utilization (remember portperfshow displays both the Tx and Rx totals for a port).

Hint, ISL traffic was low but the second ISL in the trunk group is still being used.
Trunking only uses the first link in the trunk until it is saturated, then the second, third,
fourth,.. ISL is needed. In this example ISL traffic is low to normal but the second ISL in
each trunk group is still being used. What could be causing this and why?

Note, B5100 ports 0-30 have hosts attached and B300 ports 0-7 have storage
attached.
TS300 Performance
Revision Alpha
7 60
TS300 Performance
Revision Alpha
7 61
TS300 Performance
Revision Alpha
7 62
TS300 Performance
Revision Alpha
7 63
We see tim_txcrd_z_vc counters increasing on all four B5100 ISL ports.
Remember these counter are Tx counters not Rx. All ISLs on the B5100 have a high
number of VC 4 Tx buffer credits at zero for longer then 2.5us waiting to receive a
VC_RDY.

TS300 Performance
Revision Alpha
7 64

TS300 Performance
Revision Alpha
7 65
We also see class 3 frame drop on all four B300 ISLs but none on the B5100

Again, we see tim_txcrd_z_vc counters increasing on all four B5100 ISL ports but
not on the B300. Remember these counter are Tx counters not Rx. All ISLs on the
B5100 have a high number of VC 4 Tx buffer credits at zero for longer then 2.5us waiting
to receive a VC_RDY.

This confirms class 3 frames going from the B5100 to the B300 are getting dropped
while frames from the B300 to the B5100 are not.

TS300 Performance
Revision Alpha
7 66
TS300 Performance
Revision Alpha
7 67
TS300 Performance
Revision Alpha
7 68
As only VC 4 is congested we only need to look at storage ports using VC 4 and the hosts
using those storage ports. Remember the VC assignment is based on destination PID.
TS300 Performance
Revision Alpha
7 69
TS300 Performance
Revision Alpha
7 70
TS300 Performance
Revision Alpha
Virtual Channel Selection
VC PID Area
Number
PID Area
Number
Binary
VC2 0 8 0000/1000
VC3 1 9 0001/1001
VC4 2 A 0010/1010
VC5 3 B 0011/1011
VC2 4 C 0100/1100
VC3 5 D 0101/1101
VC4 6 E 0110/1110
VC5 7 F 0111/1111
7 71
There is high BB credit to zero on storage port 6 but not port 2.
TS300 Performance
Revision Alpha
7 72
TS300 Performance
Revision Alpha
7 73

TS300 Performance
Revision Alpha
7 74
TS300 Performance
Revision Alpha
7 75
Performance TS300
Revision Alpha
7 76
Performance TS300
Revision Alpha
7 77
Device Connectivity
Revision 0213
SAN-TS 300
7 78
Device Connectivity
Revision 0213
SAN-TS 300
7 79
Footnote 1:
ABTS: Abort command
BA_ACC: Abort Accept command
BA_RJT: Abort Reject command
Frame monitor configuration is saved/persistent across reboots.
Trunk Master will monitor the data traffic for the entire trunk. For F_Port and E_Ports
trunks, the monitor is set only on the master port of the trunk. If the master changes,
the monitor automatically moves to the new master port. If a monitor is installed on a
port that later becomes a slave port when a trunk comes up, the monitor automatically
moves to the master port of the trunk.

Device Connectivity
Revision 0213
SAN-TS 300
7 80
Device Connectivity
Revision 0213
SAN-TS 300
7 81
Pre-defined frame types include the following:
ABTS: Specifies a frame of type ABTS (Abort Sequence Basic Link Service
command) with a bit pattern of "4,0xff,0x81;12,0xff,0x00".
BA_ACC: Specifies a frame of type BA_ACC (Abort Accept) with a bit pattern of
0xff,0x84;12,0xff,0x00".
BA_RJT: Specifies a frame of type BA_RJT (Abort Reject) with a bit pattern of ().
IP Specifies a frame of type IP with a bit pattern of 12,0xFF,0x05.
SCSI: Specifies a frame of type SCSI with a bit pattern of 12,0xFF,0x08.
SCSI_READ: Specifies a frame of type SCSI Read with a bit pattern of
12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x08,0x28.
SCSI_WRITE: Specifies a frame of type SCSI Write with a bit pattern of
12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x0A,0x2A.
SCSI_RD_WR: Specifies a frame of type SCSI Read or Write with a bit pattern of
12,0xFF,0x08;4,0xFF,0x06;40,0xFF, 0x08,0x28,0x0A,0x2A.
SCSI2_RESERVE: Specifies a frame of type SCSI-2 Reserve with a bit pattern of
12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x16,0x56.
12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x5F;41,0xFF,0x01.

Device Connectivity
Revision 0213
SAN-TS 300
7 82

Device Connectivity
Revision 0213
SAN-TS 300
7 83
In this bit pattern example: 40,0xFF,0x08,0x28 the 0x08 and 0x28 are SCSI read
commands

Footnote 1: When bit patterns are chained together like this the ; acts like an and. All
three bit patterns must match to get an match.
Device Connectivity
Revision 0213
SAN-TS 300
7 84

Offset Fibre Channel header
4-7 R_CTL Destination_ID
8-11 CX_CTL Source_ID
12-15 Type F_CTL
16-19 SEQ_ID DF_CT
L
Sequence Count
20-23 OX_ID RX_ID
24-27 Offset
28-31 Start of Payload
Device Connectivity
Revision 0213
SAN-TS 300
7 85
Footnote 1: If you convert the bitmask 0x0f to binary 00001111 only the last 4 bits
would be checked. If had a bit mask of 0xF1 that would be 11110001 in binary only only
bits 0, 4-7 would be checked for a match.
Device Connectivity
Revision 0213
SAN-TS 300
7 86
In SOFx frames the offset is specified as 0x0; value is specified as one of the following.
For example, the value of 0x6 matches frames of type SOFi3:
0 SOFf
1 SOFc1
2 SOFi1
3 SOFn1
4 SOFi2
5 SOFn2
6 SOFi3
7 SOFn3
Device Connectivity
Revision 0213
SAN-TS 300
7 87
Device Connectivity
Revision 0213
SAN-TS 300
7 88
Refer to the Fabric Watch Administrators Guide and the Fabric OS Command Reference
Manual for more information on these commands.
Pre-defined frame types include the following:
ABTS: Specifies a frame of type ABTS (Abort Sequence Basic Link Service
command) with a bit pattern of "4,0xff,0x81;12,0xff,0x00;12".
BA_ACC: Specifies a frame of type BA_ACC (Abort Accept) with a bit pattern of
0xff,0x84;12,0xff,0x00".
BA_RJT: Specifies a frame of type BA_RJT (Abort Reject) with a bit pattern of ().
IP Specifies a frame of type IP with a bit pattern of 12,0xFF,0x05.
SCSI: Specifies a frame of type SCSI with a bit pattern of 12,0xFF,0x08.
SCSI_READ: Specifies a frame of type SCSI Read with a bit pattern of
12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x08,0x28.
SCSI_WRITE: Specifies a frame of type SCSI Write with a bit pattern of
12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x0A,0x2A.
SCSI_RD_WR: Specifies a frame of type SCSI Read or Write with a bit pattern of
12,0xFF,0x08;4,0xFF,0x06;40,0xFF, 0x08,0x28,0x0A,0x2A.
12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x16,0x56.
12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x5F;41,0xFF,0x01.
Device Connectivity
Revision 0213
SAN-TS 300
7 89
Footnote 1: The physical port number to index number mapping can be found in the
switchshow output.
Refer to the Fabric Watch Administrators Guide for more information.

Device Connectivity
Revision 0213
SAN-TS 300
7 90
When you set a time base to a value other than none, there are two main points to
remember when configuring events:
Fabric Watch triggers an event only if the difference in the data value exceeds the
preset threshold boundary limit.
Even if the current data value exceeds the threshold, Fabric Watch does not trigger
an event if the rate of change is below the threshold limit.

Device Connectivity
Revision 0213
SAN-TS 300
7 91
Device Connectivity
Revision 0213
SAN-TS 300
7 92
Revision 0511
7 93
See appendix for more information on Frame Patterns.
Revision 0511
7 94
Not shown but at the bottom of this screen is an OK button, once you assign
Revision 0511
7 95
If the monitor is not there, click on switch view to create the monitor.
Revision 0511
7 96
Revision 0511
7 97
Revision Alpha
TS300 Performance
7 98

Content

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Content

Загружено:

Авторское право:

Доступные форматы

SAN-TS 300

Вам также может понравиться