San Unit III

Lecturer Notes
Storage Area Network

( Unit - III)
Ms. Madhu G. Mulge

( M-Tech CSE )
M.B.E.'S College of Engineering, Ambejogai.
ME - CSE [SAN- Unit III] 1

Fibre Channel Products
Fibre Channel SANS are composed of various hardware and software products, which
at the same time complement one another and compete for space in the storage solutions
market.
The manufacturers of Fibre Channel products go to market along various paths,

including large Original Equipment Manufacturer (OEM) contracts with solutions
providers (Dell, HP, IBM, Sun, and so on), arrangements with Value-Added Reseller
(VAR) and reseller channels, and direct sales to corporate accounts.
Closer cooperation between noncompeting vendors—such as manufacturers of HBAs,

switches, storage products, and software—has promoted lower-level interoperability
between the essential components of a SAN interconnection.
Interoperability of Fibre Channel products has also been encouraged by the Fibre
Channel Industry Association (FCIA) and the Storage Networking Industry Association
(SNIA).

The spectrum of hardware components:
Transceivers to enterprise-class storage arrays
Transceivers
HBAs
loop hubs
fabrics inter connect Fibre Channel-enabled RAIDs
JBODs
Fibre Channel-to-SCSI bridges
Native Fibre Channel tape subsystems
Software components include:

Device drivers that are supplied with HBAs
Management software for hubs and switches
File and volume management applications
Storage resource management
Failover software
SAN-ready tape backup applications

Gigabit Interface Converters (GBIC)
GBICs, are still commonly used for Fibre Channel applications.
GBICs are removable media and so provide flexibility for configuring either optical or
copper cabling.
GBICs are removable media and so provide flexibility for configuring either optical or
copper cabling.
The selection of GBICs for SAN interconnection is as important as the selection of the
proper switch or hub, because without signal integrity at the link level, no higher
functions can be performed.
GBICs are manufactured to a de facto standard form factor, and that enables use of
various vendors' transceivers in the same device chassis.
An optical GBIC provides an SC connector for multimode or single-mode fiber-optic

cabling on one end, and the electrical connector to plug in to the device port on the
other.

Fig: Optical GBIC with standard SC connector
First-generation optical GBICs used edge-emitting CD lasers—that is, laser components normally
used in consumer CD products.
For high-speed data transport, only the highest-quality CD lasers can be used.
Wafer manufacturing techniques for CD lasers require separation of the discrete components for
testing to ensure quality, and that contributes to manufacturing overhead and cost.
In addition, compared with other kinds of lasers, CD lasers consume more power, radiate more heat,
and are not tolerable to loss of calibration over time.
These inherent problems of CD technology have encouraged the development of alternative laser
products—in particular, Vertical Cavity Surface Emitting Lasers, or VCSELs.
VCSELs also consume less power, radiate less heat, and maintain calibration better than
their CD cousins.
VCSELs can be tested at the wafer, as opposed to the discrete component level,
manufacturing and quality testing can be performed more efficiently.
GBIC manufacturers have converted from CD lasers to VCSELs, and that helps to
ensure that these physical-layer components will maintain calibration and therefore data
integrity at the system level.
Another advance in GBIC technology is the development of serial ID functionality.
Serial ID allows a supporting enclosure—such as a hub, switch, or HBA—to query the

GBIC for inventory or status information.
Inventory information includes manufacturer, date of manufacture, serial number, speed,

media and lengths supported, and other useful tracking information.

Depending on the vendor, status information may include current power consumption,
output, and diagnostic data.
Serial ID relies on the supporting enclosure to solicit this information.
This implies that the hub, switch, or HBA can query the GBIC and report its findings to a
management applications.
Concurrent with the development of 2Gbps Fibre Channel, vendors have achieved higher
port density in the same chassis footprint by substituting new small form factor
transceivers in place of the traditional GBIC.
Small form factor transceivers provide the same functionality as GBICs but in a more
compact package.
Small form factor transceivers may be fixed (permanently attached to the device port).
The pluggable variety is typically referred to as SFP (small form factor pluggable).

It is also possible to mix small form factor transceivers and GBICs at each end of the
cabling plant so that storage devices using GBICs can be attached to SAN switches using
the more compact SFP connectors.
Fig: Small form factor pluggable transceivers

Host Bus Adapters
Fibre Channel host bus adapters provide the interface between the internal bus
architecture of the server or workstation and the external storage network.
HBAs are available for various bus types and various physical connections to the
transport.
Most commonly employed are HBAs with Peripheral Component Interface (PCI) bus
interfaces and shortwave fiber-optic transceivers.
 HBAs are supplied with software drivers to support various operating systems and
upper-layer protocols as well as support for private loop, public loop, and fabric
topologies.
Although most HBAs have a single transceiver for connection to the SAN, some dual-
ported and even quad-ported HBAs exist.
Multi-ported HBAs save bus slots by aggregating N_Ports, but they also pose a potential
single point of failure should the HBA hang.
 Most HBAs offer a single Fibre Channel port, requiring that you install additional HBAs
if you desire multiple links to the same or different SAN segment.
Fig: Host bus adapter functional diagram

The HBA embodies all four Fibre Channel layers, FC-0 through FC-4. At the FC-0 layer,
the HBA has transmit and receive functions to physically connect to the link.
For fiber optics, this connector may be a standard GBIC, a fixed transceiver, or a small
form factor transceiver.
For copper interface, the connector may be DB-9 with four active wires or the high-
speed serial direct connect (HSSDC) form factor.
 Behind the link interface, clock and data recovery (CDR) circuitry,
serializing/deserializing functions, and an elasticity buffer and retiming circuit enable the
receipt and transmission of gigabit serial data.
The FC-1 transmission protocol requirements are met with on-board 8b/10b encoding
logic for outbound data and decoding logic for incoming and error monitoring functions.
For loop-capable HBAs, the FC-1 functions must be followed by a loop port state
machine (LPSM) circuit, typically included with other features in a single chip.
The HBA provides the signaling protocol for frame segmentation and reassembly, class
of service, and credit algorithms as well as link services for fabric port login required by
FC-2.
The FC-4 upperlayer protocol mapping level, most HBAs provide SCSI-3 software
drivers for NT, UNIX, Solaris, or Macintosh operating systems.
The number of functions consolidated into one or more chips is vendor-dependent, but
current HBAs are ASIC-based and collapse most functions into an integrated architecture.
The design of HBAs typically includes Flash ROM for microcode.
This is an important feature, because compatibility issues and the need for microcode
fixes are facts of life for all network products.
Having a means to upgrade microcode via a software utility is very useful and extends
the life of the product.
Device drivers for specific operating systems are also upgradeable.
In most cases, installing new microcode or device drivers requires taking an HBA off
line, and that is additional incentive for redundant configurations in high-availability
networks.
The SCSI-3 device driver supplied by the HBA vendor is responsible for mapping Fibre
Channel storage resources to the SCSI bus/target/LUN triad required by the operating
system
ME (OS).
- CSE [SAN- Unit III] 12
These SCSI address assignments may be configured by OS utilities or by a graphical
interface supplied by the manufacturer (or both).
Because Fibre Channel addresses are self-configuring, the mapping between port
addresses or AL_PAs (which may change) and the upper-layer SCSI device.
Device drivers for IP over Fibre Channel must perform a similar function via the
Address Resolution Protocol (ARP).
When upper-layer applications send data addressed to an IP destination, the HBA's

device driver must resolve IP addresses into Fibre Channel addresses.
Most configurations assume that all IP-attached devices reside on the same IP subnet
and that no IP router engine exists in the fabric to which it is connected.
 If a SAN design requires concurrent use of IP and SCSI-3, some vendors require a
separate card for each protocol.
Some host bus adapters offer add-on features such as HBA-based RAID, which offloads
the task of striping of data across multiple drives from the server's CPU.

VI drivers will allow applications ready access to the Fibre Channel transport without
passing through traditional, CPU-intensive protocol stacks.
And advanced diagnostic features—including SCSI Enclosure Services (SES)

emulation and standardized management application programming interfaces (APIs)—
allow HBAs to participate in umbrella SAN management platforms.
The Storage Networking Industry Association has promoted interoperable HBA

management through development of the Common HBA API.

Fibre Channel RAID
Redundant arrays of independent disks, or RAIDs, are a well-established technology in
traditional storage applications.
RAID standards define methods for storing data to multiple disks and imply intelligence
in the form of a RAID controller.
The controller can be implemented in software:

Example:- Application running on a file server—but typically is a dedicated
card installed in a RAID storage enclosure.
A server is connected to a single disk drive, reads or writes of multiple data blocks are
limited by the buffering capability and rotation speed of the disk.
Increase throughput by dispersing data blocks across several disks in a RAID, a

technique called striping.
RAID is called level 0, although it boosts performance, it does not provide data security.
If a single disk fails, data cannot be reconstructed from the survivors.

RAID level 3 : writes byte-level parity to a dedicated drive.
RAID level 4 : writes block-level parity to a parity drive.

A dedicated drive contains the information required to reconstruct data.
RAID level 5 : addresses this problem by striping block-level parity information across
each drive.
RAID level 1 : achieves full data security by sacrificing the performance gain of striping
in favor of simple disk mirroring.
RAID provides for redundancy and speed can be implemented by a dedicated RAID
controller housed in the same enclosure as the disks, or by a RAID controller provisioned
in the host system or file server.
Data is passed from the operating system to the RAID controller, which then manages
the striping or mirroring task.
Fibre Channel-attached RAID enclosures use an integrated RAID controller that sits
behind the Fibre Channel interface.

The RAID controller receives SCSI-3 requests via the loop or fabric and stores or
retrieves data from the array using the appropriate RAID levels.
The RAID controller appears as an N_Port or NL_Port to the outside world but can use a
proprietary bus, a SCSI bus, or other architecture to talk to its drives.
RAID manufacturers can also incorporate Fibre Channel technology behind the RAID
controller.
The use of arbitrated loop between the RAID controller and its Fibre Channel disks is
invisible to the SAN, because the RAID controller still appears as a single N_Port or
NL_Port to the rest of the topology.
SANs, Fibre Channel RAIDs offer a number of advantages.

The advanced diagnostics support, management features, reliability, and scalability
typical of such high-end systems fill in most of the checkboxes for enterprise
storage selection criteria.
Server processing resources benefit by offloading data redundancy tasks to the
RAID controller.
Fibre Channel RAID thus enables server clustering and other applications that are
predicated on co-ownership of storage resources.

Fig : RAID with internal SCSI bus and internal arbitrated loop RAID

SAN design standpoint, Fibre Channel RAIDs simplify configurations because the RAID
controller appears as a single port address to the loop or fabric.
The RAID controller and all its storage appear as a single N_Port on its own 100MBps
segment.
A single JBOD enclosure will appear as multiple NL_Ports sharing a common

bandwidth.
you can attach more than a terabyte of storage without consuming all available AL_PAs.
JBODs do have one distinct advantage over RAIDs: Cost.

Fibre Channel JBODs
A JBOD, or Just a Bunch of Disks, is an enclosure with multiple Fibre Channel disk
drives inserted into a common backplane.
The backplane provides the transmit-to-receive connections that bring the assembled
drives into a common arbitrated loop segment.
These connections bypass electronics that let you insert or remove drives without
disrupting the loop circuit.
Fig: JBOD disk configuration with primary and secondary loop access

A JBOD brings two thing:
The receive lead going to the first drive of a set.
The transmit lead coming from the last drive of a set
Fibre Channel interface, typically DB-9 copper.
The JBOD's interface is connected to a Fibre Channel hub or switch, the connection is
not to a single Fibre Channel device but to multiple independent loop devices within the
enclosure.
If the connection is made to a switch, the switch port must be an FL_Port because the
downstream enclosure is actually a loop segment.
If the connection is to an arbitrated loop hub, the population of the entire loop is
increased by the number of drives in the JBOD.
JBODs may also include options for configuring the backplane to support dual loops to a
single set of drives.
Software RAID improves performance by avoiding the latency of reading or writing to a

single drive, but this gain must be balanced with the increased traffic load on the loop.

The problem occurs , when you use software RAID with a JBOD, because the initiator
must conduct a series of small transactions to multiple targets.
At the SCSI-3 level, commands queue to the targets and await response from each one
before more frames can be sent.
JBOD enclosures are typically marketed with eight to ten drive bays, some of which may
be configured for failover.
Some very large disk arrays are packaged in 19-inch rack form factors, with more than
20 disks per JBOD module and as many as four modules per rack enclosure.
These high-end systems may include rack-mounted arbitrated loop hubs or switches for
connecting JBOD modules to one another and to servers for single- or dual-loop
configurations.
Given the amount of customer data that is stored on such arrays, it is preferable to use
managed hubs or fabric switches rather than unmanaged interconnects.
Dual power supplies, redundant fans, and SES management options allow JBODs to be
used in high-availability environments.

Vendors may also provide an upgrade path to add a RAID controller card to the
enclosure, something that extends the life of the original investment.
You can begin with a partially populated JBOD enclosure and add disks as storage needs
dictate.
A RAID controller option can then be added to increase performance and offload
software RAID tasks from the host.

Arbitrated Loop Hubs
The successful marketing tactics of fabric switch vendors, loop hubs are less commonly
deployed for SANs than are higher-priced switches.
Hubs are available from a number of vendors in various port configurations, interface
types, and levels of management.
Unlike other Fibre Channel devices, a hub is a passive participant in the SAN topology.
Star Topology for Arbitrated Loop:
wire arbitrated loops into a ring by connecting the transmit lead of one device to the
receive lead of another and extending this scheme until all devices are physically
configured into a loop.
This technique eliminates the cost of a concentrating hub.
 It exposes the topology to considerable risk.
loop hubs simplify the cable plant by concentrating physical links at a central
location, and they minimize disruption by providing bypass circuitry at each port.
Fig: Wiring concentration with an arbitrated loop hub
Hubs are usually co-located with storage arrays or servers in 19-inch equipment racks,
and that offers a further convenience for verifying status and cabling within the
enclosure.
Large storage arrays may have multiple JBODs or RAIDs and multiple loop hubs
configured into separate or redundant loops within a single 19-inch enclosure.

Hub Architecture:
Hub design varies from vendor to vendor, but all hubs incorporate basic features
specific to arbitrated loop.
A hub embodies the loop topology by completing circuits through each port and
then joining the transmit of the last port to the receive of the first.
If the signaling is too fast or too slow, the port will remain in bypass mode.
a port in bypass mode shunts the bit stream it has received from its upstream
neighbor directly to its immediate downstream neighbor.
Some vendor implementations turn off the transmitter as long as the port is in
bypass mode.
The transmitter is then enabled only when the hub port receives valid signal
from a newly attached device.
Auto-bypass circuitry allows the loop topology to self-configure when devices

are added or removed.

Fig: Internal hub architecture with port bypass circuitry

The combination of hub auto-bypass and insertion with arbitrated loop's self-
configuring addressing algorithm offloads tedious administration tasks from day-
to-day storage network operations and facilitates changes to the topology with
minimal disruption to data traffic.
Loop hubs may offer one or more light-emitting diodes (LEDs) per port to
display port status.
Typically, two LEDs are provided:
A green LED to indicate a link connection, and an amber LED to indicate the
current bypass mode.
Some implementations use a single, multicolored LED per port, thereby reducing
the number of diagnostic display states available.
Port LEDs give the operator an at-a-glance status of a device's connection state,
and by simplifying troubleshooting they help reduce down time.

Green Amber Port State
Off Off No device attached
On Off Device Attached and loop inserted
On On Device Attached and Bypassed
Off On Bad GBIC, port bypassed
Blinking Blinking Maintenance mode via hub

management
Table : Hub Port LED State Table Green Amber
Various hub products are available that are engineered to different marketing
requirements, from simple entry-level unmanaged hubs to hubs with advanced diagnostics
on-board.

Unmanaged Hubs :
Unmanaged hubs can be used for small, single-vendor environments that are
less susceptible to the dynamics of larger, more complex SAN installations.
If only a single server and one or two disk arrays are involved, there is less
exposure to prolonged down time if a cable or other component fails.
Unmanaged hubs are a simple, low-cost, entry-level interconnection solution.
They typically provide port bypass circuitry, based on valid signaling alone and
port LEDs to display insertion or bypass status.
If an attached device is unplugged or powered off, an unmanaged hub will auto-
bypass the port and light the appropriate port LEDs.
Having no intelligence or ordered set recognition circuitry, an unmanaged hub

cannot respond to protocol violations or conditions, such as a port streaming
LIP(F8), that would bring the loop down.
Without hub management to report the backup loop failure, the redundancy
would no longer be in force, and the failure of the primary loop would bring
the system down.
Troubleshooting would then be complicated by the fact that what appeared to
be a single occurrence—system failure— was actually the result of two
separate events on two separate loop topologies.
Unmanaged hubs are a logical choice for low-cost storage network solutions in
which economical servers and small JBODs are used to meet budget restraints.
These entry-level systems bring SAN capability within reach of small business
and departmental applications.
Managed Hubs:
Managed hubs introduce intelligence into the SAN interconnection.
The degree of intelligence and management capability varies from product to

product and is normally reflected in price.
At the low end of the managed hub offering, basic hub status and port controls
are available via Web browser, Telnet, or SNMP (Simple Network
Management Protocol) management software.

First, at the hardware level, additional on-board circuitry is needed to monitor
power supply, fan, temperature, and port status.
These functions are the minimum requirement for hub management, although
some implementations concentrate on port status alone.
Managed hubs with advanced capabilities also provide more extensive

diagnostic circuits to monitor the state of the loop, protocol activity, and more
comprehensive port and hub status.
Second, for these circuits to be useful, the hub must be able to report to and
accept commands from an external management workstation.
This is typically accomplished via an Ethernet port on the hub, over which
SNMP queries and commands are sent from an NT or UNIX console.
The application software used to manage a hub is provided by the hub vendor,
either as a stand-alone program or as a utility that can be launched from more
comprehensive SAN management applications.
Managed hub products that support SNMP or Web browsers can be managed
from anywhere in an IP routed network.
Fig : Managed SAN with remote SNMP console

SNMP is preferable to browser-based management because multiple hubs can
be managed from a single workstation.
Browser-based management typically speaks to only one IP address, and

therefore to one hub, at a time.
The event log may be queried from the hub using Telnet or SNMP, or it may be
replicated on the management workstation for a permanent record of activity.
An event log is a useful diagnostic tool, especially for troubleshooting

intermittent problems that may occur during unattended operation of the
management workstation.

Switching Hubs
Switching hubs are a hybrid SAN interconnection, occupying a middle ground between
shared loop hubs and fabrics.
switching hubs provide the simplicity of arbitrated loop with the high-performance
bandwidth of switches.
switching hubs are not fabric capable and so do not support fabric login, SNS, or state
change notification.
This reduces the requirements and additional cost associated with fabric switches.
configurations that require high bandwidth but no more than 126 total devices, switching
hubs offer a reasonable price and performance solution.
Switching hubs typically provide 6 to 12 ports, each of which supports 1Gbps or 2Gbps
throughput.
The attached loop nodes are configured into one virtual loop composed of multiple loop
segments.

Fig: Switching hubs allow multiple concurrent loop transactions

Assigning high-performance servers to their own ports and RAIDs or JBODs to other
ports, switching hubs provide the aggregate bandwidth of fabric switches.
Switching hubs support SNMP, SCSI Enclosure Services, or other management features.
Depending on the vendor's design, some products offer advanced diagnostic features,
including the ability to direct, via a management interface, data capture traffic on any
other port without disrupting the topology.

Fabric Switches
Current-generation fabric switches support 1Gbps or 2Gbps per port, provide a high-
speed routing engine to switch frames from source to destination, and offer basic
services for fabric login and name server functions.
Products are differentiated based on vendor-specific issues such as port density,

performance, and value added functionality for ease of installation and management.
Fabric switches may support 8 to 16 ports for departmental applications, or 32 to 128

ports (or more) for larger enterprises.
Cascading fabrics via expansion ports (E_Ports) allows small and medium
configurations to expand as SAN requirements grow, it may become potential bottlenecks
for fabric-to-fabric communication.
This resolves the congestion issue but may introduce another problem if the source and
destination N_Ports on either side receive out-of-order frames.
In addition to bandwidth, switch-to-switch latency may limit the number of switches in a
path.

Fig : Fabric switch functional diagram

Fibre Channel standards define:
F_Ports for attaching nodes (N_Ports)
 E_Ports for fabric expansion
G_Ports for supporting either N_Ports or other fabrics
NL_Ports for loop attachment.
The standards do not define how, specifically, these port types are to be implemented in
hardware, so vendor designs may differ.
Some products offer a modular approach, with separate port cards for each port type.
Others provide ports that can be configured via management software or auto-
configuration to support any port type.
The latter offers more flexibility than the others for changing SAN topologies,
permitting redistribution or addition of devices with minimal disruption.
All fabric switches support some variation of zoning. Port zoning allows a port to be
assigned to an exclusive group of other ports.

Port zoning, normally a no-cost enhancement to the switch, is an accessible means to
segregate servers and their storage from other servers or to isolate different departments
sharing the same switch resource.
To achieve interoperability in multi-vendor switch environments, a process called zone

merging is required.
Fabric management graphical interfaces may include topology mapping, enclosure and
port statistics, routing information, and port performance graphing.

Departmental Fabric Switches:
Fabric switches that provide 8 to 32 ports are referred to as departmental switches to

differentiate them from more robust and higher-port-count director switches.
Departmental Switch is used only in stand-alone departments or small-scale,

dispersed SANs.
Departmental fabric switches may have redundant power supplies and swappable
fans, but they do not provide the high availability features of director-class fabric
switches.
Each departmental fabric switch is a FRU (field-replaceable unit), and high

availability requires installation of dual data paths and redundant switches.
A vendor data sheet for a departmental switch may declare support for as many as
239 switches in a fabric, practical guidelines typically call for no more than 32
switches, with no more than 7 switch-to-switch hops in any path.
High-port-count enterprise SAN requirements, it is more efficient to use director-

class switches at the core, with departmental switches as fan-out for device ports.
Fibre Channel Directors:
Fibre Channel directors provide high port counts in a single, highavailability
enclosure.
Directors may provide 64 to 128 or more ports (256 ports for some announced
products) and so present a streamlined solution for the storage requirements of large
data centers.
Fibre Channel director architecture implies high availability for every component,
including redundant processors, routing engines, backplanes, and hot-swappable port
cards.
departmental switches, the number of directors that can be connected in a single

fabric may be limited. This is not because of the availability of ports for inter-switch
links but rather because of the complexity of exchanging route, zoning.
you can accommodate hop count limitations by combining directors with

departmental fabric switches.
Trunked interswitch links between directors can reduce potential blocking, and the
port fan-out supplied by departmental switches increases the total population that can
be reasonably supported.
Fig: Combining directors and departmental switches in a fabric

Fibre Channel-to-SCSI Bridges
Fibre Channel-to-SCSI bridges perform this function for Fibre Channel SANs by
allowing legacy SCSI devices to participate in storage networking.
Fibre Channel-to-SCSI bridges normally provide one or two Fibre Channel interfaces
for SAN attachment, and two to four SCSI ports for SCSI disk arrays or tape backup
subsystems.
In addition to this physical and transport conversion, the Fibre Channel-to-SCSI bridge
translates serial SCSI-3 protocol to the appropriate SCSI protocol required by the legacy
devices.
The Fibre Channel-to-SCSI products, however, do not actually route data at layer 3;
instead, they simply convert one form of SCSI to another. In other words, they provide a
bridging function.
The most common application for Fibre Channel-to-SCSI bridges is to support legacy
tape backup subsystems.
Fibre Channel-to-SCSI bridges may therefore both preserve your investment in tape
subsystem hardware and satisfy the need to optimize the tape backup process itself.
Fig: A Fibre Channel-to-SCSI bridge supporting SCSI-attached tape

Placing a tape subsystem behind a Fibre Channel-to-SCSI bridge also removes ownership
of the subsystem from an individual server.
Because the tape subsystem is now addressable by any server on the storage network, all
servers can share what was previously a dedicated resource.

Fibre Channel Extension Products
Because organizations often have multiple regional or international sites, the ability to
access storage data over distance has gained increasing importance.
NEED: In addition to disaster recovery, wide area storage applications—such as

consolidated tape backup, resource sharing, and content distribution—require a means to
extend Fibre Channel-originated traffic beyond the central data center.
Fibre Channel extension—transport methods that require no protocol conversion—the

most common extension products rely on dark fiber and dense wave division
multiplexing or on encapsulation of Fibre Channel frames using IP tunneling.
Fibre Channel Extension Using DWDM:

Dense wave division multiplexing (DWDM) is a physical transport that not
tolerate optical signaling to send numerous data streams concurrently over a single
optical link.
Light traversing an optical cable is composed of various modes or wavelengths of

light.

If a data stream is associated with a specific wavelength of light at the point of
transmission, it can be separated from other data streams at the receiving end.
With current technology, as many as 64 concurrent streams can be supported, each

riding its own wavelength.
Fig: DWDM transports each data source on a separate wavelength

DWDM requires dedicated fiber cabling, commonly referred to as dark fiber.
 Dark fiber is any unused optical pair of an installed cable run. In contrast, lit fiber
is already carrying Ethernet, ATM, or some other transport.
DWDM can carry any protocol but has become linked to Fibre Channel extension
because of its ability to drive longer distances more efficiently than traditional long
wave optics and single-mode cable.
 Fibre Channel fabric switches were originally designed for data center
applications within a fairly narrow circumference (500m with multimode cabling
and shortwave optics).
Fibre Channel extension with DWDM can be used for either switch-to-switch or
switch-to-node connectivity.
From the standpoint of the fabric switch, the intervening DWDM equipment and
long haul cable plant are transparent.
Consequently, E_Ports or F_Ports can be connected to the DWDM infrastructure

as if it were a straight run fiber cable.
.
E_Port connections, switch-to-switch behavior—including principal switch selection,
route information and SNS exchange, zone merging, and so on—now occurs across an
extended link.
As long as the link is stable, this stretched E_Port connection presents no major
difficulties. If the link fails, however, the effect is the same as pulling the cable between
two adjacent fabric switches.
Each would undergo fabric reconfiguration, with all storage conversations suspended
until each fabric stabilized

Fibre Channel Extension Using IP Tunneling :
The more common and more affordable IP network services is an attractive option
for Fibre Channel extension.
Fibre Channel over IP (FCIP) is an IP tunneling solution favored by Fibre

Channel vendors because it perpetuates acquisition of Fibre Channel switches on both
sides of an extended link.
Alternatively, two native IP storage protocols (iFCP and iSCSI) can also be used to
extend Fibre Channel-originated traffic.
FCIP device typically attaches by E_Port connection to the source Fibre Channel
switch.
Fibre Channel frames destined for the remote end are wrapped in IP datagrams and
sent across the IP network.
At the receiving end, the IP datagrams are stripped off, and the original Fibre
Channel frames are delivered to the E_Port of the receiving fabric switch for routing.

Fig: Using FCIP tunneling to span distance between fabric switches

FCIP implements a standard E_Port connection over distance, normal Fibre
Channel switch-to-switch protocols are passed over the wide area link.
FCIP tunneling is suitable for applications that require connectivity only between
two sites.
FCIP is a point-to-point wide area connection.
A pair of FCIP devices is therefore required for each remote link, and each remote
site assumes the existence of a fabric switch in addition to end nodes.

Fibre Channel WAN Bridging :
Fibre Channel WAN bridging offers another solution for SAN extension.
A Fibre Channel WAN bridge transports Fibre Channel traffic over non-Fibre
Channel topologies such as ATM.
Connection to a local fabric is via a B_Port (bridge port) interface on the bridge.
B_Ports, which provide a subset of E_Port protocols, either transparently pass

through switch-to-switch Class F frames or engage in the fabric building process as
an additional switch.
WAN bridging between two Fibre Channel SANs may result in a single fabric
spread over distance.
The WAN bridges pass E_Port traffic between fabric switches, and a common
address space with a unique Domain_ID is established for the dispersed SAN.
The WAN bridge can engage in E_Port behavior with its local fabric switch, creating
an autonomous Fibre Channel region.

Fig: Autonomous regions created with Fibre Channel WAN bridging

SAN Software Products
Storage area networking has led to the development of new software applications that
leverage the potential of shared storage access.
High-availability storage access via server clustering, rationalized data backup and
restore, disk-to-disk data replication, and file sharing utilities—all these are predicated on
a SAN infrastructure that provides peer-to-peer connectivity.
Server Clustering:
Server clustering involves several functional components, including failover, load
sharing, and common data access.
Each server is tied to its own storage via parallel SCSI cabling, i.e. we cannot
efficiently combine the resources of multiple servers.
Separating servers from storage, and then reconnecting them on a high-speed

network topology, enables storage to be accessed independently.
For servers on the same SAN to be clustered, however, cluster software must be run
on each server to coordinate and monitor server-to-server communications.
Fig: Dual-pathed four-server cluster on a SAN

Failover for clustered servers can scale from 2 to more than 30 servers in a single
configuration.
Failover strategies are vendor-specific but generally require mutual monitoring of server
status via a heartbeat protocol.
The heartbeat protocol sends keepalive messages between the servers as well as
notification of status or required action.
The heartbeat can be run in-band over the Fibre Channel or Gigabit Ethernet SAN
infrastructure, or out-of-band over a Fast Ethernet connection.
In some implementations, a failover algorithm triggers on the loss of an individual

application or discrete server component (for example, failure of a host bus adapter or
LAN interface).
Other servers in the cluster assume the tasks of an individual server that has suffered a
failure.
If multiple servers are running the same application, you need additional middleware to
ensure that servers do not overwrite shared data.

Tape Backup:
Tape backup is a universal requirement, and a universal problem, for all data
networks, regardless of the specific topology employed.
Data security via backup is not only desirable but also sometimes mandated by law
(for example, in finance and banking operations).
As applications generate ever-growing storage requirements, the time required to

back up data increases.
You cannot accommodate this backup window via the production LAN without
increasing the number of switch links to accommodate block storage as well as user
bandwidth requirements.
Traditional LAN-based tape backup is based on backup of files.
A tape subsystem is attached to a backup server, which in turns sits on the LAN.

Each server on the LAN may have gigabytes of direct-attached storage that needs to
be secured through backup.
The backup server instructs each server to initiate a backup, with the data sent over
the LAN from server to backup server.
This type of backup involves multiple conversions. Upon launching a backup, the
target server must read blocks of SCSI data from disk, assemble the blocks into
files, and packetize the files for transfer over the LAN.
At the backup server, the inbound packets must be rebuilt into files, and the files, in
turn, are disassembled into blocks to be written to tape.
Therefore, the original data blocks that reside on the target storage undergo four
steps of conversion before reappearing at the destination as blocks:
Both the server and the backup server must devote considerable CPU cycles to
SCSI as well as network protocol overhead.

Fig: Traditional LAN-based tape backup

On shared media LANs, this backup operation places a significant load on the LAN
and may disrupt user messaging traffic.
Most enterprise LANs, however, use switched Ethernet infrastructures instead of

shared media.
Bottlenecks may occur on switch-to-switch links, but not on individual Ethernet

switches.
SAN-attached storage and tape offers efficiencies for tape backup by eliminating
the block-to-file conversion overhead.
For IP SANs, the block tape backup may be over a separate Gigabit Ethernet
switched network, or over a VLAN within a larger Gigabit Ethernet complex.
In both cases, by leveraging block-based SCSI transfer, you allow more data to be
transported in less time and with less overhead.
Placing servers, storage, and tape on a peer-to-peer network also enables new
backup applications such as server-free tape backup.

As long as the server is the exclusive owner of storage, the server must always be in
the data path for any type of access, including tape backup.
SAN, storage is directly accessible by a tape subsystem. As shown in Figure, the

server can be taken out of the backup data path, with blocks of data read from disk
and written directly to tape.
Server-free backup requires an extended copy (third-party copy) agent, which

typically resides in the tape subsystem, in a bridge, or in a SAN switch.

Fig: Server-free tape backup using extended copy

SAN plumbing and utilities such as extended copy facilitate efficient backup of
storage data, the application software that initiates and manages backup processes
varies in capabilities from vendor to vendor.
Although every storage administrator recognizes the necessity of data backup, it is

often difficult to verify that a backup operation was completed and that the tapes can
actually be used for data restore.
In addition, regular backup operations may repeatedly copy data that is unchanged
over time, and that adds to the volume and duration of the backup process.
Vendors of backup software may provide utilities for verification, point-in-time

(snapshot) backup for active databases, changed-block-only backup, or other value-
added backup services.
As the volume of storage data grows, the task of securely backing up data in a
reasonable time frame becomes increasingly difficult.
Backup application software and tape vendors are trying to meet this challenge.
Storage vendors are offering data replication, an alternative but complementary

solution.
Data Replication :
Tape backup has survived simply because it is the most available means to provide a
copy of data on a stable medium.
An enterprise should have a synchronized copy of data available and should be able
to access that copy immediately if the primary storage fails.
This is the goal of data replication, which uses disk mirroring algorithms to
duplicate data from one disk array to another.
Disk-based data replication, by contrast, is transparent to the host system and

offloads all duplication tasks to the disk array itself.
In this case, the disk arrays must be both targets and initiators, receiving data to be
written while managing write operations to the secondary storage.
Data replication normally implies distance, with primary and secondary storage
arrays separated by at least metropolitan area distance.
Consequently, data replication must define how data mirroring will be accomplished
at the array level and how wide areas will be spanned.
Fig: Active-passive (top) and active-active data replication
A data replication configuration has primary and secondary storage arrays.
An active-passive configuration requires data written to the primary array to be

synchronously written to the secondary array.
In the event of failure of the primary, the secondary can be accessed directly.
For companies with multiple data centers or sites, an active-active
configuration enables each site to serve as both primary for local access and
secondary for another site.
Regional centers can thus serve as mutual data replication sites for each other,
ensuring that a readily accessible copy of each site's data is always available.
Data replication can be performed with synchronous or asynchronous updates

between primary and secondary storage.
In synchronous mode, the write operation is not final until both arrays have
signaled write completion.
This guarantees that an exact copy of data is now on both arrays, although at a
performance penalty.

The primary storage must always wait until the secondary is finished before
reporting successful write to the host.
In asynchronous mode, the primary array can buffer writes intended for the
secondary and initiate them only during idle periods.
This improves performance but may result in loss of a true copy if the write to the
secondary subsequently fails.
The primary array would then be forced to break the mirror to the secondary, and
possibly track changes in data until the secondary recovers.
Data replication can be performed within the local data center to provide a current
and readily accessible copy of data, or it can be extended over distance to facilitate
disaster recovery and business continuance scenarios.
In both synchronous and asynchronous implementations, the stability of the link
between primary and backup disk arrays is critical, as is the latency that would
naturally occur over very long haul links.

Distributed File Systems and File Sharing:
A peer-to-peer storage network enables multiple servers to share a storage resource,
it does not automatically enable servers to share the same storage data.
In fact, most SAN deployments are configured for a shared nothing environment,
in which individual servers are assigned separate LUNs on the storage target, Each
server manages its own data.
In a server clustering scheme, the LUNs previously assigned to a failed server can
be mapped to active servers so that data access can continue.
The SAN provides the network that facilitates this deliberate reassignment of
resources, but at any point in time each server has access only to its authorized
LUNs.
To share the storage data itself, a layer of management prevents data corruption.
You must monitor the status of a file or record that is accessible to multiple servers
on the SAN so that you can track changes.
In read-only applications, such as multiple servers accessing display-only Web

content, a simple file locking monitor may be sufficient.
In active read/write applications, such as a load sharing server cluster, more
sophisticated management is required.
Changes to shared data must be permitted, but in an orderly fashion that

synchronizes all modifications to the original data and writes a coherent version
back to disk.
Data sharing is further complicated by the fact that the data is typically dispersed
over multiple storage arrays as a Storage Pool.
The complex of physical storage devices on the SAN must be presented as a single
logical resource on top of which sits a common view of a file system shared by all
servers in a cluster.
A distributed volume manager must thus present a coherent view of the physical
storage resources; a distributed file system presents a uniform view of directories,
subdirectories, and files.
A distributed file system must present a consistent image to the server cluster.
If a new file is created by one server, other servers must be updated immediately.

Fig: A distributed file system presents a common view of resources to all servers

Similarly, a file opened by one server cannot be arbitrarily deleted by another, or else
the file system will lose integrity.
If multiple servers may have the same file opened, and if modifications of the file
are permitted, a distributed file system must also be able to notify each server of
pending changes.
Cache coherency between multiple servers requires notification of any changes to

the file and a reread of the updated file to refresh the cache.
Device Memory Export Protocol (DMEP) has been proposed as a means to

maintain cache coherency on storage arrays and avoid the issue of conflicting
cached versions of files on each clustered server.
Applications that benefit from data sharing range from high-availability server
clusters to processing-intensive application clusters that must digest massive
amounts of data.
Example: Sistina's Global File System (GFS) for large server clusters that
analyze data on the distribution network area.

The management of distributed file systems is more complex than that of a shared
nothing implementation, a distributed file system facilitates:
high availability
simplifies system administration
high-performance
processing clusters for a wide variety of compute-intensive applications.
Tighter integration into operating systems
It will make distributed file systems and file sharing more available and further enhance
the value proposition of SANs.

Simple Problem Isolation Techniques in SANs
Mostly error conditions are encountered during changes to a network's topology.
During initial installation of the network because new equipment is being connected for
the first time.
In complex SAN installations, you must install host adapters and load their appropriate
device drivers, lay the cable plant, install and configure switches, position GBICs or
transceivers, and properly deploy and cable disk arrays.
To simplify installation, solutions providers typically pre-configure as many

components as possible.
After the various SAN components have been configured, cabled, and powered on, you usually
verify operation by testing a server's ability to access storage.
If a server cannot see part or all of the assigned disks, elementary troubleshooting begins.
This process typically begins with examination of the physical cable plant, port status, and
transceivers and continues through verification of the host system, interconnection, and storage
target.

Fig: Troubleshooting server-to-target
connectivity

You can diagnose physical-layer problems by verifying insertion or bypass status at the
switch port.
Port status LEDs should indicate whether an inbound signal is recognized or whether it is
intermittently making and breaking contact.
Depending on the server's operating system, failure to discover targets may be a matter of
the SAN boot sequence.
The configuration utility supplied with the adapter card should indicate current and
allowable addresses, as well as the microcode and device driver versions for the card.
For Fibre Channel fabrics, you can examine the switch SNS table to verify the successful
connection of end devices to the switch.
Zoning may be configured by port or World-Wide Name (WWN).
Depending on vendor equipment, the default zoning configuration may exclude all
devices.
In multi-switch configurations, status of the inter-switch links may also be an issue.

Fibre Channel switches typically autosense E_Port connections in a single-vendor fabric,
but they may need to be manually configured in multivendor fabrics.
E_Port connectivity may also require that you manually configure switch addresses or
designate which switch will serve as principal switch for the fabric.
Initiators and targets separated by multiple hops may have the appropriate entries in their
respective switch's SNS table but be unable to communicate because of excessive
switch-to-switch latency or failure of SNS updates to propagate through the fabric.
SAN vendors have attempted to provide more advanced diagnostic capabilities in their
products, the ability to crack frames and provide protocol decode at multi-gigabit speeds
would add significant cost to any product.
A full diagnostic of a storage network therefore requires protocol analyzers and either in-
house expertise or a contracted support organization with trained personnel

Fibre Channel Analyzers
As with network analyzers developed for LAN and WAN traffic analysis, Fibre
Channel analyzers provide protocol capture and decode on the transmission link.
 Even with enormous buffers, it is not possible to capture more than a few seconds of
traffic at 1Gbps and 2Gbps speeds.
You can, capture protocol events that may last some microseconds or milliseconds of
time.
Typically, an analyzer is used to trace a specific process, such as a fabric login problem.
Instead of randomly sampling traffic in the hopes of capturing a specific event,

analyzers let you trigger the start of data capture on the basis of designated ordered sets
and addresses.
Trigger criteria can be complex, with combinations of protocol states and

source/destination address, or they can be restricted to certain bit patterns.

Thank U…~!~

San Unit III

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

San Unit III

Загружено:

Авторское право:

Доступные форматы

Lecturer Notes

Storage Area Network

Ms. Madhu G. Mulge

M.B.E.'S College of Engineering, Ambejogai.

ME - CSE [SAN- Unit III] 1

The manufacturers of Fibre Channel products go to market along various paths,

Closer cooperation between noncompeting vendors—such as manufacturers of HBAs,

ME - CSE [SAN- Unit III] 2

Software components include:

ME - CSE [SAN- Unit III] 3

An optical GBIC provides an SC connector for multimode or single-mode fiber-optic

ME - CSE [SAN- Unit III] 4

Another advance in GBIC technology is the development of serial ID functionality.

Serial ID allows a supporting enclosure—such as a hub, switch, or HBA—to query the

Inventory information includes manufacturer, date of manufacture, serial number, speed,

ME - CSE [SAN- Unit III] 6

Serial ID relies on the supporting enclosure to solicit this information.

ME - CSE [SAN- Unit III] 7

Fig: Small form factor pluggable transceivers

ME - CSE [SAN- Unit III] 8

ME - CSE [SAN- Unit III] 10

The design of HBAs typically includes Flash ROM for microcode.

Device drivers for specific operating systems are also upgradeable.

When upper-layer applications send data addressed to an IP destination, the HBA's

ME - CSE [SAN- Unit III] 13

And advanced diagnostic features—including SCSI Enclosure Services (SES)

The Storage Networking Industry Association has promoted interoperable HBA

ME - CSE [SAN- Unit III] 14

The controller can be implemented in software:

Increase throughput by dispersing data blocks across several disks in a RAID, a

ME - CSE [SAN- Unit III] 15

RAID level 4 : writes block-level parity to a parity drive.

ME - CSE [SAN- Unit III] 16

SANs, Fibre Channel RAIDs offer a number of advantages.

ME - CSE [SAN- Unit III] 17

ME - CSE [SAN- Unit III] 18

A single JBOD enclosure will appear as multiple NL_Ports sharing a common

JBODs do have one distinct advantage over RAIDs: Cost.

ME - CSE [SAN- Unit III] 19

ME - CSE [SAN- Unit III] 20

Software RAID improves performance by avoiding the latency of reading or writing to a

ME - CSE [SAN- Unit III] 21

ME - CSE [SAN- Unit III] 22

ME - CSE [SAN- Unit III] 23

Star Topology for Arbitrated Loop:

This technique eliminates the cost of a concentrating hub.

 It exposes the topology to considerable risk.

ME - CSE [SAN- Unit III] 25

Auto-bypass circuitry allows the loop topology to self-configure when devices

ME - CSE [SAN- Unit III] 26

ME - CSE [SAN- Unit III] 27

Typically, two LEDs are provided:

ME - CSE [SAN- Unit III] 28

Off Off No device attached

On Off Device Attached and loop inserted

On On Device Attached and Bypassed

Off On Bad GBIC, port bypassed

Blinking Blinking Maintenance mode via hub

Table : Hub Port LED State Table Green Amber

ME - CSE [SAN- Unit III] 29

Unmanaged hubs are a simple, low-cost, entry-level interconnection solution.

Having no intelligence or ordered set recognition circuitry, an unmanaged hub

Managed hubs introduce intelligence into the SAN interconnection.

The degree of intelligence and management capability varies from product to