Lecturer Notes

Storage Area Network

( Unit One )

Ms. Madhu G. Mulge

( M-Tech CSE )

M.B.E.'S College of Engineering, Ambejogai.

ME CSE [sem-II] 1
 Introduction
Designing storage area networks requires an understanding of the upper-layer
applications that depend on storage resources as well as the storage
architectures available to satisfy them.

 A framework has been developed to help storage architects determine which

storage solutions may be most appropriate.
Example : SAN , NAS, etc

Sharing storage resources among multiple servers or workstations requires a

peer-to-peer network that joins targets to initiators.

SAN’s functions : creation of management, security, and other auxiliary

services has trailed somewhat behind the deployment of shared storage
networks.

The composition of that network and the type of storage data traversing it vary
from one architecture to another.
ME CSE [sem-II] 2
Shared Storage Architectures
 Storage Area Networks (SAN)
Fibre Channel
Gigabit Ethernet
 Network-Attached Storage (NAS)
Ethernet (Fast Ethernet or Gigabit Ethernet)

Applications
Online Transaction
Mining Database
Web services

Shared Storage Model establishes the general relationship between user

applications that run on servers and hosts and the underlying Storage
Domain.

ME CSE [sem-II] 3
Fig 1: The SNIA(Storage Networking Industry Association) Shared Storage Model
ME CSE [sem-II] 4
overview
The storage domain subdivides into three main categories:

File/record subsystem :
The file/record subsystem is the interface between upper-layer applications
and the storage resources.
SQL Server and Oracle

Block aggregation layer :

 Useful information appears to the upper-layer application as records or
files, both formats are ultimately stored on disk or tape as contiguous bytes
of data known as blocks.

 Block subsystem :
 Data Identified with specific records or files, the blocks themselves are
written to or read from physical storage.

Storage Domain is positioned as an auxiliary subsystem, i.e. Services Subsystem

contains a number of storage-specific functions, including management, security,
backup, availability, capacity, and planning.

ME CSE [sem-II] 5
Fig 2: Direct-attached storage in the SNIA Shared Storage Model

ME CSE [sem-II] 6
 The Shared Storage Model is possible to insert server and storage components to
clearly differentiate between direct-attached, SAN, and NAS configurations.

 Replacing direct-attached connections with a more flexible network interconnection

enables new storage solutions.

 In Direct attached storage a server with logical volume management (LVM) and
software RAID (redundant array of independent disks) running on the host system.

As the server receives information from the application that must be written to disk,
the software RAID stripes blocks of data across multiple disks in the block layer.

 Software RAID executes the mechanics to striping blocks of data to multiple disks.

 The logical volume manager (LVM) presents a coherent image of the database /
data to the upper-layer application in the form of volume (for example, E: drive),
directories, and subdirectories.

ME CSE [sem-II] 7
Fig 3: Changing the relationship between servers and storage via a SAN
ME CSE [sem-II] 8
SANs alter the relationship between servers and storage targets, Instead of
being bound by direct parallel SCSI cabling, servers and storage are now
joined through a peer-to-peer network.

As in direct-attached storage, logical volume management, software RAID,

and hardware RAID still play their roles, but the connectivity between servers
and storage devices allows you to attach any server to any storage target.

The SAN infrastructure can be Fibre Channel, Gigabit Ethernet, or, with the
appropriate IP storage switches, both.

To accurately represent actual customer deployments, Fig 3 model could also
depict multiple SANs servicing various upper layer applications and
providing attachment to common or distinct storage resources.

For heterogeneous environments, it might be useful to call out which SANs

use Fibre Channel fabrics exclusively and which involve a mix of Fibre
Channel and IP components.
ME CSE [sem-II] 9
Fig 4: Multiple SANs within the Shared Storage Model
ME CSE [sem-II] 10
Fig 5: The role of NAS in the Shared Storage Model
ME CSE [sem-II] 11
 The Shared Storage Model, positions of NAS (Array X and Y) devices
partly in the file/record subsystem layer, extending down to the block
Subsystem.

 NAS devices serve up files and so naturally include the block aggregation
functions required to put data to disk.

 NAS device is essentially a file server with its own storage resources.

 NAS devices typically use NFS (Network File System) or CIFS

(Common Internet File System) protocols to transport files over the local
area network (LAN) to clients.

ME CSE [sem-II] 12
Example: Carlson Companies

ME CSE [sem-II] 13
Storage and Networking Concepts
Storage and networking have evolved as two distinct technologies.

Storage: as embodied by the concepts of initiators.

Targets: assumes a master/slave relationship between
connected devices.

 SAN technology must provide:

 High-performance transport of data.

 Efficient placement of data after it has safely arrived.
 Resolve the contradiction created by thrusting master servers and storage
slaves into a democratic peer infrastructure.

ME CSE [sem-II] 14
Networking in front of the Server
The front of the server faces outward toward the user community via a
network interface.

Network dispersed computer resources across an enterprise by using Ethernet

as the physical transport and TCP/IP as the networking protocol.

The OSI model divides networking into seven functional layers.

As user data is passed down by an application for transmission across the
network, it is wrapped in successive envelopes of information, roughly
corresponding to the layers represented in some specific manner.

ME CSE [sem-II] 15
 Layer Layer Description
Number Name

7 Application Interface for email, file transfer, etc.

6 Presentation Application specific

5 Session End-to-end session control: NetBIOS,

Application specific.

4 Transport Transmission: TCP, UDP

3 Network Routing protocol : IP

2 Data link Access method: Ethernet, Token ring, etc.

1 Physical Physical transport: Twisted pair etc.

ME CSE [sem-II] 16
The most prevalent LAN transport is Ethernet over twisted-pair copper wiring.
Ethernet transmission rates are :
10Mbps (megabits per second)
100Mbps (also known as Fast Ethernet)
1,000Mbps or 1Gbps (Gigabit Ethernet)

10Mbps and 100Mbps speeds are most commonly used for attachment of
individual PCs or workstations, with effective data rates of slightly more
than 1MBps (megabytes per second) and 12MBps, respectively.

If you restrict the network to a single speed, bottlenecks will occur as more
users attempt to access shared resources.

Ethernet offers deterministic bandwidth in powers of ten, it is possible to

design scalable non-blocking infrastructures.

The core itself can be built with one or more 10Gb interswitch links to
provide a high-performance LAN backbone.

Ethernet LANs are normally limited to a single building or campus. To share

computer resources with remote sites, wide area networks (WANs) use
telecommunications lines, microwaves, or satellites to transmit data.
ME CSE [sem-II] 17
Fig 5: Scaling the network using departmental and core switches
ME CSE [sem-II] 18
Several fundamental principles of networking are common to both LAN
and WAN technologies.

 Networking principles include the following:

 Serial transport
 Access method
 Addressing
 Packetizing of data
 Routing of packets
 Upper-layer protocol support

ME CSE [sem-II] 19
SCSI bus Architecture

ME CSE [sem-II] 20
The SCSI protocol was developed to provide efficient transport of data
between computers and peripheral devices such as disks, printers, scanners,
and other resources.

SCSI sits below the file/record layer in the SNIA Shared Storage Model and
receives requests to send or retrieve blocks of data from a peripheral device.

The responsibility of the SCSI protocol is simply to ensure that the write task is
completed and to report success to the operating system, no matter how the
physical storage is configured.

SCSI targets are identified by the operating system in a three-part :

The Bus designator is one of several SCSI interfaces installed on a host

system.
The Target represents a single storage resource on a bus daisy chain.
 The LUN (logical unit number) designation identifies the SCSI
client within the target.
ME CSE [sem-II] 21
Fig 6 : SCSI Architectural Model (SAM-2)
ME CSE [sem-II] 22
The SCSI architecture defines the relationship between initiators (hosts) and
targets (for example, disks) as a client/server exchange.

The SCSI-3 application client resides in the host and represents the upper-
layer application, file system, and operating system I/O requests.

The SCSI-3 device server sits in the target device, responding to requests.

The client/server, requests and responses are exchanged across some form of
underlying transport, which is governed by the appropriate SCSI-3 service
delivery protocol for that transport.
FCP protocol or iSCSI for gigabit serial links.

The SCSI-3 commands that serve I/O requests from the host application are
differentiated from the SCSI-3 transport protocols that actually move data via
the service delivery subsystem.

Juggling multiple transactions to one or more targets requires context

switching: the ability to quickly switch from one job to another as SCSI
responses are processed. ME CSE [sem-II] 23
The bus/target/LUN identifier may be further mapped to the addressing
requirements of a specific transport.

Reads and writes of data between SCSI initiators and targets are performed
with a series of SCSI commands, delivery requests, delivery actions, and
responses.

SCSI commands and parameters are specified in the command descriptor

block (CDB).

The CDB is encapsulated within the Fibre Channel Protocol information

units (IUs), whereas in iSCSI it is carried in the iSCSI protocol data unit
(PDU).

A traditional parallel SCSI adapter card may represent a single bus, with that
bus supporting multiple daisy chained disks.

Alternatively, a Fibre Channel host bus adapter (HBA) or iSCSI network

interface may be viewed by the operating system as a SCSI bus.
ME CSE [sem-II] 24
The Parallel SCSI Bus
The most common method for connecting disk and tape devices to file and
application servers has been via traditional SCSI bus cabling.
One of the difficulties presented by parallel bus architecture is a
phenomenon known as skew.
Skew refers to the difference in arrival time for each bit comprising a data
word.

Fig 7: Timing skew on parallel bit transmission

ME CSE [sem-II] 25
 Parallel SCSI requires termination of any unused ports.

 As multiple SCSI devices are daisy-chained together, end devices must be

terminated to avoid erratic signal interference.

Proper cabling and termination are critical for stable parallel SCSI operation,
and improper termination can cause data corruption or transaction failures.

Parallel SCSI cabling to multiple SCSI host bus adapters installed in the
server (as shown in Fig 8).

Parallel SCSI has restrictions in terms of distance and number of devices

supported on a SCSI bus make it difficult to support larger storage capacities.

The primary weakness of parallel SCSI for enterprise applications

 it binds storage resources to a single server.

Storage resources cannot be shared easily by multiple servers, and unused

storage capacity behind one server cannot be utilized by another.
ME CSE [sem-II] 26
 Fig 8: Server with multiple SCSI adapter cards installed

 Network Solution : direct-attached storage configurations, the limitations of

parallel SCSI are being addressed by Serial Attached SCSI (SAS).
Efficiencies in local storage connectivity.
 increase immune to the skew and distance limitations of parallel cabling.
ME CSE [sem-II] 27
Network-Attached Storage (NAS)
NAS accomplishes the central goal of storage networking: the sharing of
storage resources through the separation of servers and storage over a
common network.

In redundant configurations, NAS can also provide highly available, non-
disruptive storage access.

NAS performance is determined in part by the bandwidth available on the

messaging network.

NAS device can rely on parallel SCSI, IDE, or Fibre Channel for storage
connectivity.

 NAS vendors is to make their products more appliance-like and to reduce the
overhead of NFS/CIFS protocols over TCP/IP.

The concept of “Appliance"—a device that is simply plugged in to the network and
requires little administration.

NAS vendors support the Distributed File Access SystemME

(DAFS)
CSE [sem-II]
over Virtual 28
Interface (VI)
Fig 9: NAS appliance can use any type of physical storage
ME CSE [sem-II] 29
Another trend in NAS technology is the separation of storage from the NAS
processor or head, so that storage capacity increased.

NAS technology is the separation of storage from the NAS processor or head,
so that storage capacity can be more easily expanded using SAN techniques
or shared with SAN-attached servers (as shown in fig 10).

Advantage: higher performance and scalable storage capacity.

Limitations: presents vulnerabilities in the event of loop failure.

Solution : Embedded-loop switch technology is used on the backplane of some

NAS storage arrays to bypass failures and maintain stable operation

ME CSE [sem-II] 30
Fig 10 : NAS head with shared SAN storage
ME CSE [sem-II] 31
Networking behind the Storage
Storage Area Networks introduce network relationships behind the server.

SANs enable more efficient use of storage resources and lower administrative
costs through storage consolidation.

 Replacing the dedicated parallel SCSI cabling bond between servers and
storage with peer-to-peer networking does two things:
 Only provides more flexibility in the deployment of storage resources.
 Frees storage from the exclusive ownership by a single server.

 SAN may be isolated from the mainstream messaging network through

protocol segregation (Fibre Channel) or network segregation (virtual LAN, or
VLAN).

SAN infrastructure has provided a foundation for new storage applications

such as server clustering, LAN-free and server-free tape backup, and storage
consolidation. ME CSE [sem-II] 32
Fig 8: A simple storage area network

ME CSE [sem-II] 33
Fibre Channel provides both 1Gbps and 2Gbps speeds and so offers an
advantage in raw bandwidth for device Attachment.

SANs are a new type of network optimized for storage.

SANs require new management tools to supervise new storage relationships.

ME CSE [sem-II] 34
Fibre channel internals
Fibre Channel standards define a multilayered architecture for moving data
across the network.

These layers are numbered from FC-0 to FC-4.

Fibre Channel is a gigabit transport, with current implementations at 1Gbps

and 2Gbps.

Fibre Channel layers fall within the lower levels of the OSI model and can be
considered a link layer network

The standards provide definitions :

Physical-layer attributes
 Transport controls
 Upper-level support of TCP/IP
 SCSI-3
 High Performance Parallel Interface (HiPPI)
 Other protocols. ME CSE [sem-II] 35
 Layer Functions Contents
Number

FC-4 Upper layer protocol Ex: SCSI-3, IP

Layer Functions Contents
interface
Number
FC-3FC-4 Common services
Upper Under construction
layer protocol
interface (Encryption and compression)
FC-3 Common services
FC-2 Data Delivery Framing , flow control, class
FC-2 Data Delivery services
FC-1 Ordered sets/bytes
encoding
FC-1FC-0 Ordered sets/bytes
Physical interface 8b/10b encoding , link
encoding controls

FC-0 Physical Interface Optical/ electrical, cable

plant

Fibre Channel Layers

ME CSE [sem-II] 36
Fibre Channel standards define a multilayered architecture’s Functions:

FC-4:- Establishes the interface between Fibre Channel and upper-

level applications.
Example: For host bus adapters, this function is typically fulfilled by the
device driver provided by the vendor.

FC-3 layer :- Under development

Example: It may include facilities for data encryption and compression.

 FC-2 layer:- Blocks of data handed by the upper-level application

Example: various classes of services and flow control mechanisms.

FC-1 and FC-0—Focus on the actual transport of data across the network.

FC-1 :- Facilities for encoding & decoding data for shipment at gigabit speeds
and defines the command structure for accessing the media.

 FC-0:- Establishes standards for various media types, allowable lengths, and
signaling.
Fibre Channel's layered architecture is implemented on three transport
topologies:

Point-to-point: -
A dedicated connection between two devices only, typically a server
and a disk.

Arbitrated loop: -
A shared medium, similar to Token Ring or Fiber Distributed Data
Interface (FDDI), and employs a special superset of commands to control
access to the medium by multiple devices.

Switched fabric: -
one or more switches providing higher-level services and switched
bandwidth of 100MBps or 200MBps (200MBps and 400MBps full duplex)
per port.

ME CSE [sem-II] 38
 Data Encoding
Proper transmission and reception of a digital bit stream would not be
possible if raw bytes of data were simply herded into a shift register and
shoved one bit at a time onto the transport.

Without a byte encoding scheme : it would be impossible to detect when one

bit ended and another began—in other words, it would be impossible to
recover the clocking used to issue the stream.

converts each 8-bit byte into two possible 10-bit characters.

Example:- Raw stream of hex FF bytes
1111 1111 = 101011 0001 or 010100 1110 (The 8b/10b mechanism)

Each of the two 10-bit products of this conversion should have no more than
six total 1's or 0's, and about half will have an equal number of 1's and 0's.
The 10-bit characters with more 1's than 0's have positive disparity.
With more 0's than 1's have negative disparity.
Balance of 1's and 0's results in neutral disparity.
ME CSE [sem-II] 39
Fibre Channel uses one of these nondata characters as a special command
character. Known as the K28.5 command character.

Unique comma sequence is used by data recovery circuitry as a trigger for

defining the boundaries of 10-bit characters.

ME CSE [sem-II] 40
 Ordered Sets
To move data across the network, Fibre Channel uses a concise command
syntax referred to as ordered sets.

Ordered sets fall into three general categories: start of frame, end of frame,
and primitive signals.

One subset is used to indicate the start and end of data frames and the class of
service used to ship the frame.

A group of ordered sets known as primitive signals is used to indicate actions

or events on the transport.

The start of frame (SOF) delimiter K28.5 D21.5 D22.2 D22.2, for example,
is used in a frame header to initiate a connectionless level of service.

An end of frame (EOF) ordered set—for example, K28.5 D21.4 D21.6
D21.6—would indicate that the transmission of this frame was complete.
ME CSE [sem-II] 41
Fibre Channel defines 11 different types of SOF ordered sets, and 8 EOF
words.

The specific SOF or EOF used informs the recipient as to what further action
(such as an acknowledgment).

Primitive sequences require that at least three consecutive words be

recognized before any action is taken.

supersets of primitive signals and primitive sequences have been defined

specifically for arbitrated loop and Fibre Channel fabric operation.

ME CSE [sem-II] 42
Framing Protocol

To move data from one Fibre Channel-attached device to another, the data
blocks handed down by the upper-layer protocol of the sender must be
organized into discrete packets for shipment across the transport.
Frames are always prefaced with an ordered set start of frame (SOF)
delimiter.
single 4-byte word that defines the class of service used and specifies
whether the frame is the first in a series or simply one in a series of related
frames.
24-byte frame header contains destination and source addressing, as
well as control fields that indicate the frame's content (control
information or data type) and position within a series of sequential
frames.
Data integrity within the frame is verified with an 32-bit cyclic redundancy
check (CRC). ME CSE [sem-II] 43
Classes of Service
To ensure proper data transport, applications may require different levels of
delivery guarantees, bandwidth, and connectivity between communicators.
Fibre Channel standards provide five classes of delivery service.
Class 1 service defines a dedicated connection between two devices.
a file server and a disk array with acknowledgment of frame delivery
Class 2 service does not require a dedicated connection between talking
pairs but it does provide acknowledgment of frame delivery.
Class 3 service is connectionless, but unlike Class 2, there is no notification
of delivery.
Similar to datagram services in LAN topologies (for example, User
Datagram Protocol/Internet Protocol, or UDP/IP).
Class 4 service introduces the concept of virtual circuits to Fibre Channel
architecture.
The switch must maintain and monitor potentially hundreds of virtual
circuits. Ex:- SNMP.
Class 6 service has been defined to provide multicast service with
acknowledged delivery.
ME CSE [sem-II] 44
Flow Control
Fibre Channel provides several flow control mechanisms based on a system
of credits.

Credit scheme is based on the number of buffers that an end node maintains
for storing incoming data.

Transmission credit is initially established when two communicating nodes

log in and exchange communication parameters.
End-to-end flow control (EE_Credit).

Buffer-to-buffer credit (BB_Credit) is used by Class 2 and Class 3 service

and relies on the receiver-ready (R_RDY) ordered set to replenish credits.

ME CSE [sem-II] 45
Naming and Addressing Conventions
The unique identity of participants in a Fibre Channel environment is
maintained through a hierarchy of fixed names and assigned address
identifiers.

In Fibre Channel terminology, a communicating device is a node.

Each node has a fixed 8-byte node name assigned by the manufacturer.
Institute of Electrical and Electronics Engineers (IEEE) for a range of
addresses.
World-Wide Name (WWN).

The 64-bit name provides a unique identity throughout the SAN.

The switch maintains a table of the device's WWN and the assigned 24-bit
address in the Simple Name Server (SNS).

ME CSE [sem-II] 46
Thank U…~!~

ME CSE [sem-II] 47