Вы находитесь на странице: 1из 52

Obligatory rubric

• Copyright © 2000,2003 Storage Networking


Industry Association (SNIA).
• SNIA-TC members may use this material freely; all
others require permission from the SNIA.

• See the speaker notes for this slide for important


usage information.
• See SNIA-SSM-colors.ppt for printing and
color/graphics conventions.

1 Copyright © 2000,2003, Storage Networking Industry Association


SNIA shared-storage model
A work in progress …

An architectural overview
This revision:
• 2001-06-05 last content update
• 2003-04-13 last graphics update

Copyright © 2000,2003, Storage Networking Industry Association


Contents

• Purpose
• The SNIA storage model
ƒ Layers, functions, and services
ƒ Networks and interfaces
• Applying the SNIA storage model
ƒ Common storage architectures
• Conclusions

3 Copyright © 2000,2003, Storage Networking Industry Association


Purpose

• Present a simple model for shared storage


architectures
• Use it to describe common examples
graphically
• Expose, for each one:
ƒ What services are provided, where
ƒ Where interoperability is required
ƒ [future] Pros and cons of the architecture

4 Copyright © 2000,2003, Storage Networking Industry Association


Benefits

• A common “architecture vocabulary”


• Reference comparisons between common
solutions
• Help to align the industry
ƒ Customers can better structure their choices
ƒ Vendors can better explain their differences

5 Copyright © 2000,2003, Storage Networking Industry Association


What the model is and is not

• It is not:
ƒ A specification, an architecture, a design, a product,
a recommendation, or an installation

• It is:
ƒ A framework that captures the functional layers and
properties of a storage system

6 Copyright © 2000,2003, Storage Networking Industry Association


Classic storage model

Application

Storage domain:
“anything goes!”
NAS?
Appliance?
Network?

Data
Array?
mover?

JBOD?

7 Copyright © 2000,2003, Storage Networking Industry Association


The SNIA shared storage model

Application
File/record layer
Storage domain

Database File system


(dbms) (FS)

Services
Host

Network
Block
aggregation Device

Storage
Storage devices
devices (disks,
(disks, …)
…)

Block layer
8 Copyright © 2000,2003, Storage Networking Industry Association
The SNIA storage model:
File/record layer

File/record layer

Database File system


(dbms) (FS)

9 Copyright © 2000,2003, Storage Networking Industry Association


The SNIA storage model:
File/record layer — functions

• Aka “access methods”


ƒ File system, database
• Primary responsibility: packing many
smaller things into a few larger ones
ƒ Fine-grain naming, space allocation
• Secondary responsibilities
ƒ Caching for performance
ƒ Coherency in distributed systems

10 Copyright © 2000,2003, Storage Networking Industry Association


The SNIA storage model:
Block layer

Block
Block aggregation
aggregation

Storage
Storage devices
devices (disks,
(disks, …)
…)

Block layer
11 Copyright © 2000,2003, Storage Networking Industry Association
The SNIA storage model:
Block layer — functions

• Storage devices – storing data


ƒ disk drives, tape drives, solid-state disks, …
• Block aggregation – address mapping
ƒ in-SN aggregation, or “virtualization”
ƒ slicing & concatenation, striping
ƒ local & remote mirroring, RAID-n
• Examples
ƒ volume managers
ƒ disk array LUs
• Secondary responsibilities
ƒ caching

12 Copyright © 2000,2003, Storage Networking Industry Association


The SNIA storage model:
Access path examples
Application
File/record layer Note:
Note: all
all 88
possible
possible paths
paths
Database File system can
can be
be used!
used!
(dbms) (FS)

Block
Block aggregation
aggregation

Storage
Storage devices
devices (disks,
(disks, …)
…)

Block layer
13 Copyright © 2000,2003, Storage Networking Industry Association
Block layer

• Block-mapping functions:
what can be done Block
Block
aggregation
aggregation

• Functional decomposition:
Storage
Storage devices
devices (disks,
(disks, …)
…)

Block
Block layer
layer
where it can be done
• Sample architectures

14 Copyright © 2000,2003, Storage Networking Industry Association


Block layer
What can be done

• Space management
ƒ making a large store from many small ones
ƒ packing many small stores into one large one
• Striping
ƒ for performance (load balancing, throughput)
• Redundancy
ƒ full(local & remote mirroring, RAID-1, -10, …)
ƒ partial (RAID-3, -4, -5, …)
ƒ point-in-time copy

15 Copyright © 2000,2003, Storage Networking Industry Association


Block layer
Where it can be done

• Host-side
ƒ logical volume managers Host

Network
ƒ device drivers, HBAs Block
Block
aggregation
aggregation Device

• SN-based Storage
Storage devices
devices

ƒ HBAs, specialized SN appliances


• Device-based
ƒ array controllers (e.g., RAID)
ƒ disk controllers (e.g., sparing)

16 Copyright © 2000,2003, Storage Networking Industry Association


Block layer
How it is done

• Building blocks
Block
Block
ƒ input: vector of blocks aggregation
aggregation

ƒ output: vector of blocks


• Result: building blocks … …

can be stacked
ƒ enables the 3 layer model for the
block layer
ƒ layers can be nested on one another
ƒ could be extended to more layers

17 Copyright © 2000,2003, Storage Networking Industry Association


Block layer
Sample architectures
1. Direct-attach
Application 2. SN-attach
3. SN aggregation
File/record

(+ software RAID?)
layer

Host. with LVM

no LVM
Host,
with LVM
Host

Host block-aggregation

Aggregation
appliance Network block-aggregation
SN
Block
layer

Device block-aggregation
Disk array

Storage devices

18 Copyright © 2000,2003, Storage Networking Industry Association


File/record layer

File/record
File/record layer
layer

• Byte-mapping functions:
Database File system
(dbms) (fs)

what can be done


• Functional decomposition:
where it can be done
• Sample architectures

19 Copyright © 2000,2003, Storage Networking Industry Association


File/record layer
What can be done

• Database management systems


ƒ tuples → tables
ƒ tables → table-spaces
ƒ table-spaces → volume

• File systems
ƒ files → volume
• New types
ƒ http caches: a kind of distributed file system?

20 Copyright © 2000,2003, Storage Networking Industry Association


File/record layer
Where it can be done

FS/dbms
Host with local

NFS/CIFS
Host with
• Host-side

client
ƒ file systems and databases
ƒ NFS, CIFS, etc. are client-server
splits inside the file system
• SN-based
NAS
ƒ NAS head
• Device-based
ƒ NAS functions in array box
device

21 Copyright © 2000,2003, Storage Networking Industry Association


File/record layer
1. Direct-attach
Application 2. SN-attach
3. NAS head
File/record

Host Host
4. NAS server
and software RAID
layer

Host. with LVM

Host. with LVM


LAN

NAS
head Host block-aggregation

NAS
Network block-aggregation
SN server
Block
layer

Device block-aggregation
Disk array

22 Copyright © 2000,2003, Storage Networking Industry Association


The SNIA storage model
A layered view

IV. Application
IV Application
III. File/record layer File/record
File/record layer
layer
IIIa. Database III Database
(dbms)
File system
(fs)

IIIb. File system


IIc Host

II. Block aggregation IIb


Block
Block
Network

IIa. Host IIa aggregation


aggregation Device

IIb. Network I Storage


Storage devices
devices
IIc. Device
I. Storage devices

23 Copyright © 2000,2003, Storage Networking Industry Association


Storage domain

(dbms)
Database

Storage
Block

Storage devices

24 Copyright © 2000,2003, Storage Networking Industry Association


Application
Services subsystem

Block layer
Block aggregation
aggregation

devices (disks,
File/record layer

(fs)

(disks, …)
…)
File system
The SNIA storage model

Services
Discovery, monitoring
Discovery, monitoring
Resource
Resource mgmt, configuration
mgmt, configuration
Security, billing
Security, billing
Redundancy
Redundancy mgmt
mgmt (backup, …)
(backup, …)
High
High availability
availability (fail-over, …)
(fail-over, …)
Capacity planning
Capacity planning
Services

• Operations off the critical path


ƒ naming, discovery, monitoring, configuration, security, billing,
redundancy management (backup, …), high availability
management (fail-over, …), capacity planning, …
ƒ strong ties into system-wide management services

• Vital for successful operation


ƒ and a major opportunity for SNIA …
ƒ … but not discussed further in this presentation

25 Copyright © 2000,2003, Storage Networking Industry Association


Caching
… can be added to almost any layer
Application Ideally,
Ideally, caching
caching only
only
affects
affects performance.
performance.
File/record

Host Host But:


But: coherency
coherency
and software RAID

Cache implications
implications dodo affect
affect
layer

Host. with LVM

Host. with LVM


LAN management
management
(protocols,
(protocols, interfaces)
interfaces)

NAS NAS
head head Host block-aggregation
Cache Cache

Cache Network block-aggregation


SN appliance
Block
layer

Device block-aggregation
Disk array
Cache

Cache

26 Copyright © 2000,2003, Storage Networking Industry Association


Clustering
Inter-box aggregation
Application Purposes:
Purposes:
•• load
load spreading
spreading across
across
peers
peers (scalability)
File/record

Host Host (scalability)


•• alternate
alternate paths
paths (high
and software RAID

(high
layer

Host. with LVM

Host. with LVM


availability,
availability, scalability)
scalability)
LAN

NAS Cluster NAS


FS
head head Host block-aggregation
Multi-node LVM

Network block-aggregation
SN
Block
layer

Device block-aggregation
Disk array

27 Copyright © 2000,2003, Storage Networking Industry Association


Q: Data versus storage?
A: Putting information into containers
ƒ user: data (“learning my preferences”)
application: container(“user keystroke history”)

ƒ application: data (“user keystroke history file”)


file system: container (“byte vector”)

ƒ file system: data (“a named file”)


volume system: container (“blocks in volume”)

ƒ volume system: data (“replicated, striped layout”)


disk array: container (“blocks in LU”)

28 Copyright © 2000,2003, Storage Networking Industry Association


Sharing
Content sharing or resource sharing?

• Content sharing (“logical”, “data”)


ƒ contents accessed and understood by multiple
clients
• e.g., file system, Oracle Parallel Server dbms
ƒ some of the hard issues:
• coherency
• heterogeneous data formats

• Resource sharing (“container”, “physical”)


ƒ e.g., disk array where hosts access disjoint LUs

29 Copyright © 2000,2003, Storage Networking Industry Association


Sharing
Content sharing and resource sharing
Application
Data
Data sharing
sharing
File/record

Host Host NAS


NAS system
system isis
shared
shared –– andand so
so
layer

LAN are
are the
the files
files
with LVM

with LVM
Host

Host

NAS system Host block-aggregation

Network block-aggregation
SN
Resource
Resource
Block
layer

sharing
sharing
Device block-aggregation
Array
Array is
is
shared,
shared,
but
but LUs
LUs are
are
disjoint
disjoint

30 Copyright © 2000,2003, Storage Networking Industry Association


Networks and interfaces
are pervasive in the model API

Bus

Application
Network
Full benefits
Operating System come only from
common, open
interfaces
File layer

Block layer

Network or interface
Storage device

31 Copyright © 2000,2003, Storage Networking Industry Association


Networks and interfaces
Composition and scaling API

Bus

API
Network or interface Network

API
Network or interface
Open interfaces
allow:
Network or interface
1. vertical
Network
composition
Network or interface Bus 2. horizontal scaling
3. supplier
independence
32 Copyright © 2000,2003, Storage Networking Industry Association
Networks and interfaces
Open interfaces require … API

Bus

• Well defined:
ƒ functions (what they do) Network

ƒ interface protocols (data formats)


ƒ access protocols
(system call, RPC, flow control, …)
• That are:
ƒ published
ƒ supported by multiple products
=> standards (which is where SNIA comes in)

33 Copyright © 2000,2003, Storage Networking Industry Association


Q: “SAN” versus “NAS”?
A: a poorly-formed question

Files Blocks
“SAN” iSCSI
• Q: hardware: FibreChannel vs
Ethernet vs InfiniBand? ? NAS

• Q: API: blocks vs files (aka “NAS”) FC E’net

vs objects (OSD)?
• Q: protocol: FCP vs TCP/IP vs … ?

Files Blocks
• A: (to all the above) it depends …

FS
/b
lo
ck
co
m
• Storage network (SN):

bo
FC E’net
ƒ any (mostly) dedicated network,
installed (mostly) for storage traffic
ƒ whatever the hardware, API, or protocol
34 Copyright © 2000,2003, Storage Networking Industry Association
Some common storage
architectures

Mapping the SNIA model onto some


current implementations

Copyright © 2000,2003, Storage Networking Industry Association


Direct-attach block storage

Application •• Direct-attach
Direct-attach
•• Multi-attach
Multi-attach box
box
File/record

and software RAID E.g.,


E.g., SCSI
SCSI
layer

Host. with LVM

no LVM
Host,

with LVM
Host
Host block-aggregation

Network block-aggregation
Block
layer

Device block-aggregation
Disk array

36 Copyright © 2000,2003, Storage Networking Industry Association


SN-attached block storage

Application “SN”
“SN” == any
any network
network
used for storage
used for storage
access.
access.
File/record

E.g.,
E.g., Fibre
Fibre Channel,
Channel,
layer

Ethernet, …
no LVM
Host, Ethernet, …

Host with
LVM
Host block-aggregation

Network block-aggregation
SN
Block
layer

Device block-aggregation
Disk array

37 Copyright © 2000,2003, Storage Networking Industry Association


SN-attached block storage
with metadata server
Application Aggregation
Aggregation map
map
(metadata) is
(metadata) is
supplied
supplied by
by an
an
File/record

external
external server
server
layer

no LVM
Host,

Host with
LVM
Host block-aggregation

Block-aggregation Network block-aggregation


SN metadata server
Block
layer

Device block-aggregation
Disk array

38 Copyright © 2000,2003, Storage Networking Industry Association


Block storage aggregation
in a storage network appliance
Application aka
aka “SN
“SN
virtualization”
virtualization”
File/record

Functions:
Functions: LVM,
LVM, caching
caching
layer

no LVM
Host,
with LVM
Host

Host block-aggregation

Aggregation
appliance Network block-aggregation
SN
Block
layer

Device block-aggregation
Disk array

39 Copyright © 2000,2003, Storage Networking Industry Association


Multi-site
block storage Functions:
Functions: point-in-time
point-in-time
copy, caching,
copy, caching,
local
local &
& remote
remote mirroring,
mirroring, …

Application Application
Host-to-host
Host-to-host
File/record

Host. with LVM


Host. with LVM

SN
SN appliance
appliance
layer

Device-to-device
Device-to-device

WAN Host block-aggregation

Aggregation
WAN
Aggregation
appliance
Network block-aggregation
appliance
SN
Block
layer

Device block-aggregation
Disk array Disk array
WAN

40 Copyright © 2000,2003, Storage Networking Industry Association


File server

Application LAN-attached
LAN-attached
“NAS”
“NAS” system
system
File/record

Host Host May


May dodo SN/block
SN/block
aggregation,
aggregation, etc.
etc. inside
inside
layer

in
in the
the NAS
NAS system
system boxbox
LAN

NAS system Host block-aggregation

private Network block-aggregation


SN?
Block
layer

Device block-aggregation

41 Copyright © 2000,2003, Storage Networking Industry Association


File server controller
(“NAS head”)
Application No
No storage
storage in
in the
the
file
file server
server
controller
controller box
File/record

Host Host box


layer

LAN

NAS
head Host block-aggregation

Network block-aggregation
SN
Block
layer

Device block-aggregation
Disk array

42 Copyright © 2000,2003, Storage Networking Industry Association


NAS/file server metadata
manager (“asymmetric”)
Application Hosts
Hosts get
get file
file meta-
meta-
data from FS/NAS
data from FS/NAS
controller,
controller, then
then
File/record

Host. with LVM Host


access
LAN
access the
the data
data
layer

directly
directly
FS
FS controller
controller can
can
File system also be NAS server
also be NAS server
metadata
File Host block-aggregation
metadata

Block Network block-aggregation


SN
accesses
Block
layer

Device block-aggregation
Disk array

43 Copyright © 2000,2003, Storage Networking Industry Association


Object-based Storage Device
(OSD), CMU NASD
Application
File/record

Host Host
File
layer

metadata
LAN
Object
metadata
Host block-aggregation

Network block-aggregation
OSD
Block
layer

device Security
metadata Device block-aggregation

44 Copyright © 2000,2003, Storage Networking Industry Association


Summary & conclusions

Copyright © 2000,2003, Storage Networking Industry Association


Storage domain

(dbms)
Database

Block
Host

aggregation
Network

46 Copyright © 2000,2003, Storage Networking Industry Association


Block layer
Application
File/record layer

Device
(FS)

Storage devices (disks, …)


File system

Services
Discovery, monitoring
Discovery, monitoring
Resource
Resource mgmt, configuration
mgmt, configuration
Security, billing
Security, billing
Redundancy
Redundancy mgmt
mgmt (backup, …)
(backup, …)
High
High availability
availability (fail-over, …)
(fail-over, …)
The SNIA shared storage model

Capacity planning
Capacity planning
Block layer
Sample architectures
1. Direct-attach
Application 2. SN-attach
3. SN aggregation
File/record

and software RAID


layer

Host. with LVM

Host. with LVM

no LVM
Host,
Host block-aggregation

Aggregation
appliance Network block-aggregation
SN
Block
layer

Device block-aggregation
Disk array

47 Copyright © 2000,2003, Storage Networking Industry Association


File/record layer
Sample architectures
1. Direct-attach
Application 2. SN-attach
3. NAS head
File/record

Host Host
4. NAS server
and software RAID
layer

Host. with LVM

Host. with LVM


LAN

NAS
head Host block-aggregation

NAS
Network block-aggregation
SN server
Block
layer

Device block-aggregation
Disk array

48 Copyright © 2000,2003, Storage Networking Industry Association


Uses for the model

• Vendors
ƒ place products in the space of architectures
ƒ clarify product differences
• Customers
ƒ understand vendor offerings better
• The industry
ƒ basis for common definitions, communication,
understanding, interoperability

49 Copyright © 2000,2003, Storage Networking Industry Association


Conclusions

• The SNIA shared storage model is both simple and


useful
ƒ to highlight similarities and differences
ƒ as a basis for comparisons
• Still a work in progress
ƒ data movers, tape drives, …
ƒ better comparisons …
ƒ suggestions?
• The SNIA-TC welcomes input:
ƒ <snia-tc@snia.org>

50 Copyright © 2000,2003, Storage Networking Industry Association


Authors

• John Wilkes, Hewlett-Packard (project lead)


• Harald Skardal, NetApps
• David Black, EMC
• Wayne Rickard (SNIA TC chairperson), Gadzoox
• Co-conspirators: the rest of the SNIA Technical
Council
ƒ Dave Anderson, Seagate Technology
ƒ Jim Carlson, IBM
ƒ Garth Gibson, CMU/Panasas
ƒ Kevin Phaup, HighGround Systems
ƒ David Thiel, Compaq

51 Copyright © 2000,2003, Storage Networking Industry Association