You are on page 1of 52

Obligatory rubric

Copyright 2000,2003 Storage Networking Industry Association (SNIA).


SNIA-TC members may use this material freely; all others must conform to usage notes in the speaker noted for this slide. See SNIA-SSM-colors.ppt for printing and color/graphics conventions.

Copyright 2000,2003, Storage Networking Industry Association

SNIA shared-storage model


A work in progress

An architectural overview
This revision: 2001-06-05 last content update 2003-04-13 last graphics update
Copyright 2000,2003, Storage Networking Industry Association

Contents
Purpose
The SNIA storage model
Layers, functions, and services Networks and interfaces

Applying the SNIA storage model


Common storage architectures

Conclusions

Copyright 2000,2003, Storage Networking Industry Association

Purpose
Present a simple model for shared storage architectures
Use it to describe common examples graphically Expose, for each one:
What services are provided, where Where interoperability is required [future] Pros and cons of the architecture

Copyright 2000,2003, Storage Networking Industry Association

Benefits
A common architecture vocabulary
Reference comparisons between common solutions

Help to align the industry


Customers can better structure their choices Vendors can better explain their differences

Copyright 2000,2003, Storage Networking Industry Association

What the model is and is not


It is not:
A specification, an architecture, a design, a product, a recommendation, or an installation

It is:
A framework that captures the functional layers and properties of a storage system

Copyright 2000,2003, Storage Networking Industry Association

Classic storage model


Application Storage domain: anything goes!
Appliance?

NAS?

Network?

Array? JBOD?

Data mover?

Copyright 2000,2003, Storage Networking Industry Association

The SNIA shared storage model


Application
File/record layer

Storage domain

Database (dbms)

File system (FS)

Host
Network

Block aggregation

Device

Storage devices (disks, )

Block layer

Copyright 2000,2003, Storage Networking Industry Association

Services

The SNIA storage model:


File/record layer
File/record layer
Database (dbms) File system (FS)

Copyright 2000,2003, Storage Networking Industry Association

The SNIA storage model:


File/record layer functions
Aka access methods
File system, database

Primary responsibility: packing many smaller things into a few larger ones
Fine-grain naming, space allocation

Secondary responsibilities
Caching for performance Coherency in distributed systems

10 Copyright 2000,2003, Storage Networking Industry Association

The SNIA storage model:


Block layer

Block aggregation

Storage devices (disks, )

Block layer
11 Copyright 2000,2003, Storage Networking Industry Association

The SNIA storage model:


Block layer functions
Storage devices storing data
disk drives, tape drives, solid-state disks,

Block aggregation address mapping


in-SN aggregation, or virtualization slicing & concatenation, striping local & remote mirroring, RAID-n

Examples
volume managers disk array LUs

Secondary responsibilities
caching
12 Copyright 2000,2003, Storage Networking Industry Association

The SNIA storage model:


Access path examples
Application
File/record layer
Database (dbms) File system (FS)

Note: all 8 possible paths can be used!

Block aggregation

Storage devices (disks, )

Block layer
13 Copyright 2000,2003, Storage Networking Industry Association

Block layer
Block-mapping functions: what can be done
Functional decomposition: where it can be done Sample architectures

Block aggregation
Storage devices (disks, )

Block layer

14 Copyright 2000,2003, Storage Networking Industry Association

Block layer
What can be done
Space management
making a large store from many small ones packing many small stores into one large one

Striping
for performance (load balancing, throughput)

Redundancy
full (local & remote mirroring, RAID-1, -10, ) partial (RAID-3, -4, -5, ) point-in-time copy

15 Copyright 2000,2003, Storage Networking Industry Association

Block layer
Where it can be done
Host-side
logical volume managers device drivers, HBAs
Host Network

Block aggregation

Device

SN-based
HBAs, specialized SN appliances

Storage devices

Device-based
array controllers (e.g., RAID) disk controllers (e.g., sparing)

16 Copyright 2000,2003, Storage Networking Industry Association

Block layer
How it is done

Building blocks
input: output: vector of blocks vector of blocks
Block aggregation

Result: building blocks can be stacked


enables the 3 layer model for the block layer layers can be nested on one another could be extended to more layers

17 Copyright 2000,2003, Storage Networking Industry Association

Block layer
Sample architectures
Application
File/record layer
Host. with LVM (+ software RAID?) Host, no LVM

1. Direct-attach 2. SN-attach 3. SN aggregation

Host with LVM

Host block-aggregation Aggregation appliance

SN

Network block-aggregation

Block layer

Disk array

Device block-aggregation

Storage devices
18 Copyright 2000,2003, Storage Networking Industry Association

File/record layer
File/record layer

Byte-mapping functions: what can be done


Functional decomposition: where it can be done Sample architectures

Database (dbms)

File system (fs)

19 Copyright 2000,2003, Storage Networking Industry Association

File/record layer
What can be done
Database management systems
tuples tables tables table-spaces table-spaces volume

File systems
files volume

New types
http caches: a kind of distributed file system?

20 Copyright 2000,2003, Storage Networking Industry Association

File/record layer
Where it can be done
Host with local FS/dbms

Host-side
file systems and databases NFS, CIFS, etc. are client-server splits inside the file system

SN-based
NAS head
NAS

Device-based
NAS functions in array box
device

21 Copyright 2000,2003, Storage Networking Industry Association

Host with NFS/CIFS client

File/record layer
Application
File/record layer
Host. with LVM and software RAID Host Host. with LVM Host LAN NAS head NAS server

1. Direct-attach 2. SN-attach

3. NAS head 4. NAS server

Host block-aggregation

SN

Network block-aggregation

Block layer

Disk array

Device block-aggregation

22 Copyright 2000,2003, Storage Networking Industry Association

The SNIA storage model


A layered view
IV. Application
III. File/record layer
IIIa. Database IIIb. File system
IV
III IIc IIb IIa I
Application
File/record layer
Database (dbms) File system (fs)

Host Network

II. Block aggregation


IIa. Host IIb. Network IIc. Device

Block aggregation

Device

Storage devices

I. Storage devices

23 Copyright 2000,2003, Storage Networking Industry Association

The SNIA storage model


Services subsystem
Application
File/record layer

Storage domain

Database (dbms)

File system (fs)

Block aggregation

Storage devices (disks, )

Block layer

24 Copyright 2000,2003, Storage Networking Industry Association

Discovery, monitoring Resource mgmt, configuration Security, billing Redundancy mgmt (backup, ) High availability (fail-over, ) Capacity planning

Services

Services
Operations off the critical path
naming, discovery, monitoring, configuration, security, billing, redundancy management (backup, ), high availability management (fail-over, ), capacity planning, strong ties into system-wide management services

Vital for successful operation


and a major opportunity for SNIA but not discussed further in this presentation

25 Copyright 2000,2003, Storage Networking Industry Association

Caching
can be added to almost any layer
Application
File/record layer
Host. with LVM and software RAID Host Host. with LVM Host
Cache

Ideally, caching only affects performance. But: coherency implications do affect management (protocols, interfaces)

LAN NAS head


Cache

NAS head

Cache

Host block-aggregation

SN

Cache appliance

Network block-aggregation

Block layer

Disk array
Cache
Cache

Device block-aggregation

26 Copyright 2000,2003, Storage Networking Industry Association

Clustering
Inter-box aggregation
Application
File/record layer
Host. with LVM and software RAID Host Host. with LVM Host LAN NAS head Purposes: load spreading across peers (scalability) alternate paths (high availability, scalability)

Cluster NAS FS
head Host block-aggregation

Multi-node LVM
SN

Network block-aggregation

Block layer

Disk array

Device block-aggregation

27 Copyright 2000,2003, Storage Networking Industry Association

Q: Data versus storage? A: Putting information into containers


user: application:
application: file system: file system: volume system:

data (learning my preferences) container(user keystroke history)


data (user keystroke history file) container (byte vector) data (a named file) container (blocks in volume)

volume system: data (replicated, striped layout) disk array: container (blocks in LU)

28 Copyright 2000,2003, Storage Networking Industry Association

Sharing
Content sharing or resource sharing?
Content sharing (logical, data)
contents accessed and understood by multiple clients
e.g., file system, Oracle Parallel Server dbms

some of the hard issues:


coherency heterogeneous data formats

Resource sharing (container, physical)


e.g., disk array where hosts access disjoint LUs

29 Copyright 2000,2003, Storage Networking Industry Association

Sharing
Content sharing and resource sharing
Application
Data sharing

File/record layer

Host Host with LVM Host with LVM

Host LAN

NAS system is shared and so are the files

NAS system

Host block-aggregation

SN

Network block-aggregation
Resource sharing Array is shared, but LUs are disjoint

Block layer

Device block-aggregation

30 Copyright 2000,2003, Storage Networking Industry Association

Networks and interfaces


are pervasive in the model
Application
API Bus

Operating System
File layer Block layer Storage device
31 Copyright 2000,2003, Storage Networking Industry Association

Full benefits come only from common, open interfaces

Network

Network or interface

Networks and interfaces


Composition and scaling
API API Bus

Network or interface

Network

Network or interface

API

Network or interface
Network

Network or interface

Bus

Open interfaces allow: 1. vertical composition 2. horizontal scaling 3. supplier independence

32 Copyright 2000,2003, Storage Networking Industry Association

Networks and interfaces


Open interfaces require
Well defined:
functions (what they do) interface protocols (data formats) access protocols (system call, RPC, flow control, )
Network
API Bus

That are:
published supported by multiple products => standards (which is where SNIA comes in)

33 Copyright 2000,2003, Storage Networking Industry Association

Q: SAN versus NAS?


A: a poorly-formed question
Files Blocks

Q: hardware: FibreChannel vs Ethernet vs InfiniBand?


Q: API: blocks vs files (aka NAS) vs objects (OSD)?

SAN iSCSI ?
FC

NAS
Enet

Q: protocol: FCP vs TCP/IP vs ?


A: (to all the above) it depends
Files Blocks FC

Storage network (SN):


any (mostly) dedicated network, installed (mostly) for storage traffic whatever the hardware, API, or protocol
34 Copyright 2000,2003, Storage Networking Industry Association

Enet

Some common storage architectures

Mapping the SNIA model onto some current implementations

Copyright 2000,2003, Storage Networking Industry Association

Direct-attach block storage


Application
File/record layer
Host. with LVM and software RAID Host, no LVM

Direct-attach

Multi-attach box
E.g., SCSI Host with LVM

Host block-aggregation

Network block-aggregation

Block layer

Disk array

Device block-aggregation

36 Copyright 2000,2003, Storage Networking Industry Association

SN-attached block storage


Application
File/record layer
Host, no LVM

SN = any network used for storage access.


E.g., Fibre Channel, Ethernet,

Host with LVM

Host block-aggregation

SN

Network block-aggregation

Block layer

Disk array

Device block-aggregation

37 Copyright 2000,2003, Storage Networking Industry Association

SN-attached block storage


with metadata server
Application
File/record layer
Host, no LVM

Aggregation map (metadata) is supplied by an external server

Host with LVM

Host block-aggregation Block-aggregation metadata server

SN

Network block-aggregation

Block layer

Disk array

Device block-aggregation

38 Copyright 2000,2003, Storage Networking Industry Association

Block storage aggregation


in a storage network appliance
Application
File/record layer
Host, no LVM

aka SN virtualization
Functions: LVM, caching

Host with LVM

Host block-aggregation Aggregation appliance

SN

Network block-aggregation

Block layer

Device block-aggregation Disk array

39 Copyright 2000,2003, Storage Networking Industry Association

Multi-site block storage


Application
File/record layer
Host. with LVM

Application
Host. with LVM

Functions: point-in-time copy, caching, local & remote mirroring,

Host-to-host SN appliance Device-to-device


Host block-aggregation
Aggregation appliance

WAN
Aggregation appliance

Block layer

SN

WAN

Network block-aggregation

Device block-aggregation Disk array

WAN

Disk array

40 Copyright 2000,2003, Storage Networking Industry Association

File server
Application
File/record layer
Host Host LAN

LAN-attached NAS system


May do SN/block aggregation, etc. inside in the NAS system box

NAS system private SN?

Host block-aggregation

Network block-aggregation

Block layer

Device block-aggregation

41 Copyright 2000,2003, Storage Networking Industry Association

File server controller


(NAS head)
Application
File/record layer
Host Host LAN

No storage in the file server controller box

NAS head

Host block-aggregation

SN

Network block-aggregation

Block layer

Disk array

Device block-aggregation

42 Copyright 2000,2003, Storage Networking Industry Association

NAS/file server metadata manager (asymmetric)


Application
File/record layer
Host. with LVM
Host LAN File system metadata File metadata Block accesses Disk array SN

Hosts get file metadata from FS/NAS controller, then access the data directly
FS controller can also be NAS server Host block-aggregation

Network block-aggregation

Block layer

Device block-aggregation

43 Copyright 2000,2003, Storage Networking Industry Association

Object-based Storage Device


(OSD), CMU NASD
Application
File/record layer
Host Host LAN Object metadata Host block-aggregation File metadata

Network block-aggregation

Block layer

OSD device

Security metadata

Device block-aggregation

44 Copyright 2000,2003, Storage Networking Industry Association

Summary & conclusions

Copyright 2000,2003, Storage Networking Industry Association

The SNIA shared storage model


Application
File/record layer

Storage domain

Database (dbms)

File system (FS)

Host
Network

Block aggregation

Device

Storage devices (disks, ) Block layer

46 Copyright 2000,2003, Storage Networking Industry Association

Discovery, monitoring Resource mgmt, configuration Security, billing Redundancy mgmt (backup, ) High availability (fail-over, ) Capacity planning

Services

Block layer
Sample architectures
Application
File/record layer
Host. with LVM and software RAID Host. with LVM Host, no LVM

1. Direct-attach 2. SN-attach 3. SN aggregation

Host block-aggregation Aggregation appliance

SN

Network block-aggregation

Block layer

Disk array

Device block-aggregation

47 Copyright 2000,2003, Storage Networking Industry Association

File/record layer
Sample architectures
Application
File/record layer
Host. with LVM and software RAID Host Host. with LVM Host LAN NAS head NAS server

1. Direct-attach 2. SN-attach

3. NAS head 4. NAS server

Host block-aggregation

SN

Network block-aggregation

Block layer

Disk array

Device block-aggregation

48 Copyright 2000,2003, Storage Networking Industry Association

Uses for the model


Vendors
place products in the space of architectures clarify product differences

Customers
understand vendor offerings better

The industry
basis for common definitions, communication, understanding, interoperability

49 Copyright 2000,2003, Storage Networking Industry Association

Conclusions
The SNIA shared storage model is both simple and useful
to highlight similarities and differences as a basis for comparisons

Still a work in progress


data movers, tape drives, better comparisons suggestions?

The SNIA-TC welcomes input:


<snia-tc@snia.org>

50 Copyright 2000,2003, Storage Networking Industry Association

Authors
John Wilkes, Hewlett-Packard (project lead) Harald Skardal, NetApps David Black, EMC Wayne Rickard (SNIA TC chairperson), Gadzoox Co-conspirators: the rest of the SNIA Technical Council
Dave Anderson, Seagate Technology Jim Carlson, IBM Garth Gibson, CMU/Panasas Kevin Phaup, HighGround Systems David Thiel, Compaq

51 Copyright 2000,2003, Storage Networking Industry Association