Вы находитесь на странице: 1из 37

IBM Almaden Research Center

Storage Class Memory:


Technology, Systems and
Applications
Rich Freitas
freitas@almaden.ibm.com
Nonvolatile Memory Seminar
Hot Chips Conference
August 22, 2010
Memorial Auditorium, Stanford University

2010 IBM Corporation

IBM Almaden Research center

Agenda
Technology
Disks
Flash
Phase Change Memory
Systems
Memory Systems
Storage Systems
Applications

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Definition of Storage Class Memory SCM

A new class of data storage/memory devices


many technologies compete to be the best SCM

SCM features:
Non-volatile
Short Access times (~ DRAM like )
Low cost per bit (more DISK like by 2020)
Solid state, no moving parts

SCM blurs the distinction between


MEMORY (fast, expensive, volatile ) and
STORAGE (slow, cheap, non-volatile)
3

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

System Targets for SCM

Megacenters

Billions!

Mobile
4

Desktop X

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Datacenter
Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

History of HDD is based on Areal Density Growth

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

-6.5%

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Many device technologies considered for SCM


Phase Change RAM
most promising now (scaling)
Magnetic RAM
used today, but poor scaling and a space hog
Magnetic Racetrack
basic research, but very promising long term
Ferroelectric RAM
used today, but poor scaleability
Solid Electrolyte and resistive RAM (Memristor)
early development, promising
Organic, nano particle and polymeric RAM

bistable material plus


on-off switch

many different devices in this class, unlikely


Improved FLASH
still slow and poor write endurance

Generic SCM Array


9

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Emerging Memory Technologies


FLASH

FRAM

Extension
Trap Storage

Ramtron

Saifun NROM

Fujitsu

MRAM
IBM

PCRAM
Ovonyx

RRAM

Solid
Electrolyte

Polymer/
Organic

IBM

Axon

Spansion

Infineon

Samsung

Infineon

BAE

Sharp

Freescale

Intel

Unity

TFE

Tower

STMicro

Spansion

TI

Philips

STMicro

Spansion

MEC

Infineon

Toshiba

STMicro

Samsung

Samsung

Zettacore

Macronix

Infineon

HP

Samsung

Samsung

NVE

Toshiba

NEC

Honeywell

Spansion

Hitachi

Toshiba

Macronix

Rohm

NEC

HP

Sony

Nano-xtal

Cypress

Fujitsu

Freescale
Matsushita

Matsushita

NEC

Roltronics

IBM
Macronix

Nanolayer

Infineon
Hitachi
Philips

Renesas

Oki

Samsung

Hynix

Hynix

Celis

Elpida

TSMC

Fujitsu
Seiko Epson

64Mb FRAM (Prototype)


0.13um 3.3V

10

4Mb MRAM (Product)


0.18um 3.3V

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

512Mb PRAM (Prototype)


0.1um 1.8V

Hot Chips 8/2010

4Mb C-RAM (Product)


0.25um 3.3V

2010 IBM Corporation

IBM Almaden Research center

Research interest

Papers presented at
Symposium on VLSI Technology
IEDM (Int. Electron Devices Meeting)

Number of papers

60
organic

50

solid electrolyte
RRAM

40

PCram
MRAM

30

FeRAM
SONOS Flash

20

nanocrystal Flash
Flash

10
0
2001

2002

2003

2004

2005

2006

2007

2008

Year
11

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

What is Flash?
gate

drain

source

Floating
Gate

oxide

control gate
e- e-

source

Flash
Memory
1

drain
Flash
Memory
0

Floating control gate


e- e- e- e- eGate
drain
source
12

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Based on MOS transistor

Transistor gate is redesigned


Charge is placed or removed
near the gate
The threshold voltage Vth of the
transistor is shifted by the
presence of this charge
The threshold Voltage shift
detection enables non-volatile
memory function.
Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Feeds and Speeds for typical NAND Flash


NAND
Cell Size

4 F2
(2 F2 virtual x 2-bit
MLC)

Read Access Time

20-50 us

Read

15-25 MB/s

Write

5-8MB/sec

Erase

2ms

Start Up Time
Market Size (2007)
Applications
13

50-100 us
$14.2B
Multimedia

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Representative NAND Flash Device


Interface: one or two bytes wide
Data accessed in pages

2112, 4224 or 8448 Bytes

B
U
F

Power circuits

Power
Circuits

Block 63

Charge Pumps
Clock drivers
Etc.

Block 1

Data erased in blocks


Block = 64 - 128 Pages

BCA
UHC
FE

Page
Page
Page
Page
Page
Page
Page
Page
Page
Page
Page
Page

Block 0

Transition to ONFI for some vendors

ONFI Open NAND Flash Interface


14

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Representative Flash SSD Classes


250

100000

200
10000

150

Rd BW MB/s
Wt BW MB/s

1000

Rd IOPS
Wt IOPS

100

100
50

10
u sb

15

commercia l

en terprise

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

usb

commercial

Hot Chips 8/2010

enterprise

2010 IBM Corporation

IBM Almaden Research center

History of Phase-change memory


late 1960s Ovshinsky shows reversible electrical switching in disordered
semiconductors
early 1970s much research on mechanisms, but everything was too slow!

Crystalline phase
Electrical conductivity

Low resistance
High reflectivity

Amorphous phase
High resistance
Low reflectivity

[Neale:2001]

16

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

[Wuttig:2007]
2010 IBM Corporation

IBM Almaden Research center

Phase-change
RAM

Bit-line
PCRAM

programmable
resistor
Word
-line

Voltage

RESET pulse

Access device

(transistor, diode)

Potential headache:
Tmelt

High power/current

affects scaling!
temperature

SET
pulse

Tcryst

If crystallization is slow

time
17

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Potential headache:
affects performance!

Hot Chips 8/2010

2010 IBM Corporation

Almaden Research center


Paths IBM
to ultra-high
density memory

2F

add

2F

sub-lithographic fins
(N2 with 2-D)

starting
from standard

4F

N 1-D
(at IEDM 2005)

go to 3-D with

store M bits/cell
with 2M

demonstrated

L layers

multiple levels

demonstrated
(at IEDM 2007)
18

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Industry SCM activities


SCM research in IBM
Intel/ST-Microelectronics
spun out Numonyx (FLASH &
PCM)
Samsung, Numonyx sample
PCM chips

IBM sub-litho PCM Alverstone PCM

Over 30 companies work on


SCM
including all major IT players

Samsung 512 Mbit PCM chip


19

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Magnetic Racetrack Memory


MRAM alternatives
a 3-D shift
register

Data stored as pattern of magnetic domains in long


nanowire or racetrack of magnetic material.
Current pulses move domains along racetrack
Use deep trench to get many (10-100) bits per 4F2

IBM trench
DRAM
20

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Magnetic Race Track Memory


S. Parkin (IBM), US patents 6,834,005
(2004) & 6,898,132 (2005)
Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Magnetic Racetrack Memory


Need deep trench with notches to pin
domains
Need sensitive sensors to read presence of
domains
Must insure a moderate current pulse moves
every domain one and only one notch
Basic physics of current-induced domain
motion being investigated
Promise (10-100 bits/F2) is enormous
but were still working on our basic understanding
of the physical phenomena
21

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Memory/Storage Stack Latency Problem


Get data from TAPE (40s)

Century

1010
109
108
107

Month
Day
Hour

Second
22

Access DISK (5ms)

106
Time in ns

Human Scale

Year

Storage

105
10

Access FLASH (20 us)

SCM

103
102

Access PCM (100 1000 ns)


Get data from DRAM or PCM (60ns)

10

Get data from L2 cache (10ns)

Memory

CPU operations (1ns)

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Architecture
CPU

I/O
Controller

Synchronous
Internal
DRAM
Memory
Controller
SCM
SCM

Storage
Controller
External
23

Hardware managed
Low overhead
Processor waits
Fast SCM, Not Flash
Cached or pooled memory

SCM

Asynchronous
Software managed
High overhead
Processor doesnt wait
Switch processes
Flash and slow SCM
Paging or storage

Disk

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

SCM in a large System

1980

Logic

Memory

Active Storage

CPU

RAM

DISK

CPU

2015

CPU

24

TAPE

FLASH

RAM

RAM

2008

Archival

SSD

...

SCM

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

TAPE

DISK

DISK

Hot Chips 8/2010

TAPE

2010 IBM Corporation

IBM Almaden Research center

CPU & Memory System Conceptual Alternatives


DRAM
DIMM

X86, PowerPC,
SPARC

PMC
CPU

VM
SCMC
SCM
Card

SCM

SCM
Controller
25

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Input from the device cost crystal ball


$ 100,000.00

$ 10,000.00

$ 1,000.00

$ 100.00

$ 10.00

DRAM
DRAM-pr
FLASH
FLASH-pr
MRAM
SRAM
SCM

$ 1.00

$ 0.10

$ 0.01
1990

26

1995

2000

2005

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

2010

2015
Hot Chips 8/2010

2020
2010 IBM Corporation

IBM Almaden Research center

SCM: Generic Storage Design


CPU

MEMORY

SCM

SCM

SCM

DRAM
2

SCM
Control
SCM

SCM

SCM

SCM

27

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

SCM

Hot Chips 8/2010

SCM

2010 IBM Corporation

IBM Almaden Research center

28

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Challenges with SCM


Asymmetric performance
Flash: writes much slower than reads

The fly in the ointment is write


endurance

Not as pronounced in other


technologies
Program/erase cycle
Issue for flash
Most are write-in-place
Data retention and Non-volatility
Its all relative
Use case dependent
Bad blocks

In many SCM technologies writes are


cumulatively destructive
For Flash it is the program/erase
cycle
Current commercial flash varieties
Single level cell (SLC) 105
Multi-level cell (MLC) 104

Coping strategy --> wear leveling


Typically hidden from applications by
infrastructure

Devices are shipped with bad blocks


Blocks wear out, etc.

29

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Write and/or read endurance and life-time of SCM devices

In DRAM and disks (magnetic) there is no known wear out mechanism


In flash and many SCM technologies there are known wear out mechanisms
Tlife = Endurance Fill-Time
Fill-Time: time to write a memory unit (whats a data unit?)

Simple wear leveling each write is done to a new (empty) location

DRAM

Disk

256GB Flash

8 GB SCM

>1016

>1011

105 104

108

Wear leveled

Memory unit

1B

512 B

128 KB

256 GB

8 GB

Data unit

1B

512 B

128 KB

128 KB

128 B

Fill Time

100 ns

4 ms

2 ms

4000 s

500 s

Life Time

>31 yrs

>12 yrs

<4 min

>12 yrs

>190 yrs

Endurance

30

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

SCM impact on software


Operating systems
Extend state information kept about memory pages
New mechanisms to manage new resource
Enhanced to provide hints to other layers of software
Potential for direct involvement in managing caches and pools
Middle ware and applications evolutionary
Improved performance impact immediate full exploitation will
occur gradually
Little near term demand for non-volatility
Cost improvements will drive memory size
Memory size will drive larger and more complex data structures.
Reload time on a crash will be exacerbated
Users need for non-volatility, persistence, etc. will be driven by
these effects blurring of memory and storage
31

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

IBM QuickSilver Project SSD proof of concept


Ultra-fast storage
performance without
managing 1000s of disks.

SAN connected hosts

Demonstrated performance of
over 1 million IOPS using 40
SSDs.
Reduced $/IOPS, significantly
lower than traditional disk
storage farm.
Reduced floor space per IOPS
Improved energy efficiency for
high performance workloads.
Reduced number of storage
elements to manage

SAN

SVC
4 x 4Gbps
FC ports per node

Quick
Silver

Quick
Silver

SAN: Storage Area Network

32

SVC: San Volume Controller

32

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Shift in Systems and Applications

Main Memory:

Storage:

Applications:

33

DRAM Disk Tape


Cost & power constrained
Paging not used
Only one type of memory:
volatile

DRAM SCM Disk Tape


Much larger memory space for
same power and cost
Paging viable
Memory pools: different speeds,
some persistent
Fast boot and hibernate

Active data on disk


Inactive data on tape
SANs in heavy use

Active data on SCM


Inactive data on disk/tape
DAS ??

Compute centric
Focus on hiding disk
latency

Data centric comes to fore


Focus on efficient memory use and
exploiting persistence
Fast, persistent metadata

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Performance increase of
230% by automatic movement
of 3% of the application's
data to SSD
60000
50000

Throughput (IO/s)

40000

230%
IOPS
Improveme
nt

30000
20000
10000

0
0 :0 0 2 :0 0 4 :0 0 6 :0 0 8 :0 0 1 0 :0 01 2 :0 01 4 :0 01 6 :0 0 1 8 :0
T im

34

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

SCM Design Triangle


Speed
Memory-type
Uses
Storage-type
Uses

Cost/bit

(Write) Endurance

Power!
35

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Summary
Storage Class Memory is a new class of data
storage/memory technology many technologies
are competing to be the best SCM
SCM blurs the distinction between memory and
storage
SCM will impact the design of computer systems
and applications
Flash, which has may SCM characteristics, is
available now and various SCMs are in the wings.
EasyTier like software will foster exploitation of
Flash and SCM
36

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

References
FAST2010 Tutorial
T2: Freitas and Chiu, Solid State Storage: Technology,
Design and Applications
http://www.usenix.org/events/fast10/tutorials

IBM Journal of Research and Development


Special issue on storage
http://www.research.ibm.com/journal/rd52-45.html
Four papers related to SCM

Questions?
37

Storage Class Memory, Technology, Systems and Applications


Rich Freitas, IBM Research

Hot Chips 8/2010

2010 IBM Corporation