(/) (http://www.vmware.

com)
Mware.com

Communities (http://communities.vmware.com)

Search Blogs

Home (http://communities.vmware.com) > Blogs (http://blogs.vmware.com) > VMware vSphere Blog

VMware vSphere Blog

(http://blogs.vmware.com/vsphere)
Begin the journey to a private cloud with datacenter virtualization
Tweet

73

Like

14

Share

Share

16

Virtual SAN Hardware Guidance Part 1 - Solid State Drives

Posted on December 3, 2013 (http://blogs.vmware.com/vsphere/2013/12/virtual-san-hardware-guidance-part-1-solid-state-drives.html) by
Wade Holmes (http://blogs.vmware.com/vsphere/author/wade_holmes)

With the advent of VMware Virtual SAN, there have been many questions around the type of
server hardware that should be used to support and congure a Virtual SAN environment. While
this may be as easy as choosing the appropriate Ready Node server or following the
appropriate Ready System recommendation for your Virtual SAN environment (more
information on the Virtual SAN Ready program can be found here
(http://www.vmware.com/resources/compatibility/help.php#Virtual_SAN) ), there will be many people who choose to
build their own conguration from individual components within the
VMware Compatibility Guide (VCG).
Virtual SAN allows the exibility to build your own conguration based on a given vendors
hardware platform, allowing one to greatly dierentiate the performance of Virtual SAN clusters
based on your choices. But with exibility comes a number of decision points. This series of blog
posts will provide initial guidance for the hardware design considerations one can make when
choosing specic hardware components for a Virtual SAN environment.
Virtual SAN and Solid State Drives
Virtual SAN utilizes Solid State Drives (SSDs) as both a write buer and read-cache to accelerate
the performance of the platform. Solid State Drives (SSD) are devices that store persistent data
on solid-state NAND ash modules. Virtual SAN supported SSDs can utilize one of three
common interfaces, either SATA, SAS, or PCIe. So what are the tradeos between these
interfaces? Well from a pure interface speed perspective, each interface provides dierent
maximum throughput levels, as noted below.
SAS drives commonly support either 6Gb/s or 12Gb/s (based on the SAS-2 or SAS-3
specication)
SATA drives commonly support either 1.5Gb/s, 3Gb/s, or 6Gb/s (based on the SATA 1.0, 2.0 or
3.0 specication)

PCIe drives commonly support from 1 to 32 lanes, with throughput dependent on the PCIe
generation
Note: SATA 3Gb/s or 6 Gb/s drives may be connected to a SAS interface, but SAS drives cannot
be connected to a SATA interface.
The above interface performance numbers are straightforward for SAS and SATA (with interface
speeds clearly being stated as 3Gb/s, 6Gb/s, or 12Gb/s), but you may be wondering what PCIe
generations and lanes mean to interface performance? Well the rst variable is the PCIe
generation. Each subsequent generation has increased the speed possible per lane, as detailed
below.
v1.x: 250 MB/s (2.5 GT/s)
v2.x: 500 MB/s (5 GT/s)
v3.0: 985 MB/s (8 GT/s)
v4.0: 1969 MB/s (16 GT/s)
A PCIe device can contain from 1 to 32 lanes, meaning a PCIe Gen 2 card with x8 lanes can
produce a maximum of 4GB/s (32 Gb/s) of throughput. As general guideline, PCIe SSD devices
will typically outperform SAS/SATA SSDs. But interface performance is only one factor in
choosing an SSD. Next lets look at drive I/O performance.
SSD IOPS
When selecting hardware for a Virtual SAN cluster, you may notice that SSD's on the VCG site
are categorized into dierent classes, based on write performance. All SSDs are not created
equal, so to simplify choosing an SSD VMware has categorized SSD devices into ve
performance classes. The class of the SSD you chose can greatly aect the performance of your
VSAN cluster. Below are the designated SSD classes specied within the
VMware Compatibility Guide.
Class A: 2,500-5,000 writes per second
Class B: 5,000-10,000 writes per second
Class C: 10,000-20,000 writes per second
Class D: 20,000-30,000 writes per second
Class E: 30,000+ writes per second
While write performance is a good general guideline to gauge SSD performance (and we chose
write performance because in SSDs, writes are much more of a limiting factor of SSD
performance than either random or sequential reads), to gain further insight into the
performance characteristics of an SSD and how it may eect workloads in Virtual SAN, one
should also consider factors such as the queue depth at which an SSD vendor is reporting its
metrics, and maximum drive latency numbers.
SLC vs MLC vs eMLC
As you look at dierent SSDs, you may notice SSD vendors will categorize their components as
either single-level cell (SLC), multi-level cell (MLC) or enterprise multi-level cell (eMLC) NAND.
NAND ash modules are the non-volatile storage components that comprise SSDs.
In single-level cell (SLC), each cell can store a single bit (0 or 1) of information. In multi-level cells
(MLC) NAND ash uses multiple levels per cell to allow more bits to be stored. An MLC will
usually have 4 bits (00, 01, 10, 11). MLC is typically lower in cost than SLC, but the NAND modules

typically have a shorter life span. Because MLC uses the same number of transistors as SLC,
there is potentially a higher risk of errors within each module. eMLC is a middle ground between
the cost/lifespan of SLC & MLC modules, and will typically utilize 2 bits. Because eMLC ash
media has more program-erase (P/E) cycles than consumer MLC, it has greater endurance and
can tolerate the types of workloads that enterprise applications require.
So does this mean that any SSD utilizing SLC is more reliable than eMLC, which is more reliable
than MLC? Not necessarily. The reliability and performance of the individual NAND modules can
be bolstered by features within an SSD's controller. Dierent vendors utilize dierent SSD
design techniques to increase the reliability of their drives to achieve enterprise class (such as
NAND over provisioning and SSD controller endurance features). Because of this, VMware does
not dierentiate in the support of any of these NAND technologies. What matters is that the SSD
device meets the minimum performance and reliability metrics dened by VMware within its
given performance class, regardless of vendor SSD design choices (i.e. NAND type .vs SSD
controller features) utilized to reach those metrics.

(http://blogs.vmware.com/vsphere/les/2013/11/VSAN-HardwareGuidanceP1.png)

Note: This graphic displays dierent SSD design approaches combined with dierent NAND
types to achieve similar SSD endurance levels. Figures here are for illustrative purpose only.
VMware SSD Endurance Requirements
SSD write metrics are the primary measurement used by SSD vendors to gauge SSD reliability.
While there is no standard metric across all vendors, most vendors measure SSD reliability in
either Drive Writes Per Day (DWPD) or Petabyes Written (PBW). VMware requires that any SSD
device within the VCG meet the following minimum endurance metrics during a ve year
lifespan.
VMware endurance requirements for SAS and SATA SSDs
The drive must support at least 10 full Drive Writes per Day (DWPD), and
The drive must support random write endurance up to 3.5 PB on 8KB transfer size per NAND
module, or
The drive must support random write endurance up to 2.5 PB on 4KB transfer size per NAND
module.
VMware Endurance requirements for PCIe SSDs
The drive must support at least 10 full Drive Writes per Day (DWPD), or
The drive must support random write endurance up to 3.5 PB on 8KB transfer size per NAND
module, or

The drive must support random write endurance up to 2.5 PB on 4KB transfer size per NAND
module.
While Virtual SAN focuses on enabling performance and operational eciency for the majority
of workloads without the need for complex design decisions, there will always be those who
want to customize their Virtual SAN environment as much as possible through the selection of
specic hardware components. Hopefully this post has given you some insight into SSD
hardware considerations around Virtual SAN. In the next part of this series, we will delve into
storage controller considerations.

This entry was posted in Uncategorized (http://blogs.vmware.com/vsphere/uncategorized) and tagged Virtual SAN
(http://blogs.vmware.com/vsphere/tag/virtual-san) ,

VSAN (http://blogs.vmware.com/vsphere/tag/vsan) on December 3, 2013

[http://blogs.vmware.com/vsphere/2013/12/virtual-san-hardware-guidance-part-1-solid-state-drives.html] by

Wade Holmes

(http://blogs.vmware.com/vsphere/author/wade_holmes) .

About Wade Holmes

Wade Holmes, VCDX #15, CISSP, CCSK, is a Global Cloud Architect at VMware, currently
focused on working with vCloud Air Network service providers. Wade has over 18 years of
industry experience in the design and implementation of complex computing environments of
all scopes and sizes. Wade has presented at many industry conferences, and is a co-author of
the VMware vCloud Architecture Toolkit book. Wade holds a Bachelors degree in Information
Technology and a Masters Degree in Information Assurance. Wade also blogs on
www.vwade.com, and you can follow Wade on Twitter @wholmes.
View all posts by Wade Holmes (http://blogs.vmware.com/vsphere/author/wade_holmes)

5 thoughts on Virtual SAN Hardware Guidance Part 1 - Solid State Drives

Pingback: What happens in a VSAN cluster in the case of a SSD failure? (http://www.yellowbricks.com/2013/12/19/what-happens-in-a-vsan-cluster-in-the-case-of-an-ssd-failure/)

Pingback: Virtual SAN Hardware Guidance Part II Storage Controllers | VMware vSphere Blog VMware Blogs (https://blogs.vmware.com/vsphere/2014/01/virtual-san-hardware-guidance-part-ii-storage-controllers.html)
Pingback: Technology Short Take #38 - blog.scottlowe.org - The weblog of an IT pro specializing
in virtualization, networking, storage, and servers (http://blog.scottlowe.org/2014/02/06/technology-short-take-38/)
Pingback: VSAN Links Welcome to vSphere-land! (http://vsphere-land.com/vsphere-links/vsan-links.html)
Pingback: VSAN Cluster SSD Failure (http://www.tayfundeger.com/vsan-cluster-ssd-failure.html)