Вы находитесь на странице: 1из 379

NTS-2001-I: Nimble

Introduction and Administration

Course Revision 5
Introductions

 Name
 Employer
 Job function
 Data storage experience
 Hands on experience with Nimble products?
 What do you hope to get out of this course?

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-4


Course Topics

 Product Introduction  Backup and Disaster Recovery


» Module 1: AF-Series Introduction » Module 8: Introduction to Nimble Storage
» Module 2: CS-Series Introduction Snapshots
» Module 3: NimbleOS WebUI Introduction » Module 9: Introduction to Nimble Storage
» Module 4: Introduction to Customer Replication
Support and InfoSight
 Exam completion – required for course
 Product Utilization completion
» Module 5: Pre-Installation, Racking, and
Cabling
» Module 6: Initial Configuration of the
Unified Flash Fabric Overview
» Module 7: Working with Nimble Storage
Volumes

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-5


Documentation
Documentation location

1. Log into InfoSight  Categories


2. Click Resources and then » Release Notes
Documentation » Product User Guides
» Solution and Integration Guides
» Support and Knowledge Base Articles

 Refine Searches By:


» Document type
» Nimble Software and Solutions
» NimbleOS Version
» Integration
» Platform

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-7


Units of Measure Defined
Raw versus Usable versus Effective Capacity

AF-Series arrays using


deduplication will see
Subtract additional savings.
capacity
for Add storage
Raw RAID Triple capacity due
to inline Effective
Capacity Parity,
Usable compression Capacity
& system
Capacity (typical 30%
reserves
to 75%)

Effective: 49.5TB
Raw: 48TB Usable: 33TB
(assuming 50% compression)

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-9


TiB Vs. TB

collateral utilizes
TB and TiB
Nimble OS uses TiB

Tebibyte (TiB) and terabyte (TB) are related

 TB is base 10 (i.e., 1 TB = 1,000,000,000,000 bytes)

 Ti, or tebi, represents multiplication by 10244. So the math


using Ti is more accurate:
» 1 tebibyte = 240 bytes or 1,099,511,627,776 bytes or more commonly
stated as 1024 gigabytes

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-10


Module 01:

The All Flash Array (AF-Series)


Module 01 Objectives

At the end of this module, you will be able to:


 Describe the AF-Series hardware
 Understand the AF-Series Write Operations
» Deduplication
» Compression
» Sequential stripe writes
» Triple+ Parity with Integrated Spare
» Integrated spare rebuild
» Quick RAID rebuild
» SmartSecure Encryption
 Understand AF-Series Read Operations
» Changed block operations
 Scale-to-Fit with the AF-Series

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-12


Product Line Introduction
AF-Series: a product family of the Unified Flash Fabric
All Flash Array Family: AF-Series

AF9000

300,000 IOPS
AF7000

230,000 IOPS
AF5000

120,000 IOPS
AF3000

50,000 IOPS IOPS based on a 70%


AF1000
Read and 30% write
workload
35,000 IOPS

www.nimblestorage.com/technology-products/all-flash-array-specifications/

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-14


AF-Series Hardware Tour
What’s in an AF-Series Array?
Back: Dual Power
Supplies (AC and DC
available)

Front: SSD Drives Back: Dual Controllers


(CPU, Network)

Controller Head Shelf


Expansion Shelves (optional)

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-16


AF-Series Chassis Front View

 4U Chassis
 48 SSD Drive capacity
 2 Banks of 24 x 3.5” SSD drives
 Bank A – SSD Drives 1 -24
 Bank B – SSD Drives 25 - 48 Slot 21 Slot 22 Slot 23 Slot 24

Slot 17
Slot 13
Slot 9
Slot 5

Slot 1 Slot 2 Slot 3 Slot 4

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-17


AF-Series Chassis Front View / Dual-Flash Carrier (DFC)

Slot 21 Slot 22 Slot 23 Slot 24

Power On LED
Slot 17 Heartbeat
NIC1/2 LED
Slot 13
Slot 9 Power Fault Over
Slot 5 Temperature

DFC latch
Slot 1 Slot 2 Slot 3 Slot 4

Bank B Ejector latch

Bank B

Bank A

Bank A Ejector latch


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-18
AF-Series Dual-Flash Carrier (DFC)

 Hot swappable base carrier hosting two banks.


 Hot swappable SSD carrier provides tool-less
drive installation.
 LEDs on each SSD Carrier and on the Base
Carrier

Presence & Drive Fault LED


Activity Slot Fault

Base Carrier
Release
SSDCarrier
Release

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-19


AF-Series Hardware Tour - Back
1

5
3
4

6 7
1. Power Supply 3. Fans 5. Data Networks 7. SAS Ports
2. Power Intel 4. Management Network 6. KVM/Serial Port 8. Controller
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-20
Controller Detail View (AF-Series and Newer CS-Series)

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-21


Mid-Chassis Fans

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-22


AF-Series Spares / FRUs (Controller Head Shelf)

Or
Spares Kit
Controller
(NICs/HBAs not included) Power Supply

Additional Spares
AF-Series SSD
NIC / HBA

Cables

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-23


AF-Series Write Operations
CASL Architecture

8K 4K 4K
ack

Mirrored
1) Writes are sent by a variety of
applications in variable block sizes
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)
2) CASL places incoming writes into the
8K 4K 4K 8K 4K 4K
active controllers NVRAM
3) CASL mirrors the active controllers
NVRAM to the standby controllers DRAM
NVRAM
4) CASL acknowledges the write

CS-Series AF-Series

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-25


CASL Architecture

8K 4K 4K
ack

Mirrored
5) Blocks are copied into DRAM
a) What happens next depends of
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)
type of array
8K 4K 4K 8K 4K 4K
b) All-Flash Array
1. Variable block deduplication
is applied DRAM
2. Variable block compression 8K 4K 4K
Variable Block Variable Block
is applied Deduplication Compression
Sequential Stripe 10MB
3. Blocks are formed into a
10MB sequential stripe
write Index
4. Blocks written to SSD and
are indexed in DRAM
CS-Series AF-Series

Index

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-26


Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM

Variable Block
Deduplication

AF-Series

Index

AF-Series: Deduplication
Deduplication Primer

Inline Deduplication

Vs.
Source / Server Post Process
Deduplication

Target / Storage Array

Inline Deduplication:
• Dedupe applied as the writes are processed

Post Process Deduplication:


• Writes the data as-is and uses a background operation to remove duplicates

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-28


Process Review

Incoming writes Batched in NVRAM/NVDIMM

Batch Flush

Variable Block Deduplication


• Secured by strong SHA2-256 encryption

Variable Block Compression

Full Stripe Writes

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-29


Deduplication Block Sizes

Variable Block Dedupe Compression

Fixed Block Dedupe


Target / Storage Array

Variable Block Level Dedupe:


• Unit of duplicate detection is volume block which depends on the performance policy.
• Requires less metadata by adapting the detection size to the application block size, thereby
improving efficiency and speed.

Application Aware:
DB VDI DB Exchange
VDI Exchange VDI VDI

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-30


Fingerprint Matching or Byte Comparison?

With most forms of deduplication,


performance is adversely affected Fingerprint Index
in some way.

Deduplication
Performance • Better than block-by-block
• Secured by strong SHA2-256 encryption

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-31


Dedupe Isn’t Free

 Fingerprinting content costs CPU


 Identifying duplicates costs:
» memory
» I/O (flash reads)
» CPU
 Tracking content costs storage
» Flash content/FDR

 Not every workload benefits!


 These costs are true of dedupe in general, not our implementation.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-33


Deduplication Global, Domain Level, or Off?

On – On –
Off
Globally Domain/Application

• Per Application Category Eliminate the


• Per Storage Pool deduplication
• Per Volume “tax”

Make informed decisions about whether to use


deduplication or not and what workloads or volumes
would be best served by deduplication.
InfoSight
Predictive Analytics
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-34
Deduplication Management – AFA only

Global Deduplication

Deduplication at Domain/Application Level

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-35


Application Category Management – AFA only

 Application Categories are introduced with


Deduplication and are found as part of the
performance policy
 Application Category:
» A data structure in the WebUI that limits the scope
of duplication on volumes attached to an
Application Policy
» There are 8 fixed application Categories

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-36


Nimble Differentiators – More about App Category

App Category VDI Database

VDI-Policy-1 VDI-Policy-2 VDI-Policy-3 VDI-Policy-4 DB-Policy-1


Perf Policy BS=4K BS=8K BS=4K BS=8K BS=32K
Dedupe=ON Dedupe=ON Dedupe=ON Dedupe=OFF Dedupe=OFF

Volume V1 V2 V3 V4 V5 V6 V7 V8 V10 V11

V9
Like colored volumes
denote block sharing Dedupe = OFF

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-37


Viewing Data Reduction Savings
Monitor >> Space
Data Reduction Savings
5.2TB | 5.93X
Clones 0 B 1.0X
Deduplication 3.79 TB 2.55X
Compression 1.4 TB 2.33X

Deduplication Savings
Savings Ratio Participating
Volumes
3.79 TB 2.55X 18 of 18 vols
0B 1.0X 1 of 1 vols
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-38
Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM

Variable Block Variable Block


Deduplication Compression

AF-Series

Index

AF/CS-Series: Compression
What you need to know about Lempel-Ziv 4

 Compression performed in-line


» Applied after deduplication, if deduplication is enabled
 LZ4 is a fast lossless compression algorithm
» Provides compression speeds of 300 MB/s per CPU core
» Provides a fast decoder that provides speeds up to and beyond 1GB/s per
CPU core
• It can reach RAM speed limits on multi-core systems.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 40 1-40


Viewing Data Reduction Savings
Monitor >> Space
Data Reduction Savings
5.2TB | 5.93X
Clones 0 B 1.0X
Deduplication 3.79 TB 2.55X
Compression 1.4 TB 2.33X

Compression Savings
Savings Ratio
1.4 TB 2.33X
0B 1.0X

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-41


Application Compression with Nimble

*March 2016

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-42


Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Index

Sequential Stripe Write


Common File Systems– Write in place file system (EMC, EQL)

Pros
•Simple to implement, long
history

•Good sequential read


performance without cache

Cons
•Poor random write performance

•Slow, high overhead compression

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-44


Common File Systems – Hole filling (WAFL, ZFS)

WAFL – Write Anywhere File Layout


ZFS – Copy on write transactional model
Pros
•Good random write performance until
disk fills up

•More efficient redirect-on-write


snapshots

Cons
•Performance degrades over time

•Slow, high overhead compression

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-45


Nimble Storage File System – Always write full stripes

 Good AND consistent write


performance
 Very efficient snapshots
 Fast inline compression
 Efficient flash utilization,
long flash life
 Ground up design
 Enables variable block size
 Uses a sweeping process to ensure full
stripe write space

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-46


Background Sweeping

• Data blocks are indexed as they are written


• Over time the deletion of snapshots and data leaves stale data blocks
• Sweeping removes stale blocks and forms new stripe writes with the
remaining active blocks

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-47


Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Triple+ Parity with


Integrated Spare Index

AF-Series: Triple+ Parity With Integrated Spare


Triple+ Parity With Integrated Spare

Triple+ Parity RAID Stripe


D1 D2 D3 P Q R
 Triple Parity: 3 drives
worth of parity information
is written

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-49


Intra-Drive Parity: Data Layout

Triple+ Parity RAID Stripe


 Every chunk in a stripe consists of Data, D1 D2 D3 P Q R
Metadata (checksums, sequence number
etc…), and Parity (XOR of Data)
 This parity can be used to recover 1 sector
corruption or IO error, when there is no
redundancy in the stripe (3 drive failure)
 This recovery can be done independent of the
condition of other drives: can recover from 1- Data
sector corruption on every drive read in the Chunk
stripe. (1 per drive)
 Note: Intra-Drive Parity is also available in CS- Metadata
Series starting with Nimble OS 3.X. Chances Parity
of data recovery in HDD is lower than SSD
due to errors/corruptions in HDD are often
larger.
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-50
Integrated Spare: Data, Parity, and Spare layout

AFA Triple+ Parity


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
D D D D D D D D D D D D D D D D D D D D P Q R S
D D D D D D D D D D D D D D D D D D D P Q R S D
D D D D D D D D D D D D D D D D D D P Q R S D D Tolerates Simultaneous Failure of Any 3 SSDs
D D D D D D D D D D D D D D D D D P Q R S D D D
D D D D D D D D D D D D D D D D P Q R S D D D D
D D D D D D D D D D D D D D D P Q R S D D D D D
D D D D D D D D D D D D D D P Q R S D D D D D D
D D D D D D D D D D D D D P Q R S D D D D D D D
D D D D D D D D D D D D P Q R S D D D D D D D D
D D D D D D D D D D D P Q R S D D D D D D D D D
D D D D D D D D D D P Q R S D D D D D D D D D D Built in virtual spare allows 4th failure
D D D D D D D D D P Q R S D D D D D D D D D D D
D D D D D D D D P Q R S D D D D D D D D D D D D
D D D D D D D P Q R S D D D D D D D D D D D D D
D D D D D D P Q R S D D D D D D D D D D D D D D
D D D D D P Q R S D D D D D D D D D D D D D D D
D D D D P Q R S D D D D D D D D D D D D D D D D
D D D P Q R S D D D D D D D D D D D D D D D D D
D D P Q R S D D D D D D D D D D D D D D D D D D Intra-drive parity fixes sector loss in single read
D P Q R S D D D D D D D D D D D D D D D D D D D
P Q R S D D D D D D D D D D D D D D D D D D D D
Q R S D D D D D D D D D D D D D D D D D D D D P
R S D D D D D D D D D D D D D D D D D D D D P Q
S D D D D D D D D D D D D D D D D D D D D P Q R

Data Quick RAID rebuild

Metadata
Parity
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-51
Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Index

Integrated Spare: Rebuild Operation


Integrated Spare RAID : Single Drive Failure Scenario
Failed
Disk
Disk 0 Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6 Disk 7 Disk 8 Disk 9 Disk 10
D D D D D D D P Q R D
D D D D D D P Q R D D
D D D D D P Q R D D D
Spare Chunk
D D D D P Q R P D D D
D D D P Q R Q D D D D
D D P Q R R D D D D D

D P Q R S D D D D D D
P Q R D D D D D D D D
Q R D D D D D D D D P
R D D D D D D D D P Q
D D D D D D D D P Q R

 Generate data/parity belonging to failed drive


 Write data to spare chunk

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-53


Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Index

Quick RAID Rebuild


Quick Raid Rebuild: Motivation & Goal

Long rebuilds create frustration when:


 User pulls a drive out by mistake
 Drive is not completely failed, but needs to be reseated for various reasons
 S/W stack needs to power cycle the drive for recovery purposes
 S/W stack needs to reset interposer for recovery purposes that may lead HBA link timeout

Goal:
 Enable the NOS to take action to help reset or recover an unhealthy drive.
 See whether there is any way the RAID system can bypass the painful long rebuild or reduce its
impact to minimum

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-56


Quick Raid Rebuild: Impact to end-user

 When a drive is mistakenly taken out, it can be reinserted immediately without


lengthy rebuild time
 No timeout period for quick rebuild eligible (!)
 Rebuild is essentially incremental in some scenarios
 The feature is ready on both AFA and CS in the NimbleOS 3.X release

Notes:
 Not applicable when replacing a failed drive
 “Incremental” rebuild, subject to certain conditions

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 59 1-59


Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Index

SmartSecure Encryption
SmartSecure Software Based Encryption

What is SmartSecure?
Encryption that:
 Ensures the secrecy of data “at rest”
» Uses AES-256-XTS cipher for cryptographic protection of data
» FIPS 140-2 level 1 certified *****

 Protects against threat vectors Hello Hello


» Theft of entire storage array
» Theft of disks (HDDs or SSDs)
 Is transparent to host access to encrypted volumes
» Data is encrypted at write time when entering the array
» Data is decrypted at read time when exiting the array

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-61


SmartSecure Software Based Encryption

Why Deploy SmartSecure?


 Theft or disposal of drives or an entire array does not expose data
» Data Center physical security breach
» Shipping or transporting an array to an affiliate site or new location
» RMA of a failed component
 Ability to irrecoverably shred (destroy) data at a volume level
» Volume keys marked for deletion when an encrypted volume is deleted
 Requirements for data to be encrypted on disk
» Government, compliance, customer, etc.
 Ensures data secrecy on replication streams over a WAN

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-62


SmartSecure Software Based Encryption

Key Data Points


 No additional cost
» Familiar Nimble Storage licensing/pricing model
 Supported on all Nimble Storage arrays running NimbleOS 2.3 and higher
» Leverages the Intel AES-NI instruction set
• Best performance on later model CS series arrays
– CS235, CS300, CS500, CS700

 Supports encryption of data on both HDDs and SSDs


 Preserves compression feature
» Data is compressed prior to encryption
 Easy to administer & manage

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-63


Administration >> Security >> Encryption

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-64


Optional
SmartSecure Software Based Encryption Content

Technical Data Points


 The Passphrase…
» In “Secure Mode” the passphrase must be entered after an array restart or power on…
• In “Secure Mode”, encrypted volumes remain offline and cannot be accessed until the passphrase is
entered
• If the passphrase is not available, the data in encrypted volume is lost forever
» In “Available Mode” the passphrase may be required…
• If controllers are being swapped during an upgrade process
• In the rare scenario involving NVRAM loss

 Do not lose the passphrase!


» The array user must keep track of and maintain the passphrase
» The passphrase is never stored on disk or SSD within the Nimble Storage array
» The passphrase is not transmitted to Nimble Storage Technical Support
» The passphrase is not copied into Email Alerts, SNMP, or Syslog

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-71


SmartSecure Software Based Encryption

Technical Data Points


 Volume Keys
» The encrypted state of a volume is defined at volume creation time
• Encrypted volumes cannot be “converted” to unencrypted volumes
• Unencrypted volumes cannot be “converted” to encrypted volumes
» New encrypted volumes each get a new encryption key
• 256 bits in length
• Encrypted by the master key using AES-256-KeyWrap
• Stored in the key table
» Cloned encrypted volumes are also encrypted
• Clones get their own new volume key
• But also have access to their ancestor’s key to read shared blocks

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-72


AF-Series Read Operations
CASL Architecture

Mirrored
1) Read from NVDIMM
2) If not found, check DRAM
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)
3) If not found, read from SSD using the
index for a quick lookup 1 8K 4K 4K 8K 4K 4K

• If found, validate checksum,


decompress, dedupe, and return 2 DRAM
data
Variable Block Variable Block
Deduplication Compression

Index
3
AF-Series

Index

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-77


Changed block operations review
Compression Performance Comparison During Changed Block
Operation

Fixed block Architecture CASL Variable Blocks


8 blocks grouped Individual blocks
& compressed 1 2 3 4 5 6 7 8
compressed and coalesced
Group placed into into stripe
N fixed size slots 1 12 2 33 4 5 56 67 87 8
Updated data block
Entire group read & compressed and coalesced
uncompressed into new stripe
Block updated with
new data
New group compressed &
re-written

Other Array Manufacturers Nimble Storage


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-79
Compression Performance Comparison For A Changed Block

Fixed block Architecture CASL Variable Blocks


8 blocks grouped Individual blocks
& compressed compressed and coalesced
Group placed into Cost of fixed block architecture relative to CASL: into stripe
N fixed size slots 1. Additional blocks read from disk
2. Additional CPU cycles for decompression & Updated data block
Entire group read & recompression of all N blocks compressed and coalesced
uncompressed 3. Additional M-1 blocks written to disk into new stripe
Block updated with
new data
New group compressed &
re-written

Other Array Manufacturers Nimble Storage


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-80
Scale-to-Fit with the AF-Series
Nimble Storage AF-Series Scale-to-Fit: Scale Out

4 Node AF9000
Cluster
AF9000
300K IOPS Up to 1.4M IOPS
8180TB Capacity

AF7000
230K IOPS
IOPS based on
a 70% Read
and 30% write
workload AF5000
120K IOPS
SCALE UP

AF3000
50K IOPS

SCALE DEEP

AF3000 and 5000 + 1 AFS2 AF7000 + 2 AFS2 AF9000 + 2 AFS2

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-82


AF-Series Scale Out

SCALE DEEP

AF3000 and 5000 + 1 AFS2 AF7000 + 2 AFS2 AF9000 + 2 AFS2

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-83


Scale Deep with the AFS2 Expansion Shelf

The AF-Series AFS2 Expansion Shelf

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-84


AFS2 Front View

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-85


AFS2 Bezel Panel

Power Button

Power On LED Heartbeat LED


On=PowerOK Always On
Off=No Power
AFS2 shelves have the same
front panel LEDs, except the SAS
port LEDs replace the NIC LEDs
SAS-out SAS-in
On=Link On=Link
Off=No Link Off=No Link

Power Fault Over Temperature


Off=No Fault Off=OK
or No PSU On=System OverTemp
On=PSU Fault, Blink=At least 2 fans failed
No AC, or Internal Fault

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-86


AFS2 Rear View

Path A Path B

SAS IN port
Expander FW Status
Green-OK
Red-FW Failed to Load SAS Link Status
Green-12Gbps Red-Fault
Orange-6Gbps Off-No link

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-87


AFS2 Connections – AF3000/AF5000

 One AFS2 is supported.


 Connect P1 SAS-out from the
head to SAS-in of AFS2
– P2 would work just as well but P1
is preferred for consistency
SAS expansion port (p1) SAS expansion port (p1)

SAS IN port SAS IN port

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-89


AFS2 Connections – AF7000/AF9000

 Support up to 2 AFS2.
 First AFS2 - Use P1
 Second AFS2 – Use P2
SAS
– Do not daisy chain AFS2 expansion
port (p2)

AFS2 shelves ship with 1 meter cables. If


you do not order a 3 meter cable you will
need to install one AFS2 shelf above the
controller shelf and one below.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-90


AFS2 Web GUI

 When a AFS2 shelf is


added, it will need to be
activated using the
Activate button.

 Once activated the shelf


will be added to the
storage pool.

Once and expansion shelf


is activated, it cannot be
removed from the solution.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-91


Expanding Existing System

1 Fill Expansion Shelf until 2 Then balance capacity between them


capacity utilization
matches the control
head

50% Capacity 50% Capacity

Expansion Shelf Controller Shelf

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-92


Power On/Off Order

 On  Off
» Power expansion shelves first, then the – Power off the controller shelf and then
controller shelves the expansion shelves

Controller Shelf

Expansion Shelf

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-93


AF9000
300K IOPS

AF7000
IOPS based on
230K IOPS AF-Series Scale Up
a 70% Read
and 30% write
workload AF5000
120K IOPS
SCALE UP

AF3000
50K IOPS

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-94


Scale Up with Model Upgrades

Model Upgrades

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-95


AF-Series Controller Upgrades

 To ensure the most accurate information regarding controller upgrades for the AF-
Series array download the Nimble All Flash Array Configuration Support Matrix from
InfoSight.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-96


Upgrading Controllers

1 Determine which controller is currently in Standby mode.


• In WebUI – Manage Arrays >> Select individual array
• In CLI – use command ctrl --list

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-97


Upgrading Controllers

2 Label and disconnect all cables running to the Standby controller.

Management
ETH2

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-98


Upgrading Controllers

3 Remove the Standby controller.


• Loosen the controller thumb screw.
• Pull the controller release lever out and down.
• Pull the controller straight out.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-99


Upgrading Controllers

4 Remove the onboard USB stick from the old controller and install it into the new controller.

During controller upgrade, you must remove the USB stick from the existing controller and
install it into the new controller. Failure to perform this step will prevent the new controller
from coming online.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-100


Upgrading Controllers

5 Install new controller and reconnect all cables.

6 Verify the controller powers up and is in standby mode.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-101


Upgrading Controllers

7 Perform a failover to the new controller and ensure that the new controller is in active mode.
• In WebUI – Manage Arrays >> Select individual array >> click “Make Active”
• In CLI – use command failover

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-102


Upgrading Controllers

8 Confirm the non-upgraded controller is in standby mode and repeat steps 2 – 7 for the opposite
controller:
2. Label and disconnect all cables running to the Standby controller.
3. Remove the Standby controller
4. Remove the onboard USB stick from the old controller and install it into the new controller.
5. Install new controller and reconnect all cables.
6. Verify the controller powers up and is in standby mode.
7. Perform a failover to the new controller and ensure that the new controller is in active mode.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-103


Upgrading Controllers

9 Verify that the model number in the WebUI or CLI reflects the new model number.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-104


4 Node AF9000
Cluster
Up to 1.4M IOPS
8180TB Capacity

AF-Series Scale Out

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-105


Scale Out with Clustered Arrays

4 Node AF9000  Cluster up to 4 of any


Cluster Nimble Storage arrays
AF9000
125K IOPS Up to 1.4M IOPS
8180TB Capacity
 Simple configuration
 Automated host connection
AF7000 management
270K IOPS
 Dynamic load-balancing and
automated capacity rebalancing
AF5000
140K IOPS
 Linear performance
scaling
SCALE UP

AF3000,
60K IOPS
 Automatic data
SCALE DEEP
migrations
AF3000 and 5000 + 1 AFS2 AF7000 + 2 AFS2 AF9000 + 2 AFS2

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-106


Keep Applications Running – No Disruptions!

 Add or remove arrays


non-disruptively
» Automatically rebalances data

 Non-disruptively perform a technology


refresh
» Migrates data to upgraded hardware Repurpose older
array for Disaster
Recovery

Exchange + SQL Server VDI VDI VDI

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-107


Scale-Out Management - Groups

 Group: Manage arrays from a single


console
» 1-4 arrays per group
» Add and remove arrays on the fly
» Uses a single management IP
• Get status and view performance and capacity
reports
• Create and manage storage pools and volumes

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-108


Scale-Out Pools

 Pools
Nimble Scale-Out Group
» Simplify management
Array A1* Array A2 Array A3 Array A4
» Easy to grow or shrink
• Array is used in only 1 pool at a time L
» Can migrate HW (live) Pool Pool

Nimble Scale-Out : Finance-1


 How To Use Array A1* Array A2 Array A3 Array A4
» Use to scale out apps Pooling by L
Application
» Use to simplify allocation to SharePoint VDI
departments
Nimble Scale-Out: Remote Office-1
Array A1* Array A2 Array A3 Array A4
*As of OS 3.x, multi-array pools, group merges, etc are Pooling by L
supported on both FC and iSCSI Organization
The initial FC release does not support volume moves, Sales IT/Back Office Apps
striped pools and group merge.
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-109
Scale Out – Managing Hosts & Paths

Nimble Connection Mgr (NCM)


• Automatic MPIO configuration and path management
• Eliminates manual connection setup to individual arrays
Nimble Connection
Manager or NCS/PSP
plug-in for VMware

NIC 1 NIC 2

Switch 1 Switch 2

NCM Nimble Scale-Out Group

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 1-110


Module 02:

The Adaptive Flash Array (CS-Series)

Course Revision 5a
Module 02 Objectives

At the end of this module, you will be able to:


 Describe the CSx000 hardware
 Understand the CSx000 Write Operations
 Understand CSx000 Read Operations
 Scale-to-Fit with the CSx000

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-2


Product Line Introduction
CSx000: a product family of the Unified Flash Fabric
Adaptive Flash Product Portfolio – Mapping

CS7000
• 4U24
CS5000 • Up to 230K IOPS
• 21TB to 882TB raw
CS3000 • 4U24 • 6 Expansion shelves
• Up to 120K IOPS • 1 AFS
• 4U24 • 21TB to 882TB raw
CS1000/H • Up to 50K IOPS • 6 Expansion shelves
• 4U24 • 21TB to 882TB raw • 1 AFS
• Up to 35K IOPS • 6 Expansion shelves
• 11TB to 882TB raw • 1 AFS
• 6 Expansion shelves CS700
• 1 AFS
• 3U16
• 125K IOPS
CS500 • 12TB to 612TB raw
• 3U16 • 6 Expansion shelves
CS300 • 100K IOPS • 1 AFS (25.6TB max)
CS235 • 12TB to 612TB raw
• 3U16
• 6 Expansion shelves
• 30K IOPS
CS215 • 3U16 • 12TB to 612TB raw
• 1 AFS (25.6TB max)
CS210 • 15K IOPS • 6 Expansion shelves
• 3U16 • 24TB to 612TB raw
• 3U16 • 15K IOPS • 1 AFS (25.6TB max)
• 3x 6TB Expansion shelves
• 15K IOPS • 12TB to 282GB raw • No AFS
• 8TB to 98TB raw • 3x 6TB Expansion shelves
• 1x 6TB Expansion shelves • No AFS, no FC
• No AFS, no FC
Portfolio Consolidation
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-4
Adaptive Flash Array: CSx000

CS7000

Up to
230,000 IOPS
CS5000

Up to
120,000 IOPS
CS3000

Up to
50,000 IOPS
CS1000 IOPS based on a 70%
CS1000H Read and 30% write
workload
Up to
35,000 IOPS
www.nimblestorage.com/technology-products/all-flash-array-specifications/
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-5
CSx000 Hardware Tour
What’s in an CSx000 Array? Similar components as an AF-Series
Back: Dual Power
Supplies (AC and DC
available)

Front: SSD Drives Front: HDD Drives Back: Dual Controllers


(CPU, Network)

Controller Head Shelf


Expansion Shelves (optional)

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-7


Drive Layout – CS1000, CS3000, CS5000, CS7000

 4U 24 Chassis  HDDs: 18 + 3 RAID


 24 x 3.5” Slots carry 21x HDDs + 3x DFCs » New Nimble-branded HDD carriers
 DFCs : minimum of 3 SSDs in Bank A
» Bank B available for cache upgrades

Slots 21-24

Slots 17-20

Slots 13-16

Slots 9-12

Slots 5-8

Slots 1-4

21X Disk Carriers

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a DC latch 2-8
Drive Layout – CS1000, CS3000, CS5000, CS7000

Slots 21-24

Slots 17-20 Power On


Heartbeat
Slots 13-16 NIC1/2

Power Over
Slots 9-12
Fault Temperature
Slots 5-8

Slots 1-4
DFC latch
Cache: 3x Dual Flash Carriers
Bank B Bank A latch

Bank A Bank A latch

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-9


Drive Layout – CS1000H: A half populated CS1000

First 11 HDDs
referred to as
CS1000H

Cache:
2x Dual Flash Carriers Fully
populated
CS1000H is
referred to as
CS1000FP
once upgraded

The WebUI will only display CS1000. To identify a CS1000H or CS1000FP look at the
controller shelf capacity or navigate to Manage >> Array >> [Select array] and view the visual
representation.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-10


CS-Series Write Operations
CS-Series Write Operations

AF-Series features shared with the CS-Series


 Compression
 Stripe writes Variable Block
Compression
 Encryption
 Sweeping Variable Block
Sequential Stripe Write

Encryption

Sweeping

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-13


CS-Series Write Operation

8K 4K 4K
ack

Mirrored
5) Blocks are copied into DRAM
a) What happens next depends of
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)
type of array
8K 4K 4K 8K 4K 4K
b) All Flash Array
1. …
c) Hybrid Flash Array DRAM
1. Variable block compression 8K 4K 4K
Variable Block
is applied Compression
Sequential Stripe 4.5MB
2. Blocks are formed into a
4.5MB sequential stripe
write
Index
3. Sequential stripe is written
to hard disk drives
CS-Series
4. Cache worthy data and any
data destined for pinned
volumes is also sent to SSD
5. Blocks are indexed
Index

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-14


CS-Series: Triple+ Parity RAID
CS-Series Triple+ Parity

 Utilizes the same Intra-Parity as the AF-Series


 Left synchronous rotation – first two parities (P & Q) are rotational and the
third parity (R) is non-rotational.
 Supports the loss of three disks.

Stripe 0 D1 D2 D3 P Q R

Stripe 1 D2 D3 P Q D1 R
The system will shutdown if
Stripe 2 D3 P Q D1 D2 R there are three disk failures
prior to any one of those failed
Stripe 3 P Q D1 D2 D3 R disks being rebuilt.

Stripe 4 Q D1 D2 D3 P R

Stripe 5 D1 D2 D3 P Q R
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-17
CS-Series Read Operations
CASL Architecture

Mirrored
1) Read from NVDIMM
2) If not found, check DRAM
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)
3) If not found, read from SSD
• If found, validate checksum, 1 8K 4K 4K 8K 4K 4K

decompress, and return data


4) If not found, read from disk using the 2 DRAM
index on the SSD for a quick lookup
Variable Block
• If found, validate checksum, Compression
decompress, and return data
5) And, if cache-worthy, write to cache

3 4
The CS-Series arrays have the CS-Series
same changed block benefit
demonstrated in the AF-Series
arrays. 5
Index

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-19


AF-Series and CS-Series Architecture Review
AF-Series/CS-Series Write Operation Comparison

8K 4K 4K
ack

Mirrored
Blocks are copied into DRAM
a) What happens next depends of
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)
type of array
8K 4K 4K 8K 4K 4K

DRAM
8K 4K 4K
Variable Block Variable Block
AF-Series Sequential Stripe 10MB
Deduplication Compression
CS-Series

Sequential Stripe 4.5MB


Index
Index

CS-Series AF-Series

Triple+ Parity with


Triple+ Parity Integrated Spare
Index
Index

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-21


Scale-to-Fit with the CSx000
Nimble Storage CSx000 Scale-to-Fit

CS7000 4 Node
230K IOPS CS7000
AFS Cluster
Up to 25.6TB
of flash Up to 920K IOPS
Multi-PB Capacity

CS5000
IOPS based on 120K IOPS
70% Read and
30% Write
workload CS3000
50K IOPS
SCALE UP

CS1000 /
CS1000H
35K IOPS
SCALE DEEP

Add expansion shelves to meet storage growth

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-23


CSx000 Scale Deep

SCALE DEEP

Add expansion shelves to meet storage growth

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-24


Scale Deep with the ES2 Expansion Shelf

The ES2 Expansion Shelf

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-25


Drive Layout – ES2

 4U24 Chassis
 24 x 3.5” Slots carry 21x HDDs + 3x DFCs
 HDDs: 18 + 3 RAID
» New Nimble-branded HDD carriers
 DFCs : Bank A pre-configured with 3 SSDs
» Bank B available for cache upgrades

Cache: 3x Dual Flash Carriers


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-26
CSx000 Scale Deep

Start with what is needed…

Scale deep as needs change

CSx000 supports up to 6 ES2 expansion shelves

CSx000 CSx000 CSx000 CSx000 CSx000 CSx000 CSx000


1X ES2 2X ES2 3X ES2 4X ES2 5X ES2 6X ES2

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-27


ES2 Rear View – Same as AFS2

SAS
Out port

SAS In
Expander FW Status
port
Green-OK
Red-FW Failed to Load SAS Link Status
Green-12Gbps Red-Fault
Orange-6Gbps Off-No link

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-28


CSx000 Expansion Cabling

CSx000 ES2
Controller Shelf Expansion Shelf

ES2 ES2
Expansion Shelf Expansion Shelf

ES2 ES2
Expansion Shelf Expansion Shelf

ES2
Expansion Shelf

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-29


Activating a New Shelf (under Manage >> Arrays >> [select one array]

Activated

Once and expansion shelf is activated, it cannot be removed from the solution.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-30


CS7000
230K IOPS
AFS
Up to 25.6TB
of flash

CS5000
IOPS based on
50% Read and
120K IOPS CSx000 Scale Up
50% Write
workload CS3000
50K IOPS
SCALE UP

CS1000 /
CS1000H
35K IOPS

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-31


Do you need to expand cache on your CS-Series array?

Nimble OS >> Monitor >> Performance

InfoSight >> Manage >>


Assets >> Cache tab

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-32


CSx000 Cache Upgrade

Increase On-Board Flash Cache

Cache: 3x Dual Flash Carriers


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-33
CSx000 Cache Upgrade

Populate bank B of all expansion shelves with


additional cache

Bank B

Cache: 3x Dual Flash Carriers


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-34
Adding / Removing SSDs

When an SSD is removed… When an SSD is added…

Cache Pool Cache Pool

… the Cache Pool shrinks by the size of … the Cache Pool grows by the size of
the removed SSD the added SSD
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-35
All-Flash Shelf

 SSDs can be ordered in 4-packs


» CSx000 supports up to twelve 4-packs for a total of 48 SSD
» CS1000H supports up to six 4-packs for a total of 24 SSD

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-36


CSx000 Expansion Cabling

CSx000 All-Flash Shelf


Controller Shelf

ES2 ES2
Expansion Shelf Expansion Shelf

ES2 ES2
Expansion Shelf Expansion Shelf

ES2 ES2
Expansion Shelf Expansion Shelf

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-37


CSx000 Controller Upgrades

 Non-disruptive controller upgrades within CSx000 controller family


 Cache upgrades may be required to meet MFR, Memory to Flash Ratio
 Timeless Storage supported at launch

Upgrade Upgrade Upgrade


controllers controllers controllers
to CS3000 to CS5000 to CS7000

CS1000 CS3000 CS5000 CS7000

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-38


CS1000H “Half-pop” Controller Upgrades

 11TB must be upgraded to 22TB “Full Pop” before upgrading to CS3000


 Capacity and controller upgrades are non-disruptive
 No further controller upgrades beyond CS3000 are supported
 Timeless Storage supported at launch

Capacity Controller
upgrade to upgrade to
full-pop CS3000

C1000H-11T C1000H-22T C3000H-22T

The WebUI will only display CS1000 or CS3000. To identify a CS1000H-11T, CS1000-22T, or
CS3000H-22T look at the controller shelf capacity or navigate to Manage >> Array >> [Select
array] and view the visual representation.
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-39
Nimble Storage CS-Series Scale-to-Fit

4 Node
CS7000
CSx000 Scale out Cluster
Up to 1.4M IOPS
 Same capabilities as the AF-Series Multi-PB Capacity

» Add or remove arrays non-disruptively


» Automatically rebalances data
» Non-disruptively perform a technology refresh
» Migrates data to upgraded hardware
» 1-4 arrays per group
» Add and remove arrays on the fly
» Uses a single management IP
» Mix and match:
• CSx00
• CSx000
• AFx000

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5a 2-40


Module 03:

NimbleOS WebUI Introduction

Course Revision 5
Module 03 Objectives

At the end of this module, you will be able to:


 Access and navigate the NimbleOS WebUI
 Understand user administration
 Describe how to use Microsoft’s Active Directory with user administration
 Understand how to monitor the array with the WebUI
 Understand how to update NimbleOS

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-2


Nimble OS WebUI Tour
Accessing the WebUI Interface
https://{arrays management IP address}

https://10.206.9.110

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-4


WebUI Tour

Capacity

Events

Performance

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-5


WebUI Tour

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-6


User Administration
User Administration

Administration >> Security >> Users and Groups


 RBAC = Role Based Access Control
» Multiple Users
» Pre-defined Roles
» Default Username: admin
» Support for Microsoft
Active Directory

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-8


Roles

 Administrator (Default “admin” user)


» Complete access and security capabilities – no
limits.
 PowerUser
» Same as Administrator but without the privilege to
modify security settings. Eg., no user account
creation and password changes.
 Operator
» Same as PowerUser but without the privilege to
delete data and operations that are rarely
performed. Eg., volume delete.
 Guest
» Read Only privilege except for security settings.

See Appendix A of the Nimble Storage User Guide for a complete listing of capabilities by command

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-9


User Management

 Usernames can’t be any of the system users like “root”


 SSH key based password-less login is supported for all users in the Administrator role
 All user accounts except for built-in Administrator can be enabled/disabled and removed

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-10


Joining Microsoft Active Directory
Joining Microsoft Active Directory

1 Ensure AD has an existing


group for use with the array.

Create a new AD group if


>
needed. New >> Group

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-12


Joining Microsoft Active Directory

1 Ensure AD has an existing


group for use with the array.

Create a new AD group if


>
needed. New >> Group

> Provide the group name, set


the scope, and group type.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-13


Joining Microsoft Active Directory

1 Ensure AD has an existing


group for use with the array.

Create a new AD group if


>
needed. New >> Group

> Provide the group name, set


the scope, and group type.

> Add group members.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-14


Joining Microsoft Active Directory
2 Navigate to Microsft Active Directory in the WebUI

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-15


Joining Microsoft Active Directory
3 Complete all fields

Click Join 4

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-16


Joining Microsoft Active Directory

5 Test the connection

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-17


Assigning Users/Groups to Microsoft Active Directory
1 Navigate to Users and Groups in the WebUI

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-18


Assigning Users/Groups to Microsoft Active Directory
2 Click Add and then select User or Group

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-19


Assigning Users/Groups to Microsoft Active Directory
3 Provide name, desired role, description (optional) and desired inactivity timeout value

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-20


Microsoft Active Directory
3 Test the login

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-21


Introduction to Monitoring With the WebUI
Monitor >> Space

Review Select what


overall data to
system usage display

Open Review data


InfoSight for reduction
predictive savings
analytics of
space
utilization
Set timeframe
Display
and pause
usage,
the capture
savings, or
for analysis
both

Breakdown of
usage and
savings by
application
category
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-23
Monitor >> Performance

Pause data Select


stream timeframe
and volumes
to display

View performance
metrics for selected
timeframe on selected
volume(s)

Launch InfoSight
CS-Series – monitor to view a
cache hit/miss. If miss breakdown of
is high, upgrade cache latency factors

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-24


Monitor >> Interfaces

Pause data
Select
stream
timeframe to
display

View performance
metrics for each
interface

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-25


Monitor >> Replication

 Monitor replication throughput for


both send and receive

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-26


Monitor >> Connections

 View the initiator and number of connections for each volume Mouse
over
connection
number to
view
Mouse
connection
over
addresses
volume to
view
details

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-27


Monitor >> Audit Log

 View activity of each WebUI user

Filter audit log

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-28


Manage >> Arrays >> [Select Array]

 Graphical display of array health


CSx000

AF-Series

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-29


Events

 View list of system events and full event description

Filter event log

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-30


Updating NimbleOS
Controllers – Nimble OS Upgrade

One-click, zero-downtime Nimble OS upgrades


 Before you begin:
– Check your current version
– Obtain the most recent version
– Check system health

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-32


NimbleOS Upgrade Process

1. Load new firmware to standby


2. Reboot standby to run new rev.
Firmware
3. Load new firmware to other controller
4. Reboot active to activate new rev. – causes failover and the
standby becomes active

Controller A Controller B

Active
Standby Standby
Active
Failover
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 3-33
Module 04:

Introduction to Customer Support & InfoSight

Course Revision 5
Module 04 Objectives

At the end of this module, you will be able to:


 Present Nimble Support’s mission statement
 Understand the InfoSight Customer portal
 Describe support coverage and logistics

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-2


Introduction to Support
Customer Support Mission Statement

Our Mission
Maintain a maniacal focus on providing the
industry’s most enviable Customer Support

 It’s not just a Customer Support mission

 Its part of Nimble’s DNA across the board

 We monitor, assess and evolve as fast as necessary

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-4


Why Do We Do Things This Way?

 We believe users should expect data to be available always


and instantly, regardless of media type.
 We believe in lowering risk and accelerating business
outcomes.
 We believe solutions should be easy to consume.
 We believe solutions should not have architectural dead ends

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-5


Consider These Issues…

 All your hosts are sporadically disconnecting from VMware.


Nobody can figure out why.
 A VM host experiences high latency. But only after a certain
time.
 An application has abnormally slow I/O speeds, regardless of
how fast the storage it’s attached to.
 A server exhibits unstable behavior unpredictably.
 App data growth is much faster than anticipated

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-13


Consider These Issues…

 All your hosts are sporadically disconnecting from VMware.


Nobody can figure out why.
 A VM host experiences high latency. But only after a certain
time.
How Are You
 An application Solving Such
has abnormally slowProblems
I/O speeds,Today?
regardless of
how fast the storage it’s attached to.
 A server exhibits unstable behavior unpredictably.
 App data growth is much faster than anticipated

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-14


InfoSight Customer Portal
InfoSight Engine: Data Collection

Comprehensive Telemetry
Comprehensive Telemetry
• Log
 Log filesfiles
• Sensors (>30Million per day!)
 Sensors (>30Million per day!)
• Health status
 Health status

5-minute Heartbeats
• Array <------> Nimble Support
InfoSight
• Proactive Monitoring
Engine

>30M sensor values per array per day


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-17
Collecting sensory data

Reads vs Writes Temperature

Network Statistics
CPU Utilization
Snapshot status Write Latency

Statistics
Write IOPS
Compression

Cache Utilization Read IOPS


PS Voltages
Replication status
Fan Speeds
Read Latency
Data block size Block Sizes
Random vs Sequential IOPS

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-18


Preventing Problems to Avoid Downtime

 9 out of 10 problems are automatically detected


 Rapid root-cause analysis cuts hours of
troubleshooting
 Correlated analysis enables visibility across
the application-to-storage stack
 No level-1 and level-2 support pain Measured
uptime

InfoSight predicts and prevents issues across the application-to-storage stack,


enabling you to avoid disruption and time wasted dealing with infrastructure problems

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-19


Real-time monitoring & alerts

Instant Email Alerting


 Easy and flexible setup
 System alerts to Nimble Support and the
customer
» All warning level and alerts
» Proactive disk monitoring
» System utilization
» Protection (Snapshot) monitoring
From: <customer>
Sent: Monday, June 20, 2011 2:03 PM
 Automated case created To: alerts
Subject: Nimble Alert on <array name> - WARNING: System temperature is warm
 Nimble support proactively Time: Mon Jun 20 14:03:01 2011
contacts customer
Type: 12323
Id: 76501
 Ability to ‘remote’ in for real- Message: current temperature for sensor bp-temp1 at backplane left side is 44 C
time troubleshooting
Array name: <name>
Serial: <AA..>
Version: 1.0.6.1-9650-opt
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-27
Visibility Beyond Storage

InfoSight VMVision gives


visibility up to the VM layer

• Determine VM latency
factors: storage, host
or network
• Take corrective action
on noisy neighbor VMs
• Reclaim space from
underutilized VMs
InfoSight VMVision pinpoints VM-related issues
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-28
Predict Future Needs and Simplify Planning

 Accurately forecasts future capacity,


performance and bandwidth needs
 Prescriptive guidance ensures optimal
long-term performance
 Predicts performance hotspots and
tells you how to avoid them
 Eliminates planning guesswork

Leverage predictive analytics to identify future needs and potential hot-spots specific to
your environment, with prescriptive guidance to ensure optimal long-term performance

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-29


Visualize Intuitively

• Executive dashboards give you


peace-of-mind that everything is
running perfectly

• Correlated visualization provides a


use-case view of what’s happening
across the “applications-to-storage”
stack

• Manage your global storage


environment from anywhere

Complete visibility through the cloud to all information you need to maintain a resilient
environment and ensure smooth operations

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-30


InfoSight: A Customer Portal and Much More

Blacklisting and
Dynamic Upgrade
Paths

Software & Takeover Fault Behind-the-Scenes


Analysis Operational Monitoring

Support Aggregate
Automation Studies

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-34


Proactive Wellness: Powered By InfoSight Data Sciences

Proactive Support Prevents Problems

of the cases automatically


> 93% opened by Nimble

field proven availability


across all GA software
99.999% releases

of support cases auto-


86% closed
by Nimble

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-39


Support Coverage and Logistics
How to get level 3 support – the traditional method

Call Fill out verbal Caller motives


Level 1
Support questionnaire questioned

Answer more Wait on hold


Provide
questions, provide Level 2 or wait for a
logs
logs. Again. call back

Wait on hold
Caller motives Welcome to level 3
or wait for a
questioned support
call back
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-42
Level 3 support as easy as 1-2-3

nimblestorage

1 Pick up the phone 2 Dial 1-877-364-6253 3 Level 3 support

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-43


Global Support Centers – Seamless coverage

EMEA
Reading, UK
AMERICA’S HQ
San Jose, Ca RTP Japan
Raleigh, NC Tokyo

APAC
Local toll free numbers Singapore
land in any of 5 Centers
around the globe seamlessly
7 days a week, 365 days a year.

Note: Japan install base is handled by TSC.


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-44
Support for SmartStack

 Nimble SmartStack Integration Support


• Support.SmartStack@nimblestorage.com or Option 7 from 1-877-3Nimble
• Coordinate third-party vendor support
• No additional charge

 Cisco Solution Support for Critical Infrastructure


• Cisco will offer issue management and solution expertise
• Will coordinate with Nimble and other vendors for support
• Fee-based option on top of product support contract

 All configurations validated and supported by participating vendor partners


• Product Certifications
• Hardware Compatibility Lists

 Backed by TSAnet as global alliance framework that enables fast-track


technical support cooperation between several partners
• SLA’s are required for each vendor and cases are opened by customer

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-47


Service Logistics Regional Capabilities

 Next Business Day – “NBD”  4 Hour Parts Delivery  4 Hour Parts Replacement - Onsite Engineer
• United States
• United States
• Canada
• Canada • United States
• Europe
• Europe • Canada
• Australia
• Australia • Europe
• New Zealand
• New Zealand • Australia
• Bermuda
• Bermuda • New Zealand
• China
• China (Major Cities) • China (Major Metros)
• Hong Kong
• Hong Kong • Hong Kong
• Indonesia
• Indonesia • Indonesia
• Israel
• Israel • Israel (Tel Aviv)
• Kenya
• Thailand • Thailand
• Thailand
• United Arab Emirates • United Arab Emirates
• Nigeria
• Singapore • Singapore
• United Arab Emirates
• Malaysia • Malaysia
• Singapore
• Philippines • Philippines
• Malaysia
• South Korea • South Korea
• Philippines
• India (Major Cities) • India (Major Cities)
• South Korea
• South Africa (Johannesburg) • South Africa (Johannesburg)
• India
• Taiwan • Taiwan
• South Africa
• Vietnam (Major Cities) • Vietnam (Major Cities)
• Taiwan
• Vietnam
How do we decide on new locations?
• We look at sales/partner coverage in the area
• Anticipate a minimum of 5 installed systems in the area within next 12 months
• 10 to 14 days to open a new Depot in most locations around the world
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 4-51
Module 05:

Pre-Installation, Racking, and Cabling

Course Revision 5
Module 05 Objectives

At the end of this module, you will be able to:


 Use the pre-installation workbook
 Understand rack mounting principles
 Cable an array
 Understand networking and fibre channel concepts

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-2


Pre-Installation Workbook
Pre-Install Checklist

 Collect all necessary data to perform an installation


 Organized in the same order that you will be entering in the data
 Can be left with the customer

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-4


Pre-Install Checklist

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-5


Racking
Use the Quick Start Guide

Download Quick Start Guides from InfoSight before starting

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-11


Racking

AF = 4U CSx000 = 4U

AFS2 = 4U AFS = 4U

ES2 = 4U

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-12


Racking

Precaution
 Ensure there is at least 24” of cable slack
at the back of the array or shelf.
» Allow enough room to slide the array or
expansion shelf out 12” from the front of the
rack in order to replace a center chassis fan.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-13


Cabling
Power

Connect one to
commercial power and
one to backup power.

Controller A Controller B

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-15


Networking Terminology

 Interface Pairs
»Controller A eth1 & Controller B eth1
»IP addresses float between

Interface Pairs

Controller A Controller B
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-16
Ethernet Ports

eth5 eth6

eth1 eth2 eth3 eth4 SAS Out Ports

When cabling to switches ensure you cable even


ports to one switch and odd ports to the other.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-17


Fibre Channel Ports

fc5 fc9 fc10 fc6

eth1 eth2 fc1 fc2 SAS Out Ports

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-18


Networking Concepts
Understanding IPs

 Management Network
» The Array Management IP address
» The 2 Controller Diagnostic IP addresses

 Data Network
» The Data IP addresses

 Discovery Addresses
» iSCSI Discovery Address

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-20


Discovery Addresses

 Typical iSCSI Discovery


iSCSI
Send Targets
Discovery

» Returns a list of Targets from the Array


» Makes it easy to find Volumes from the Host

Targets
Volume1
Volume2
Volume3

 Nimble iSCSI Discovery (per Subnet)


» Acts as a load balancer (Automatic Mode) Connect to Data IPs
» Redirects to a Data IP address to be used to connect to the array Discovery
.20 .21
(172.16.0.20) .22
• Used to virtualize connections to each array in the group 2 times
• Used by the array to distribute/balance hosts across several data IPs
» Need a Discovery for each Data Subnet
Data Addresses
• Discovery cannot redirect across subnet boundaries 172.16.0.21
Redirects & Returns 172.16.0.22
from Array:
172.16.0.21 & 17.16.0.22

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-21


Typical Subnets

One Shared Network Two Dedicated Networks Advanced Networks


Management Subnet Management Subnet
• Array Management Address • Array Management Address
• Acts as Discovery IP • Acts as Discovery IP
• 2 Controller Diag Addresses • 2 Controller Diag Addresses
Management • Replication (by default, 2.x) • Replication (by default, 2.x)
• Array Management Address
• Acts as Discovery IP
• 2 Controller Diag Addresses
• Replication (by default, 2.x)

Data Subnet Data Subnet Data Subnet


Data • All Data Addresses • 1 Set (Data) • 1 Set (Data)
• All Data Addresses • Discovery IP • 1 Discovery IP • 1 Discovery IP

Note: in 2.x, every array in the group must have access to each subnet
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-22
iSCSI Switch Selection Guidelines

Network Design
 Best if dedicated, redundant iSCSI network
 Otherwise, use VLANs to keep iSCSI traffic separate

Switch Attributes
 Good Quality Layer 2 or Layer 3 Managed
» Stacked preferred, but be aware of issues around stacking such as what happens when the
master switch in the stack fails and upgrading of switch firmware may require an outage
» ISLs at least, concern is under or over specifying the total bandwidth required
» ISLs – may want to use IP Address Zones to prevent iSCSI traffic from crossing the ISL
 Support for Jumbo Frames (with Flow Control) is Desirable
 Non-Blocking Backplane
» Bandwidth of the backplane >= (# of ports) * (bi-directional port speed)

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-23


Optimizing Switch Links: IP Address Zones

 Challenge
» How to minimize Inter-Switch Links for traffic? 172.16.0.81 172.16.0.82
eth1 eth2
» Especially if connections are automatic? Discovery redirects to Discovery redirects to
172.16.0.21 172.16.0.22
Single Data Subnet
 Answer: IP Address Zones
» Discovers Data Address via Discovery IP
» Uses Host IP address to select Data Address Avoid
» Examples: Traffic?
• Even/ Odd: Odd vs Even #s kept together 172.16.0.21 172.16.0.22
(Data IP) (Data IP)
• Bisect: Low vs High #s are kept together
• Default: Single (no division)
Using Odd/Even zone .21 Management
.22
Subnet
172.16.0.20 .20
(Discovery)

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-24


Best Practices

Practice Notes
Do not use Spanning Tree Protocol Do not use STP on switch ports that connect to iSCSI initiators or the
(STP) or use PortFast Nimble storage array network interfaces.
Configure flow control on each switch Configure Flow Control on each switch port that handles iSCSI
port connections. If your application server is using a software iSCSI initiator
and NIC combination to handle iSCSI traffic, you must also enable Flow
Control on the NICs to obtain the performance benefit.
Disable unicast storm control Disable unicast storm control on each switch that handles iSCSI traffic.
However, the use of broadcast and multicast storm control is
encouraged.
Use jumbo frames when applicable You must have jumbo frames enabled from end to end for them to work
correctly
Testing Network connectivity Use the Ping command to test network connectivity and to help
determine if jumbo frames are enabled across the network, example:
vmkping -d -s 8972 x.x.x.x

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-25


Fibre Channel Concepts
Fibre Channel Components

Dual Fabric Design Array (Target)

Configure one FC port on


controller A to FC Switch A and
one FC port on Controller A to FC
Switch B

Configure one FC port on 2 3 4 2 3 4

controller B to FC Switch B and CISCO UCS

ID
6296 UP CISCO UCS

ID
6296 UP

one FC port on Controller B to FC


STAT 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 STAT 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

FC Switch A FC Switch B
Switch A

All FC ports must be in the same

S
A
S
S
A

PCIe3
PCIe4

PCIe1
CIMC

PORT 1

PORT 0
S

S
A

PCIe5

PCIe 2
zone
PSU
2

650 W AC
1 2

PSU
1 
M

Server (Initiator)

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-27


Fibre Channel Config Steps

1. Verify Fabric/Zoning
» Single Initiator
» Single Target
2. Configure Initiator Group
» Server port
» Volume
3. Create volume and link to the initiator group
4. Verify connection from volume to assigned initiators

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 5-28


Module 06:

Array Initialization & Setup

Course Revision 5
Module 06 Objectives

At the end of this module, you will be able to:


 Perform an array initialization
 Initialize an array configuration
 Understand port and firewall considerations
 Understand SNMP setup
 Understand SYSLOG use
 Understand NimbleOS’s event log

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-2


Array Initialization
Before You Begin

Important: The computer used to initially configure the array must be on the same
physical subnet as the Nimble array, or have direct (non-routed) access to it.
• Ensure Adobe Flash Player is installed

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-4


Prerequisites

 Before launching the NWT:

» Set a static IP: Set your IP address to the same subnet that your array management IP address
will be on.

» Have your array controllers A & B correctly cabled to your switch fabric per the previous
drawings.

» Complete all your switch configurations for Flow Control, Jumbo Frames, Spanning tree,
Unicast, etc.

» Install the Nimble Windows Toolkit (NWT) on the Laptop or Server you are using for the
installation.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-5


Nimble Windows Toolkit Installation

 NWT 3.3 includes


» Nimble Set-Up Manager (NSM)
• Available via InfoSight
» Nimble VSS Requestor
» Nimble VSS Provider
» Nimble Connection Manager
(NCM)*
» Nimble Connection Service
(NCS)*
» NimbleDSM*

* Windows Server 2008 R2 or 2012 /


MPIO

Reboot Required after Install


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-6
NWT– Nimble Array Setup Manager

Shows
• Array(s)
• Model
• NOS Version Number

Steps
• Select Array to Setup
AF-000917 All Flash Array
• Click Next Button
• The next Popup screen informs
you that Setup Manager is going AF-000917

to connect using the IP address


indicated in the Popup

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-7


End User Agreement

Accept the End User


Agreement by clicking in
the checkbox and the
“Proceed” button

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-8


3.x Arrays, Select “New Group” or “Add Array”

Option 1: New Group


» Configure first array
» Mgmt IP, etc.
» See next step
AF-000917
All Flash Array
Option 2: Add To Group
» Inherits group’s identity
& network
» See Multi-Array Section
for example

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-9


NWT– Nimble Array Setup Manager

Enter the array name


• Make it useful such as
• Row and rack number AF-000917

• Data center & app

Set your management IP address Subnet


mask & Default Gateway

Enter and confirm your array password

Click “Finish”

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-10


NWT– Nimble Array Setup Manager

You should get this screen


after a few seconds.

Click “Continue”, your default


browser window will be
opened and directed to the
management IP. If it does
not, open a browser and
point it to the management
IP address

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-11


Array Initialization Using the CLI
Array Initialization

The steps to initialize an


array using the CLI are:

1. Connect to the array


using the provided USB
dongle and a terminal
program

2. Login to the Array

3. Run setup and use the


installation workbook
to answer presented
questions

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-14


Initial Array Configuration
Initial Array Configuration
1 Launch the WebUI and Login

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-16


Configure Subnets
2 Click “Add Data Subnet” to add a subnet for your data network or networks

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-17


Adding Data Networks
3 If you have additional data networks you want to add, click the “Add Data Subnet” again.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-18


Interface Assignments
4 Assign Interfaces to desired ports

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-19


Domain & Time Configuration
5 Configure the domain and DNS server(s)

6 Configure time zone and NTP Server

Next add the domain, DNS, Time Zone &


NTP server information
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-20
Configure Support Information
7 Configure email alerts, auto support, and HTTP proxy

The ‘From’ address should identify the array

Add one or more destination


addresses

Add your SMTP Server information


NOTE:
Additional SMTP
Server settings are
available after Check “Send AutoSupport …”
initial
configuration
Select Alert Type
Click ‘Finish’ to complete setup
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-21
Setup Complete

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-22


AutoSupport Setup and Testing
1 Navigate to AutoSupport/HTTP Proxy
Administration >> Alerts & Monitoring >>
AutoSupport/HTTP Proxy

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-23


AutoSupport Setup
2 Enable AutoSupport and test

Check – Send AutoSupport data to Nimble Storage

Click – Test AutoSupport Settings

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-24


AutoSupport Setup

Once the “Test


AutoSupport
Settings” have
completed a green
or red dot will be
displayed indicating
the test status.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-25


AutoSupport Setup

Clicking on the dropdown


will reveal all the tests
that were performed.

If one or more tests


failed, you will see a
“red” dot to the left of the
test that failed.

Send the AutoSupport to Nimble Support


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-26
AutoSupport Setup

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-27


Setup Email Alerts
1 Navigate to Email Alerts
Administration >> Alerts & Monitoring

Email Alerts

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-28


Setup Email Alerts
2 Complete all fields and test

Add your SMTP Server information

Select Alert Type

Click “Test”
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-29
Setup Email Alerts
3 After testing Email Alerts, click the “Save” button to save the settings

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-30


Controller Failover Check

1 Navigate to Manage >> Arrays

Manage

Arrays

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-31


Controller Failover Check

Click the Array 2

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-32


Controller Failover Check

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-33


Controller Failover Check

4 Confirm the message

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-34


Controller Failover Check

5 Verify the Standby Controller is now set to “Active” and that all connections are good.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-35


Port and Firewall Considerations
Incoming Server Ports

To use: Open local For local IP addresses: Notes:


port:
SSH 22 Array management IP
HTTP 80 Array IP
GUI (HTTPS) 443 Array management IP HTTP (port 80) communication is
redirected to HTTPS

iSCSI 3260 Discovery and data IP Needed for data access


SNMP 4290 SNMP daemon
GUI charts, NPM 4210 Array management IP

Control 4211 Array management IP


Replication (data) 4213 Array management IP

See Appendix B in Nimble Storage Installation and Configuration Guide 2.2 for complete listing
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-37
Outgoing Server Ports

Port # Service Protocol IP Address

N/A Manual SCP of cores to Nimble SCP support.nimblestorage.com

N/A AutoSupport, heartbeat HTTPS nsdiag.nimblestorage.com

N/A alerts* HTTPS nsalerts.nimblestorage.com

N/A software downloads HTTPS update.nimblestorage.com

2222 secure tunnel SSH hogan.nimblestorage.com

25/UDP & TCP** SMTP SMTP SMTP server IP

*An array sends alerts through HTTPS POST back to Nimble Support if AutoSupport is enabled

**Default (configurable)
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-38
SNMP
Setup SNMP

 SNMP versions 1, 2 and 2c

 Configure through the NimbleOS GUI (Administration > SNMP) or CLI snmp command

 Nimble Storage arrays send traps but do not receive them

 UDP traps are supported

Arrays use the alert level setting for email alerts to determine the
events that are sent as SNMP traps
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-40
SYSLOG
SYSLOG

 Nimble Storage arrays use the standard SYSLOG protocol

 Support for Red Hat Enterprise Server and Splunk implementations of SYSLOG

 Nimble Storage SYSLOG stores


» records of administrator manipulation of the array
» history of alerts
» array issues

 SYSLOG files can be shipped from an array to a centralized, remote server

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-42


Event Logs
System Event Monitoring

 Events are displayed on the Events page of the NimbleOS GUI


» Access by clicking “Events” located on the main NimbleOS menu selection

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-44


LAB 1

Tasks:
 Launch Nimble Setup Manager (NSM)
 Recall key configuration steps
 Launch array GUI and configure basic array parameters including autosupport

Time to complete: 20 Minutes

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-45


LAB 1b (Optional)

Tasks:
 Utilize the CLI to complete initial setup

Time to complete: 15 Minutes – this is an optional lab

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 6-46


Module 07:

Working with Nimble Storage Volumes

Course Revision 5
Module 07 Objectives

At the end of this module, you will be able to:


 Understand basic volume concepts
 Understand performance policies and how to create one
 Understand access control and initiator groups and how to create them
 Understand protection templates and how to create one
 Understand volume collections and how to create them
 Understand volume pinning
 Create a volume using the WebUI

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-2


Volume Concepts
Volumes Overview

Physical storage
resource

Volume Logical storage


resource
Triple+ Parity
Storage Pool

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-4


Thin Provisioning

Triple Parity
RAID or Triple Consumed Space
Volume
Parity+

Space from the pool is


consumed as data is
written

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-5


Volume Reserves

Triple Parity
Volume
Volume Reserve
RAID or Triple
Parity+

A reservation reserves a
guaranteed minimum amount
of physical space from the
pool for a volume
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-6
Volume Quotas

Volume Reserve
Volume Quota
Volume

Pool

A quota sets the amount of a volume that can be consumed before an alert is sent and
writes are disallowed.
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-7
Snapshots

Snapped Volume
Snapshot Reserve – An accounting for a
set amount of space that will be
guaranteed available for the snapshot.
Snapshot Reserve
Snapshot Quota – An accounting for the
Snapshot Quota
total amount of space a snapshot can
consume.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-8


Performance Policies
What is a Performance Policy?

 A performance policy is used to define parameters for volumes assigned to the


policy:
» Pre-configured policies can
be used or a custom policy
can be created.
» NOTE: AF-Series arrays
performance policies include
a column for deduplication

AF-Series Performance
Policy list

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-10


Custom Performance Policies

 The following parameters are set when building a custom performance policy:
» Application Category
» Storage Block Size
» Compression On/Off
» Caching On/Off
» Space Management Parameters
» Deduplication Enabled/Disabled
- AFA only

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-11


Access Control / Initiator Groups
Access Control

 Access control – which hosts have access to a volume based on an initiator


group
 Best Practice: Always limit access to a host initiator group
 NOTE: Allow multiple initiator access – for use with clusters, not MPIO

Fibre Channel iSCSI

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-13


Initiator Groups

 A set of host initiators that can be assigned access to a specified volume via Access
Control.
 Can be created at volume creation or as a separate task
» Manage >> Initiator Groups

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-14


Initiator Groups

 The parameters within an Initiator Group will depend on the protocol - iSCSI or Fibre Channel

iSCSI Initiator Group Fibre Channel Initiator Group

NOTE: An IQN can only be assigned to one initiator group


© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-15
iSCSI Initiator Groups

• Initiator Groups are applied to:


• Volumes
• Volumes+Snapshots
• Snapshots only

• Can limit to specific subnets

• Supports up t0 255 subnets

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-16


iSCSI Initiator Group Target Subnets

 Limiting Subnets
» Only targets on specified subnets can be
accessed by an Initiator Group

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-17


Fibre Channel Initiator Groups

• Initiator Groups are applied to:


• Volumes
• Volumes+Snapshots
• Snapshots only

• Limit based on World Wide Port


Name, WWPN
• Use of a Fibre Channel fabric
alias is supported which then
automatically completes the
WWPN

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-18


Volume Collections
Volume Collection

 A grouping of volumes that share snapshot/replication


schedules
» Manage >> Protection >> Volume Collections
» Created via a blank schedule or Protection Template
» All volumes in a group will be snapped and replicated as a
group
 Best practice: Create a volume collection for each
application
Volume Collection
» Oracle Database and log files
 Ensure you do not create overlapping schedules

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-23


Creating A Volume Collection

 Name the Volume Collection

 Using a Protection template (for


Schedules)
» Or create a custom schedule

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-24


Volume Collection >> App Synchronization

• App flushes/quiesces I/O


while we take a snapshot
and then unfreezes

• VMFS consistent snapshots

• SQL consistent snapshots

• Exchange consistent
snapshots

• SQL/Exchange uses MS
VSS framework and
requires NPM on the
Application Host – more
later
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-25
Add Schedules

 Sets Snapshot Schedule


» Include Retention Policy
» App Synchronization
» Replication

 Create Multiple Custom Schedules


» Examples
• Hourly
• Daily
• Weekly
» Do not create overlapping schedules

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-26


Volume Pinning – CS-Series Only
Volume Pinning – CS-Series Only

 Manage >> Volumes >> [Selected


Volume] >> Edit >> Performance
 Pins the entire volume in cache
» Writes are still held on disk
 Should only be used on workloads that
require peak performance

Cache is a shared resource. Pinning a


volume may impact the performance of
other volumes.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-28


Creating a Volume Via the WebUI
View of Volume Wizard

 Same wizard is used to create both Fibre Fibre Channel Volume


Channel or iSCSI
 Creates protected volume in 4 easy steps
» General Properties
» Space
» Protection
» Performance
 Build volume components separately or as
needed
» Initiator Groups
• Fibre channel and iSCSI dependent
» Custom Performance Policies
» Volume Collections
• Custom Protection Templates
iSCSI Volume

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-30


Volume Create Wizard: Step 1 – Performance & Access

 Volume Name
» Helps to include host and app in
volume name

 Performance Policy
» Use existing policy (based upon
app type)

 Set Access Control


» Best to include at least 1 initiator group

» NOTE: Multiple initiator access is for


apps such as clustering, not MPIO.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-31


Volume Wizard: Step #2 – Size & Space

 Size
» Set the volume size seen by the
application

Click to
 Thresholds expand
» Best to use the defaults
» Setting Reserve > 0 reduces thin
provisioning

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-32


Volume Wizard: Step #3 - Protection

 Protection
» Best to use a volume collection for all
of your volumes

 Protection Schedules
» Displays Protection Schedule
associated with selected volume
collection

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-33


Volume Wizard: Step #4 - Performance

 Caching
» Normal – cache all hot data
» Pinned – cache entire volume

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-34


Viewing Volume and Replica Usage

Volume
Filter
Volume Volume
Summary Usage Key

 Under “Manage > Volumes”


» Displays all pertinent volume data

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-35


Nimble Windows Toolkit & Nimble Connection Manager
Nimble Windows Tools

 Download NWT from InfoSight


 Prerequisites must be installed prior to NWT or the installation will stop
» .NET Framework 4.5.2 or above
» Microsoft PowerShell 3.0 (required to run Nimble PowerShell Cmdlets)
» MPIO enabled
» Windows Server hotfixes must be applied (the installation will inform you which hotfixes are
needed)
 What is installed:
» Nimble Setup Manager (NSM)
» Nimble Connection Manager & Connection Service (iSCSI)
» Nimble VSS Requestor/VSS Hardware Provider (iSCSI & Fibre Channel)
» Nimble Device-Specific Module (DSM) for MPIO
» Nimble Diagnostic Utility
» Microsoft Windows PowerShell module
» Nimble Hotfix Monitor Service

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-37


Nimble Connection Manager (NCM)

 Simplifies making and maintaining the optimal number of iSCSI connections


between host and array
 Requires that the “Automatic iSCSI Connection Method” be used
» Found in the Network and Configuration Group settings
 NCM performs the following tasks:
» Gathers interface, subnet, and volume information
» Calculates the optimal number of connections
» Determines which host network interfaces are in the same subnets as the array network
interfaces
» Attempts to make the optimal number of connections
» Displays the connections and all necessary information so users can easily verify connections
are optimal

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-38


Nimble Connection Manager for VMware

 Install onto each ESX host


 Prerequisites
» ESXi 5.x or 6.x with Enterprise or Enterprise Plus
» vCenter Server 5.x or 6.x
» Root access to ESX host(s)
» 15 minute window when ESXi host can be offline
 Install using vSphere Update Manager or ESXCLI
• Use the correct package for your version of ESXi (NCM for ESX 5.x or NCM for 6.x)

 NCM for vSphere 6.0 or higher required to support VVols on iSCSI arrays
 What is installed:
» Nimble Connection Service (NCS)
» Nimble Path Selection Plugin (PSP)

Note A fresh installation of NCM on ESXi 5.x does not require a


reboot. However, new installations of NCM on ESXi 6.0 do
require a reboot.
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-39
Nimble Connection Manager for Linux

 Linux NCM
» RHEL OS versions 6.5, 6.7, 7.0, 7.1
» Ensure that connection redundancy is always maintained to the Nimble array.
» Manage multipath connections (at the I/O region level) to volumes striped across multiple
arrays.
» Configure block device level settings for optimal performance.
» Automatically manage iSCSI and multipath configuration.

 Prerequisites
» sg3_utils and sg3_utils-libs
» device-mapper-multipath
» iscsi-initiator-utils (for iSCSI deployments)

 Linux NCM is bundled into a single executable binary called ncm_installer

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-40


LAB 2

Tasks:
 Build a volume
» Configure an initiator and an initiator group
» Create a volume collection
» Configure protection schedules

Time to complete: 15 minutes

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-52


LAB 3 (Optional)

Tasks:
 Connect to a host
» Launch NCM
» Configure the Windows host via NCM
» Connect and examine the new volume
» Prepare and mount the volume on a Windows host

Time to complete: 15 minutes – this lab is optional.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 7-53


Module 08:

Introduction to Nimble Storage SmartSnap

Course Revision 5
Module 08 Objectives

At the end of this module, you will be able to:


 Describe how Nimble snapshots work
 Understand snapshot scheduling
» Need to add RPO RTO definitions
» Define Change rate
 Understand recovering from a snapshot
» Clones Zero Copy Clones

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-2


Understanding Snapshots
Snapshots

What is a Snapshot?

Snapped Data

New Data (non-snapped)

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-4


COW Snapshots

Changed Block

Snapped Data

New Data (non-snapped)

Snapshot Reserve

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-5


ROW Snapshots

Changed Block

Snapped Data

New Data +
Changed Blocks

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-6


File and Snapshot Lifecycle 09:00

filename A B C D State of data at 09:00 filename 4-block file created


A B C D

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-7


File and Snapshot Lifecycle 10:00

filename A B C D State of data at 09:00 filename 4-block file created


Snap10 A B C D 10:00 snapshot A B C D

10 snap!

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-8


File and Snapshot Lifecycle 10:20

filename A B’ C D State of data at 10:20 filename 4-block file created


Snap10 A B C D 10:00 snapshot A B C D B’

10 snap!
If block B is changed, the original state
can be recovered by rolling back to the
snap taken at 10:00

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-9


File and Snapshot Lifecycle 11

filename A B’ C D State of data at 11:00 filename 4-block file created


Snap10 A B C D 10:00 snapshot A B C D B’
Snap11 A B’ C D 11:00 snapshot

10 11
10 snap!
The next snap taken captures the
change made to block B

Any snapshots can be used to recover from without the loss of snapshots taken before or
after the snapshot being recovered from.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-10


Basic Terms
Things to consider and DR Terms

RPO – Recovery Point Objective


• defined by business continuity planning. It is the maximum targeted period in which
data might be lost from an IT service due to a major incident. The RPO gives systems
designers a limit to work to.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-12


Things to consider and DR Terms

RTO – Recovery Time Objective


• is the targeted duration of time and a service level within which a business process
must be restored after a disaster (or disruption) in order to avoid unacceptable
consequences associated with a break in business continuity.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-13


Things to consider and DR Terms

Synchronous Replication –
• the process of copying data over a storage area network (SAN), local area network
(LAN) or wide area network (WAN) so there are multiple up-to-date copies of the data.
RPO = Zero

Zero Data Loss

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-14


Things to consider and DR Terms

Asynchronous Replication –
• The write is considered complete as soon as local storage acknowledges it. Remote
storage is updated, but with some lag. Asynchronous Replication is a schedule based
event. Lost Data ≠ 0

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-15


Things to consider and DR Terms

Change Rate –
• Is the amount of data that is changed/modified in a given period of time
• Note: the higher the change rate, the more bandwidth you may need to ensure
RPO/RTO can be met.

Changing Data

A A’ A’’ A’’’

Time

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-16


Understand Snapshot Scheduling
Volume Collections and Schedules

 Snapshot Schedules
» Determined by Volume Collections which
include the following parameters:
• Schedule name
• How often the snapshot should be taken
• Timing of the snapshot
• Which days to run the snapshot on
• Number of snapshots to retain

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-18


Taking a manual snapshot

1. Manage >> Volumes >> [Select desired volume]


2. Click “Take Snapshot”

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-19


Taking a manual snapshot

3. Enter a name for the snapshot and select desired Status and Writability settings.
4. Click OK to immediately take a snapshot

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-20


Taking a manual snapshot

5. Click the Snapshot tab to view the snapshot.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-21


Snapshot Status

Manage >> Protection >> Volume Collections >> [Select a volume collection]

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-22


“New Data” is not Snapshot Capacity

This is not the capacity being retained by a snapshot


This is the “new data” since the prior snapshot.
There is only one place to see snapshot capacity
- It is for ALL snapshots of the volume, not broken out by individual snapshot

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-23


Understand Recovering from a Snapshot
Recovery Scenarios

 Recovery from local snapshots


» Single volume, volume collection
» Replacing entire volume

 Testing my DR site without interrupting replication


» Use of clones

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-28


Zero Copy Clone

 Snapshots are ROW


» Snapped data is held as a single dataset
» New writes are directed to available space in the storage pool
 Zero copy clone
» Allows a volume to be created for online use based on a snapshot
» Any changed data is handled like a ROW snapshot
» Occupies no additional space until new data is written or changed

Pointers

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-29


Recovery Scenarios – Recovery from local snapshots

Clone the snapshot (creates a first-class volume)

1 3

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-30


LAB 4

Tasks:
 Recover from a Snapshot
» Simulate a data loss event
» Create a zero-copy clone
» Connect to the clone and recover data
» Disconnect and delete the clone

Time to complete: 15 minutes

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 8-31


Module 09:
Introduction to Nimble Storage
SmartReplicate

Course Revision 5
Module 09 Objectives

At the end of this module, you will be able to:


 Describe basic replication concepts
 Understand SmartReplicate components
 Explain how replication works
 Configure Replication
 Understand the replication considerations
 Perform SmartReplicate Disaster Recovery

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-2


Replication Introduction
Enterprise Level Data Protection, Efficiency & Security

 Options, flexibility & effortless management with all-inclusive packaging

SmartReplicate:
• Efficient (thin, block diffs+ data
reduction)
• WAN optimized
• Secure (AES-256bit encryption)
• No license required

CS700

DR

AF9000
Production

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-4


Data Protection & Copy Data Management
Others Nimble Storage
D2D + Replication Snapshots + Replication
Tier 1 Tier 2 Fast, cost-effective hybrid storage

Primary 9:00
9:15
9:30
9:45

 No backup window
 Rapid local recovery
Backup  Cost-effective, simple DR
Tier 3
(+Dedupe)

9:00
Disaster Tier 3 10:00
Recovery (+Dedupe) 11:00
12:00

 Instantaneous zero-
Space 9:00 copy clones (e.g., for
efficient 10:00 test and dev instances)
11:00
clones 12:00

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-5


Integrated Data Protection Efficiency

Advanced flash endurance


management

Application consistent backups Backup, DR and archival


• Protect more frequently • DR
1/2 thedown
costtoand
5 minfootprint
RPO of eMLC flash
• Recover rapidly from online backups • Retain backups for months cost-effectively
• Cost-optimized Test/Dev with cloning

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-6


Replication Use Cases

Primary to DR Many to 1 (ROBO) Reciprocal


9:00 AM
10:00 AM
11:00 AM
9:00 AM 9:00 AM
12:00 PM
9:00 AM 9:00 AM 10:00 AM 10:00 AM
9:00 AM
10:00 AM 10:00 AM 11:00 AM 11:00 AM
10:00 AM
11:00 AM 11:00 AM 12:00 PM 12:00 PM
11:00 AM
12:00 PM 12:00 PM
12:00 PM

9:00 AM 9:00 AM 9:00 AM


10:00 AM 9:00 AM 9:00 AM 10:00 AM 10:00 AM
11:00 AM 10:00 AM 10:00 AM 11:00 AM 11:00 AM
12:00 PM 11:00 AM 11:00 AM 12:00 PM 12:00 PM
12:00 PM 9:00 AM 12:00 PM
10:00 AM
11:00 AM
12:00 PM

Flexible, efficient WAN-optimized replication facilitates rapid recovery


at another site in the event of a disaster
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-7
Replication Components
Software Components

 Partner:
» Identifies a Nimble array that will replicate to and/or from

 Snapshot Schedule:
» Attribute of a volume collection
» Details when to snapshot and replicate and to which partner (one or more of
these per volume collection)

 Throttle:
» Provides the ability to limit replication transmit bandwidth

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-9


Partner

 Identifies a Nimble array that can replicate to and/or from this array
 Must be created on upstream and downstream arrays
 Attributes:
» Name: must match group name
» Hostname: must match array’s management IP address
» Secret: shared secret between partners
 Connected: successfully established communications
» Management process re-affirms 1/minute
» Test function performs this on demand
 Synchronized: successfully replicated configuration, updated as needed and every
4 hours

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-10


Partner (cont’d)

 Pause/Resume:
» Terminate all in-progress replications inbound or outbound, to/from this
partner do not allow new ones to start until Resume
» Persists across restarts
 Test (button in GUI):
» Perform basic connectivity test
• Management process Controller A to B and B to A
• Data transfer process Controller A to B and B to A
 Throttles:
» Limit transmit bandwidth to this partner
» Scheduling parameters include days, at time, until time
» Existence is mutually exclusive with array throttles (a system can contain
array-wide throttles or partner-wide throttles, but not both)
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-11
Replication Partner Notes

• Replication happens on
Management IP
(default)
• Custom option available
IP/Subnet e.g. 10G interface

• You can have many


replication partners

• You can pause


replication by partner
but NOT by Volume
Collection or schedule

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-12


Replication Schedules

• Replication configured using


Volume Collection schedule
attributes

• Different Schedules in the


same Collection must
replicate to the same partner

• Calculate your change rate


and bandwidth – can you get it
all done??!!!

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-13


Replication QOS – Bandwidth Limit

• Support Multiple QOS


Policies

• Applies to Partner

• Can define a Global QOS


for all partners – Under
Manage Replication
Partner

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-14


How Replication Works
Introduction

Replication creates copies of volumes on a separate Nimble array


» Primarily for the purpose of off-site backup and disaster recovery

 Transfers
» Asynchronous / triggered by snapshots
» Transfers compressed snapshot deltas

 Controlled by two processes:


» Management (scheduling)
» Data transfer
 Topologies supported: 1:1, N:1, bi-directional (reciprocal)
 Replica volume can be brought online instantaneously

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-16


How Replication Works – The Basics

1. Create a replication partnership


2. Define replication schedule
3. At first replication the entire volume is
Network

copied to the replica partner


Replica Snapshot 4. Subsequent replicas contain only
changes that have occurred

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-17


Setup Replication Partner

1. Navigate to: Manage >> Protection >> Replication Partners >> New Replication Partner

2. Enter the partner arrays name.


» Review the notes included
3. Type in a description (Optional)
4. Enter the Hostname or IP address
5. Enter a shared secret

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-18


Replication Partnership over Data Subnet

 Manage -> Protection -> Replication Partners -> New Replication Partner

 Using Data Networks


» Can now specify replication over data
networks
» Data networks must be reachable/routable
» Initial connection will still go through
management network

 Pools
» Destination replicas will be created by
default in the specified pool

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-19


Volume Collection Schedules

Snapshots and replication are configured via Volume Collection Schedules

 Groups related volumes into a set that is snapshotted and replicated as a unit
 Contains one or more Snapshot Schedules that specify:
» When to take snapshots
» To/from replication partner
» Which snapshots to replicate
» How many snapshots to retain locally
» How many snapshots to retain on the replica
» Alert threshold
 Created on upstream array, automatically replicated to downstream

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-20


Volume Collection (cont’d)

 Replicated as configuration data along with all snapshot schedules that define a
downstream partner
» Sent to downstream partner as changes are made (transformed on
downstream, i.e. “Replicate To”  “Replicate From”
» Volumes created in offline state downstream as needed
» Clones created downstream only if parent snapshot exists

 Partner considered “synchronized” only if all relevant configuration is successfully


replicated (volcolls, schedules, volume creation)

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-21


Viewing Replication Status
Replication Status

 Replication status:
» Completed: Replication to partner is completed.
» Pending: Replication to partner not yet started (pending completion of prior snapcoll)
» In-progress: Replication in progress and status shows amount of progress
» N/A: Upstream: non-replicable, Downstream: always shows this status

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-23


Replication Considerations
Snapshot Collection

 Creation of replicable snapcoll triggers its replication to start


» I.E. Replicate every # of snapshots
• Counter starts at creation of the schedule and does not reset

 Must replicate in the order they are created


 Replication deferred if volume collection busy replicating prior snapcoll
 Replication will not proceed unless partner is synchronized
 Replicable snapcoll cannot be removed by user unless replication to the partner is
paused

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-26


Replication & Deduplication Notes

 Replication works between any combo of dedupe & non-dedupe

 Data is rehydrated on  Data is deduplicated (if


transmission from source applicable) on the destination

Replication

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-27


Let’s talk about Keys

 There are two types of Encryption Keys


» A Volume Key
• When Encryption is turned on and there is NO Dedupe
» A Domain Key
• When Encryption is turned on and there is (YES) Dedupe

NO Dedupe YES Dedupe


Uses Volume Key Uses Domain Key

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-28


Replicating between 3.x Arrays using Dedupe and Encryption

 No encryption and enabled deduplication on both ends – no changes to


workflow for replication
Concurrent deduplication and
encryption on the same volume is
unsupported on software
releases prior to 3.4.0.0.

No Encryption
No Dedupe B
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-29
Replicating between 3.x Arrays using Dedupe and Encryption

 No encryption and deduplication enabled on both sides – no changes to


workflow for replication

No Encryption
Yes Dedupe B
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-30
Replicating between 3.x Arrays using Dedupe and Encryption

 Encryption on and no deduplication on both sides – no changes to workflow for


replication

Yes Encryption
No Dedupe B
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-31
Replicating caveats

 Encryption enable on both ends and deduplication on upstream only – data is


un-deduped and sent encrypted with a volume key to the downstream array.
Data is stored encrypted on the destination array.

Will un-dedupe & change


Domain encryption to
Volume Encryption

Yes Encryption Yes Encryption Volume Encryption


Yes Dedupe No Dedupe No Dedupe

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-32


Replicating caveats

 Encryption enabled on both ends and deduplication in only on the downstream


array - the data is sent over the wire encrypted using the upstream key, and
then on downstream array data is unencrypted and then re-encrypted using
the downstream domain key.

Encrypted
Yes Encryption
keeping Yes
Re-encrypted
Encryptionusing
Upstream
No Dedupe
domain key Downstream
Yes Dedupe
domain key

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-33


Replicating between 3.x to 2.x Arrays

 If the destination is not trusted (NO Dedupe Not using Encryption but supports encryption).
» In this case, the data is decrypted using upstream domain key, un-deduped and re-encrypted using the
upstream volume key, the data is then stored downstream as is

Re-Encrypt
Will decrypt
Un-dedupe
before sending

Stored as is
No Encryption
Yes Encryption No Dedupe
No Dedupe
Yes Dedupe Yes Encryption with Upstream
Volume Encryption key
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-34
SmartReplicate Disaster Recovery
General DR Operations

 Handover
» Graceful transition between two site (no data loss)
» Examples:
• Handover to DR site for non-DR situations
• Handover from DR site when recovering from a DR situation

 Promote
» Making the DR site primary with the data it has (possible data loss)
» Example:
• Promote a DR site during a disaster

 Demote
» Clearing ownership status from a former source
» Example:
• Production system comes back up after promotion to DR site.

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-36


Volume Ownership

 Schedules and volumes have a notion of “ownership”


» On downstream array, replicated objects are owned by upstream array and cannot be directly
modified

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-37


Replication Concepts - Promote

Upstream Array Downstream


Temp Upstream Array Array)
(Orig. Downstream

10AM Snap Snap 10AM

11AM Snap Snap 11AM


Promote
11:30AM

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-38


Replication Concepts - Promote
Promote
Upstream Array Temp Upstream (Orig. Downstream Array)
When you promote a downstream replication partner,
the system:
1. Suspends the replication relationship associated with the volume
collection.
2. Give ownership of volumes to the downstream array.
10AM Snap
3. Creates a second (local) instance of the volume collection and Snap 10AM
assumes ownership.
4. Clears “Replicate From”
11AM Snap Snap 11AM
5. Brings the most recently replicated snapshots online as volumes.
The contents of the newly available volumes are then consistent Promote
11:30AM
with the last replicated snapshots.
6. Begin taking snapshots per defined schedules

Only use promote if the upstream array is no longer


available.
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-39
Replication Concepts - Promote

Upstream(Orig.
Temp Downstream Array Upstream Array) Temp Upstream (Orig. Downstream Array)

10AM Snap Snap 10AM

11AM Snap Snap 11AM


Promote
11:30AM
Reconfigure role - downstream
Snap 12PM

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-40


Replication Conceptsm – Promote
Handover
Temp Downstream (Orig. Upstream Array) Temp Upstream (Orig. Downstream Array)
When you Perform a handover the system will:
1. Take all associated volumes offline
2. Takes a snapshot of all associated volumes
3. Replicates these snapshots to a downstream replication
partner
10AM Snap Snap 10AM
4. Transfers ownership of the volume collection to the
partner
5. BringsSnap
11AM the newly replicated volumes online Snap 11AM
6. Reverses replication roles/direction Promote
11:30AM

Snap 12PM

Handover
© 2014 Nimble
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-41
Replication Concepts - Handover

Upstream(Orig.
Temp Downstream Array Upstream Array) Downstream
Temp Upstream Array Array)
(Orig. Downstream

Reverse roles

10AM Snap Snap 10AM

11AM Snap Snap 11AM


Promote
11:30AM Snap
Snap

Automatic Snap Snap 12PM


Taken Before
Restore Handover Snap
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-42
Demote

 Demote:
»Offlines volumes
»Relinquishes ownership of volcoll objects
»Stop taking local snapshots
 Example:
»After an outage and promote away from an Upstream system;
Upstream System Comes back on-line:
• Prepares a system to become new downstream partner (to manually re-
establish reverse replication)
• or Prepares a system for Fail-Back (handover to the original Upstream
or Production System)
• Looks for common snapshot as a starting point to replicate from
© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-43
Recovery Scenarios – Testing at DR site (still replicating)

1. Go to downstream replica
2. Clone the snapshot (create a first class volume)
3. Add/adjust ACLs on the volume
4. Mount the volume
5. Interrogate/Test the data and applications (via Windows,
ESX, etc.)
6. Unmount the volume
7. Delete the cloned volume

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-45


Recovery Scenarios – Full Disaster Recovery

 Failover to DR site
1. Promote downstream volume collections at DR site
2. Add/adjust ACLs on the volumes
3. Mount volumes to application servers (Windows/ESX)
4. Start production environment at DR site

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-46


Recovery Scenarios – Failback after Disaster

 Failback to Primary site


1. Install new array and configure as downstream partner
2. Allow replication of volumes while still running at DR site
3. Gracefully shutdown apps at DR site
4. Perform Handover to primary site
5. Start production environment at primary site

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-47


LAB 5

Tasks:
 Setup partner replication
» Configure the upstream array
» Configure the downstream array
» Test the connection stats

Time to complete: 15 minutes

© 2016 NIMBLE STORAGE | CONFIDENTIAL: DO NOT DISTRIBUTE Course Revision 5 9-48