Netwell

.
, ,
.
Netwell NetApp.

NETAPP TECHNICAL REPORT

NetApp
VMware vSphere
Vaughn Stewart, Larry Touchette, Mike Slisinger, Peter Learmonth,
Sitakanta Chaudhury, Julian Cates, NetApp
2010 | TR-3749 | Version 3.0

:
NetApp
NetApp
VMware vSphere 4.x 5.0.
Ethernet (iSCSI, NFS, FCoE),
, ,
NetApp vCenter NetApp Virtual
Storage Console 2.x.


1 .................................................................................................................................................. 7
1.1 ............................................................................................................................ 7
1.2 .................................................................................................................................. 7
1.3 ........................................................................................................................... 8
1.4 ........................................................................................................................ 8
2 VMware ................................................................................................... 8
2.1 .......................................................... 8
2.2 ................................................................. 9
2.3 '80/20' ................................................................................................................................. 9
......................................................................................... 9
( ) ............................................... 9
2.4 VMFS..............................................................................................................................10
Spanned VMFS............................................................................................................11
VMFS LUN NetApp.......................................................................................................11
2.5 NFS .................................................................................................................................12
NFS NetApp .................................................................................................................12
2.6 Raw Device Mapping ........................................................................................................................13
RDM LUN NetApp ...........................................................................................................................13
2.7 ......................................................................14
2.8 VMware ........................................................................................15
Thick ...........................................................................................................15
Thin ............................................................................................................16
Eager Zeroed Thick ....................................................................................16
2.9 ..........................................................................17
2.10 Thin Provisioning .................................................................................................18
Thin Provisioning NetApp .....................................................................................................18
2.11 .............................................................................18
VMFS RDM LUN ........................................................................20
NFS.................................................................20
VMware .........................................................................................21
2.12 vStorage Array Integration VMware vSphere .............................................................................21
vStorage APIs for Array Integration (VAAI) .........................................................................................21
Storage I/O Control (SIOC) ..................................................................................................................22
....................................................................23
2

3 .................................................................................................25
......................................................25
3.1 SAN NAS .................................................................................................25
3.2 Fibre Channel ............................................................................................26
........................................................................................................26
......................................................................................26
3.3 Ethernet.....................................................................27
VLAN Tagging 802.1Q...................................................................................................................27
Flow Control ........................................................................................................................................27
Spanning Tree ....................................................................................................................29
IP- .................................................................................................30
Jumbo Frames .................................................................................................................30
Ethernet .................................................32
NetApp Virtual Interfaces (EtherChannel)...........................................................................................32
3.4 Cisco Nexus 1000V VNetwork Distributed Switch............................................................................33
Cisco Nexus 1000V System VLAN ........................................................................................................33
Jumbo Frames .................................................................................................................33
3.5 IP-, ..............................34
3.6 Multiswitch Link Aggregation .................................................36
MSLA .........................................................................................................................36
.....................................................................................................................36
IP- NetApp................................................................37
VMkernel ESX/ESXi ............................................41
3.7 Ethernet- ...............................43
Single Mode.........................................................................................................................................43
Single Mode EtherChannel .............................................................43
Single Mode EtherChannel ...................................................................43
multimode .........................................................44
.....................................................................................44
VMkernel .............................................................................................................44
ESX iSCSI ......................................45
ESX NFS........................................46
............................................................................................................................46
IP- NetApp................................................................46
VMkernel ESX/ESXi ............................................50
3

3.8 Multiple TCP Sessions iSCSI ...............................................................51

iSCSI VMKernels Multiple TCP Sessions iSCSI...................52
4 ..................................................................................55
4.1 : ...............................................55
NetApp VCenter............................................................................................................57
4.2 ................................................................................................57
4.3 NetApp ...................................................................................57
RAID-DP .................................................................................................................58
Aggregates...........................................................................................................................................58
Flexible ..............................................................................................................................59
LUN ......................................................................................................................................................59
........................................................................................................................59
4.4 NetApp...........................................................................59
NetApp System Manager ....................................................................................................................59
Data ONTAP .........................................................60
............................................................................60
NetApp System Manager ......................................................61
NetApp Operations Manager ................................................62
TCP/IP, VSC ............................................................62
4.5 VSC 2.0 ...................................................................63
..........................................................................................63
4.6 Provisioning and Cloning VSC 2.0....................................................65
.................................................................................................................................66
5 vSphere....................................................................68
5.1 vCenter ...............................................................68
NetApp Virtual Storage Console 2.0 vCenter plug-in......................................................68
5.2 Virtual Storage Console 2.0 ............................................................................................70
VSC 2.0...............................................................................................................................70
5.3 Virtual Storage Console .....................................72
5.4 ESX/ESXi..............................................................................73
5.5 Provisioning and Cloning ...................................74
5.6 Provisioning and Cloning ........................................................76
5.7 vCenter ..................................................................................77
5.8 .........................................................80
.....................................................................................80
4

 VM NetApp Snapshots .......................................81

: VMkernel swap ............81
: VM swap/Pagefile ...............................83
5.9 vCenter ...................................................................................84
5.10 vCenter .............................................................85
NFS ............................................................................................................86
Capacity.................................................................................................................................86
Volume ..................................................................................................................................87
Deduplication ........................................................................................................................87
6 .....................................................88
6.1 VM OS Windows.......................................................88
Windows .....................88
......................................................................................................88
6.2 .............................................89
VM SCSI BUS ................................................................................................................89
6.3 .....................................................89
VM VMFS ..........................................................89
..................................................................................................................90
.............................................................................90
6.4 VM ...........................................................90
......................................................................................90
6.5 ...............................................................................................91
OS Windows ................................................................................91
NetApp MBRtools: .................................................91
6.6 VM .................................................................92
VM ...........................................................................................92
NetApp MBRtools ..............................92
...............................92
6.7 VM ......................................................93
VMDK VM DISKPART ..........................93
6.8 VM ...............................................................................93
(VMDK) ........................................................................................93
OS (NTFS EXT3) .................................................95
OS ...................................................................................95
7 .............................................................................................95
5

7.1 , ......................................................................95
7.2 NetApp ...........................................................96
8 vSphere 5............................................................................................................................99
8.1 VMware ESXi ....................................................................................................................................99
8.2 VMware vCenter Server ...................................................................................................................99
8.3 ..............................................................................................................100
8.4 Host Profiles ...................................................................................................................................101
9 vSphere 5 ............................................................................................................................101
9.1 ..................................................................................................101
9.2 VMFS 5............................................................................................................................................103
9.3 NFS vSphere 5 .......................................................................................................104
NFS Heap Memory ..............................................104
..................................................................................104
IP- NFS ............................................104
9.4 Storage Distributed Resource Scheduler (Storage DRS) ................................................................105
Datastore Cluster (PODs) ..................................................................................................................105
Storage DRS ..............................................107
.....................................................................................................108
(Affiity Rules) Maintenance Mode......................................109
SDRS Data ONTAP................109
NetApp Snapshot SDRS ..................................................................................................................110
NetApp SDRS ........................................................................................................110
NetApp Datastore Thin Provisioning SDRS.....................................................................................111
9.5 vSphere Storage I/O Control ..........................................................................................................111
10 vSphere 5 ....................................................................................................................112
10.1 vSphere Standard vSwitches ........................................................................................................112
MTU/Jumbo Frames ..........................................................................................................................112
Flow Control ......................................................................................................................................113
10.2 Multiple TCP Sessions iSCSI...........................................................114
11 ...............................................................................................................................................120
12 .............................................................................................121
...................................................................................................................................................121
NetApp ..................................................................................................................................................121
VMware .................................................................................................................................................122
13 ............................................................................................................................123
6

1

Virtual Infrastructure

1.1
NetApp
, .

, ,
, ,
,
.

NetApp
VMware vSphere. NetApp VMware
2001 . NetApp
Data ONTAP ESX/ESXi Server.
.
.

1.2
, , ,
. NetApp VMware ,
, , , ,
,
,
, . NetApp ,

( )
.

NetApp VMware vSphere. ,
,
NetApp, NetApp c vSphere.
: NetApp vSphere Data ONTAP
7.3.1.1 . Data ONTAP,

.
, NetApp professional services
, .
7


, .

1.3
, NetApp
Technical Library ,
, VMware Virtual Infrastructure.
, , VMware ESX/ESXi Server 4.0,
vCenter Server 4.0, NetApp Data ONTAP 7G.

, ,
.

1.4
, TR-3749, ,
NetApp, NetApp ,
VMware.
,
-, .
, TR-3880: CLI Configuration Processes
for NetApp and VMware vSphere Storage Arrays Running Data ONTAP and ESX/ESXi Server.

.

2 VMware

Virtual Infrastructure

2.1
VMware ESX
: VMware Virtual Machine File System (VMFS),
Network-attached Storage (NAS), , raw
device mappings (RDM). ,
VMware ,
VMware, High-Availability (HA), Distribution Resource
Scheduler (DRS), VMotion, Fault Tolerance (FT).
.
VMware
, .


.

2.2

, ,
.
, ,
(datastore). - ,
,
.
, , ,
.
VMware NetApp .
,
,
. SAN NAS,
, ,
.
, Fibre Channel (FC) Ethernet (NFS, iSCSI, FCoE),
NetApp ,
,
, .
, , .

2.3 '80/20'

80/20, 80%
. 20%
business-critical, , , ,
, .


.

.
VM
-, VM
.
,
().
( )

.

-.
9

, .
, ,
.

NFS,
, , SAN,
VM.
; ,
NFS / VMFS.
, ,
, ,
80/20, NetApp VMware
, ,
.

2.4 VMFS
VMware Virtual Machine File System (VMFS)
,
. VMFS LUN-, Fibre Channel,
iSCSI, Fibre Channel over Ethernet (FCoE). VMFS LUN-
ESX .

1) ESX VMFS FC, FCoE, iSCSI.

VMFS VMware
. (shared) ,
VMware ,
.
VMware vCenter Server.
10

, ,
, VMFS.
NetApp ,
,
VM.

, . ,
, -
,
- .
Spanned VMFS
VMware VMFS, LUN-
, spanned datastore. spanned datastore
2TB LUN,
- (I/O queues)
LUN. ,
LUN ( ), - .
NetApp, OS Data ONTAP
, NetApp
spanned VMFS datastores.

2) ESX, spanned VMFS.

VMFS LUN NetApp
NetApp VMFS
, thin provisioning ;
; , ;
11

 , Site Recovery Manager (SRM), SnapManager for

Virtual Infrastructure, Rapid Cloning Utility (RCU), Virtual Storage Console (VSC),
SANscreen VM Insight. LUN, ,
VMFS ,
.

2.5 NFS
vSphere NFS
ESX.
VMFS. NFS, NetApp,
, ,
.
3 . , ,
VMFS,
- (I/O queue), NetApp FAS.

3) ESX, NFS.
NFS NetApp
VMware NFS NetApp
, ,
,
, , , Fibre Channel.
VM
. NFS,
,
, ,
, . NFS,
VMware WAFL,
NetApp.
12

 VM, , VM
, thin provisioning ,
,
, .
NetApp , Site Recovery Manager (SRM),
SnapManager for Virtual Infrastructure (SMVI), Rapid Cloning Utility (RCU), Virtual Storage Console
(VSC).

2.6 Raw Device Mapping

ESX LUN-
, , , P2V clustering
. raw device
mapping (RDM)
Fibre Channel, iSCSI, Fibre Channel over Ethernet (FCoE). , ESX
VM . VMFS NFS,
RDM VM. .4
.
RDM , physical-tovirtual, , s, Microsoft Cluster Server (MSCS). RDM
LUN ,
-,
.

4) ESX, VM RDM LUN FC iSCSI.

RDM LUN NetApp
RDM : (physical)
(virtual). VMware, VMotion
HA DRS.
13

 SCSI, VM.
MSCS VMware.
NetApp RDM thin provisioning
, , ,
SnapDrive,
SnapManager, RDM LUN FlexClone.
:

, MSCS,
Most Recently Used (MRU). Round Robin (RR)
VMware RDM LUN MSCS.

2.7

.
. VMware
VMware ESX and ESXi Server Configuration Guide.
1) .

LUN-

LUN/ NAS
queue
depth LUN

FC/FCoE
VMFS RDM
256

iSCSI
VMFS RDM
256

NFS
NetApp WAFL
64

64TB

64TB

16TB 100TB*

2TB 512b

2TB 512b

16TB 100TB*

64

64

N/A

4 8Gb FC 10Gb
1 10Gb Ethernet
1 10Gb Ethernet
Ethernet
* 100TB 64-bit aggregates
NetApp.
2) VMware.

VMotion
Storage VMotion
VMware HA
DRS
VCB
MSCS VM
Fault Tolerance

Site Recovery Manager

Thin Provisioned VM
VMware Native Multipathing

FC/FCoE

, RDM
Shared LUN
, eager-zeroed
thick VMDK
Virtual Mode RDM**

iSCSI

GOS

, eager-zeroed
thick VMDK
Virtual Mode RDM**

NFS

, eager-zeroed
thick VMDK

*
N/A
14

Boot from SAN

HBA

* VM NetApp NFS
** NetApp SnapDrive for Windows virtual-mode RDM devices
3) NetApp.

Thin Provisioning

FC/FCoE

RDM

iSCSI

RDM

NFS

SANscreen VM Insight

SnapDrive OS

*
ESX Host Utilities/VSC 2.0

Backup Recovery VSC 2.0

Provisioning Cloning VSC 2.0

* NFS- VMDK SnapDrive

SnapDrive.
4) .

Replicated backup SRM

SnapMirror
SnapVault
VMDK

VMDK
NDMP

FC/FCoE

RDM
RDM
VCB

iSCSI

RDM
RDM
VCB

VCB,
Windows

VCB,
Windows

NFS

VM
VM
VCB, VIC File
Explorer
VCB,

VM

2.8 VMware
,
vSphere. , , , ,
.
Thick
,
VM. ,
. VMDK
. , , , ,
, , .
,
, .

15

 5) thick VMDK .
Thin
thick, ,
. ,
VMDK (chunks),
. VMFS 1MB 8MB (
), NFS 4KB.
VMFS , SCSI locks
, .
, , ,
.

6) thin VMDK .
thick, thin VMDK . ,

.
, ,
.
thick thin- , ,
- ,
,
, , thin,
.
Eager Zeroed Thick
thick,
; , thick thin, eager-zeroed thick
,
.
16

 7) eager-zeroed thick VMDK .

2.9
NetApp ,
, VMware thin-provisioning.
FAS data deduplication thin provisioning VMFS RDM LUN

FAS. NetApp, , ESX.
: , NetApp NFS, thin-provisioned
VMDK VMware.

8) , VM NetApp NFS thin provisioned

.
17

2.10 Thin Provisioning

, ,
, VMware .
, ,
,
.
100% ,
, oversubscribe
, , (
CPU, , , ).
thin provisioning.

; thin provisioning
. ,
thin-provision , ,
, VM
.
, ,
,
.
thin provisioning oversubscribing , VM

,
.
Thin Provisioning NetApp
thin provisioning NetApp thin
provisioning VMDK VMware, LUN-, VMFS,
, VMDK
( thick-, thin-). , LUN-,
RDM, s thin provisioned.
NetApp thin-provisioned LUN-,
FlexVol, thin provision , 2x LUN.
LUN , FlexVol (quota).
, LUN FlexVol aggregate.

2.11
VMware
(VM templates). VM template
VM (.vmx) (.vmdk),
, -,
.
,
, ,
18

.
VM. 9
vSphere.

9) .
NetApp FAS data deduplication.
NetApp FAS data deduplication,
VMware ,
. , ,

NetApp FAS , ,
.
,
VMware.
NetApp FAS , , CPU ESX.
. 10
vSphere.
,
,
. , NetApp
OS ,
.

19

 10) .
VMFS RDM LUN
LUN- .
LUN ,
LUN- . ,
,
LUN-, ,
LUN- .
, LUN-, NetApp LUN
thin provisioning. ,
, VMware
, LUN, LUN-
, , (thick-) thinprovisioned. NetApp Virtual Storage Console (VSC) VI
.
NetApp , thin-provisioned LUN-,
FlexVol, thin provision , 2x
LUN. LUN- , FlexVol
(quota). , LUN FlexVol
aggregate
NFS
LUN-, NFS,
VMware.
VI.
- .

20

 VMware
NetApp vCenter plug-ins, VMware ,
, .
5.

2.12 vStorage Array Integration VMware vSphere

vSphere 4.1 VMware
vSphere. : vStorage APIs for array
integration (VAAI), storage I/O control (SIOC),
.
vStorage APIs for Array Integration (VAAI)
VAAI , ,
. VAAI
, (CPU, , -,
) , , , .
VAAI VMFS.
NFS NetApp, ,
, ,
Virtual Storage Console FlexClone. VAAI
VMFS. VAAI :

Full copy: ESX/ESXi , VAAI

,
.

vMotion.
Block zeroing: , eager-zeroed
thick VMDK, .
VAAI
,
.
Hardware-assisted locking: VMFS
, ,
,
. VAAI ,
SCSI-2 reservations, .
,
,
VMFS, .

VAAI .
VAAI NetApp, Data ONTAP
8.0.1 ( . VAAI Data ONTAP ESX/ESXi. 2.0 Virtual Storage
Console , VAAI .

21

Storage I/O Control (SIOC)

Storage I/O Control, vSphere 4.1, quality of service
, ,
, CPU . SIOC ,
VM ,
VM, ,
IOPS, , . SIOC
VMFS, FC iSCSI.
SIOC ,
VM . VM Resources VM Edit
Settings. VMs
IOPS.

11) SIOC VM vSphere.

, SIOC ,
. ,
, VM
. ,
(latency). 30ms,
. low, normal, high; 500, 1000,
2000 resource shares, , *.
resource shares , ,
SIOC
. , SIOC , VM 1000 shares
, VM 500 shares.
, VM,
.
*

resource shares , .

22

SIOC vSphere, , virtual

machine.
VM IOPS.
maximum IOPS VM vSphere
VM IOPS, latency, ,
. , VM MB/,
IOPS, -
: , VM 8k 10MB/, maximum IOPS
VM 1280. : MB/sec I/O size = IOPS.
: 10240000 8000 = 1280.

, vSphere 4.1
, vCenter,
esxtop. VI3
vCenter
.
NetApp, , FC
NetApp, LUN ,
vCenter performance monitor .
LUN- ,
;
vSphere 4.0. ,
NFS ,
VI3 vSphere 4.0. , VM vSphere 4.0
. 12.

12) , vSphere 4.0.

, vSphere 4.1,
-
, FC
FibreChannel. : FC, FCoE,
23

iSCSI, NFS. VM
VM . . 13
, vSphere 4.1.

13) , vSphere 4.1.

NAS ,
NFS , VM
. . 14 , vSphere 4.1.

14) , vSphere 4.1.

5) ,
, vSphere.

ESX/ESXi

Datastore

(throughput)
(latency)

vCenter
ESXTOP*
FC, iSCSI, NFS 4.1
4.1
24

Storage Adapter

(throughput)
FC
4.1
4.0+
(latency)
Storage Path
(throughput)
FC
4.1
4.0+
(latency)
LUN
(throughput)
FC, iSCSI
4.0+
4.0+
(latency)
VM
Datastore
(throughput)
FC, iSCSI, NFS 4.1
4.1
(latency)
Virtual Disk
(throughput)
FC, iSCSI, NFS 4.1
4.1
(latency)
* esxtop ESXi vSphere CLI resxtop.

3

Virtual Infrastructure


. ,
Ethernet, .
Ethernet . -, Fibre Channel , Fibre
Channel Protocol (FCP). ,
. -,
Ethernet (
converged networks) Ethernet
, , .


:

3.1 SAN NAS

vSphere SAN NAS
multipathing. ESX/ESXi, NFS
, , iSCSI, FC, FCoE ,
multipathing (, iSCSI ).
Multipathing NFS
LUN masking ( iSCSI FC, FCoE) ,
VMware .
, RCU,
.

25

3.2 Fibre Channel

Fibre Channel
VMware. FC ,
, ESX 2.0. , FC
,
VMware Fibre Channel NetApp.

ESX, NetApp SAN
host bus adapter (HBA). FCoE
converged network adapter (CNA). HBA/CNA initiator (ESX/ESXi)
target (NetApp). ,
World Wide Name (WWN).
WWN, , LUN NetApp.
NetApp, VMware
ESX/ESXi .
VMware FC VMware Fibre Channel SAN Configuration Guide.

SAN,
-
(zoning). SAN zoning Fibre Channel
Fibre Channel.
(hard zoning) (soft zoning).
port zoning,
. LUN ,
. port zoning, ,
. , port zoning
.
WWN zoning, World
Wide Port Names (WWPNs) , .
WWPN zoning ,
.
NetApp VMware single initiator, multiple
storage target.
FC FCoE. , WWN IQN
ESX, Storage Adapters Configuration ESX/ESXi
vCenter Server WWN.

26

 15) WWPN IQN vCenter.

3.3 Ethernet
10GBE Data Center Ethernet (DCE)
NetApp Data ONTAP VMware ESX/ESXi 4, 10GbE. 10GbE
, ,
, blade-.
VMware compatibility guide.
VLAN Tagging 802.1Q
VLAN,
VLAN VLAN- VLAN
tagging.
, , , ,
blade-, VLAN- . channel
ESX .
VLAN-, IP- VLAN . NetApp ,
- VLAN, VMkernel IP-
VMotion VLAN.
VLAN- VLAN tagging , IP-
. NFS IP-, IP
storage VLAN. NetApp
iSCSI / VLAN.

, IP-. VLAN-
, ,
.
Flow Control
Flow control
, ,
. Flow control
ESX/ESXi, FAS, .
, 10GbE, NetApp flow control (send
off receive off), .
27

 , Gigabit Ethernet,
, NetApp flow control
, ESX NetApp,
send.
Flow control FAS
FilerView or NetApp System Manager ( . 16). , X1139A/X1140A Unified Target
Adapter flow control full.

16) NetApp System Manager > Edit Network Interface >Advanced settings > Flow Control.
flow control ESX ESXi ethtool
esxcfg-module, NIC ESX.
ethtool :
[root@esx8 ~]# ethtool -A vmnic2 autoneg off rx off tx off

:
[root@esx8 ~]# ethtool -a vmnic2
Pause parameters for vmnic2:
Autonegotiate: off
RX:
off
TX:
off

28

 autonegotiate RX TX.
, autonegotiate (flow
control), , .
/etc/rc.local .
flow control vSphere, esxcfg-module
Intel e1000, VMware KB 1013413.
NetApp flow control
send off receive off.
( 10G Ethernet)
, ESX FAS Desired ,
, send off receive on.

17) flow control.

Spanning Tree
Spanning Tree Protocol (STP) ,
. OSI STP OSI Layer-2. STP

, , ,
bridge loops, /
. bridge loops,
(flood).
ESX NetApp Ethernet, NetApp
, Ethernet , ,
RSTP edge ports Cisco - portfast.
Cisco portfast
802.1q VLAN trunking ESX NetApp,
spanning-tree portfast trunk.
edge port RSTP , edge
port .
Cisco portfast. ,
edge port
portfast.
29

 IP-
, , NetApp IP-
, .
.
Jumbo Frames
Ethernet, maximum transmission unit (MTU)
. MTU 1500 .
,
. jumbo frame, 9000 .
jumbo frames ,
ESX ,
.
:

FCoE , baby jumbo frame

(MTU size 2112 ), ,
FCoE.

jumbo frames
, ,
. ( ) jumbo frames (, ,
NetApp ESX), jumbo frames ( Ethernet).
, jumbo frames,
VLAN trunking, - . Ethernet
, , Ethernet header (18 ),
VLAN tag (4 , VLAN). , jumbo frames (
)
Ethernet ( 9000 ). ,
, VLAN, 9018
9022 ( VLAN tagging). - ,
( Ethernet)
.
, ,
9216 . ,
, Ethernet
, (18 ), VLAN tag (4 ),
.
jumbo frame
, ,
, .

30

 18) MTU jumbo frames.

ESX/ESXi vSwitch VMkernel,
jumbo frames.
jumbo frames vSwitch, MTU size,
:
1. ESX/ESXi ( vSphere CLI).
2. MTU size:
vicfg-vswitch -m <MTU> <vSwitch>

MTU vSwitch.
MTU MTU,
, vSwitch.
3. :
vicfg-vswitch -l

jumbo frames VMkernel, .

1. ESX/ESXi ( vSphere CLI).
2. MTU size :
esxcfg-vmknic a I <ip address> -n <netmask> -m <MTU> <port group name>

VMkernel jumbo frame.

3. :
esxcfg-vmknic l

, esxcfg- vicfg-
. , , ,
, ESX SSH ( )
vSphere CLI ( Windows Linux, appliance
VM: vSphere Management Assistant, vMA) ESX ESXi. vMA
esxcfg-* vicfg-* command. vMA
vSphere CLI :

vMA: www.vmware.com/appliances/directory/178973
vSphere CLI: www.vmware.com/support/developer/vcli/

MTU size FAS, (

16 Flow Control):
31

1.
2.
3.
4.

NetApp System Manager.

.
Edit.
, ,
jumbo frames.

Ethernet
NetApp IP-
IP- ,
VLAN. ESX vSwitch
NIC. ESX VMkernel vSwitch.
VMware HA ESX
(service console port).
NetApp
VMkernel . , default gateway
VMkernel storage network. NFS service console port
VMkernel ESX.
IP-c , VMkernel, vmkping.

NFS,

:
vmkping <NFS IP address>.
NetApp Virtual Interfaces (EtherChannel)
(VIF), NetApp EtherChannel,
.
, VIF . VIF-
, ,
, .
NetApp VIF :
LACP (IEEE 802.3ad). NetApp active-passive
VIF, single-mode VIF, . NetApp VIF
VIF-
VIF .
Multimode VIF EtherChannel. multimode VIF
, VIF, .
, ,
. , ,
MAC-
. , VIF

.
LACP VIF (IEEE 802.3ad) EtherChannel. LACP VIF
, multimode
VIF, .
32

LACP VIF
.
, , ,
, ,
EtherChannel. LACP VIF ,
static multimode VIF,
. standard multimode VIF
(link loss).
Multimode LACP EtherChannels, IP. IP- , MAC,
TCP/UDP*.
NetApp IP-
, , , ,
IP- .
single-mode VIF .
,
, standby. single-mode VIF
, , VIF,
. , single-mode VIF.

3.4 Cisco Nexus 1000V VNetwork Distributed Switch

Cisco Nexus 1000V System VLAN
1000V vNetwork Distributed Switch (vDS), ,
, Virtual Supervisor Module (VSM) Virtual Ethernet Module (VEM).
VSM , 1000V. VSM ,
- , vDS. VSM-
active-passive. appliance
ESX.
DRS.
VEM ESX/ESXi, VSM.
, Cisco Nexus 1000V vDS.
, VSM , VSM
VM, VSM . ,
NetApp Cisco , service console VMkernel (vswif
vmknic) VLAN (System VLAN). VLAN
, .
VM VLAN
Jumbo Frames
Cisco Nexus 1000V vDS jumbo frames,
jumbo frames VMkernel. jumbo frames
*

VIF TR-3802

33

 VMkernel, vSwitch VMkernel,

jumbo frames, 1000V vDS.
jumbo frames vSwitch, :
1.
2.
3.
4.

vSwitch VMkernel jumbo frame ( 3.3).

ESX/ESXi vCenter client.
VMkernel vSwitch.
vSwitch 1000V vDS (. . 19).

19) vSwitch 1000V.

3.5 IP-,

, ,
IP-.
Ethernet multiswitch link aggregation (MSLA).
, Cisco Nexus (virtual port channels),
Cisco Catalyst 3750 series (cross-stack EtherChannel), Cisco Catalyst 6500 series VSS
1440 (multichassis EtherChannel), .
3.6 .

34

 20) multiswitch link aggregation.

, multiswitch
link aggregation,
ESX/ESXi, Data ONTAP, .

21) IP-.
3.7 .
35

3.6 Multiswitch Link Aggregation

IP- multiswitch link aggregation.

; , multimode
LACP VIF IP.
,
,
,
.
MSLA
.
NIC .
.
.
EtherChannel.
VMkernel IP ,
.

22) Multimode VIF multiswitch EtherChannel.

IP-
, EtherChannel IP-
.
.
Rapid Cloning Utility (RCU).
23
ESX .

36

 23) multiswitch EtherChannel.

IP- NetApp
, IP, NFS,
iSCSI, Ethernet
EtherChannel. EtherChannel
.
, ESX/ESXI.
NetApp EtherChannel, 802.3ad LACP, / static
EtherChannel. Data ONTAP EtherChannel NetApp virtual interfaces
(VIF). NetApp EtherChannel VIF LACP VIF,
.
multiswitch link aggregation, multilink
EtherChannel VIF, IP-. IP-
vmnics, -
ESX/ESXi. NSM.

37

 24) LACP EtherChannel NIC .

: EtherChannel
;
NIC. ,
EtherChannel. ,
EtherChannel,
,
EtherChannel, .

38

 25) LACP EtherChannel NIC

.

39

 26) LACP EtherChannel

. , EtherChannel
.

40

 27) LACP EtherChannel. ,

EtherChannel,
. IP- EtherChannel.
VMkernel ESX/ESXi
, IP- , multiswitch
EtherChannel trunking virtual port channeling, ESX
,
. VMkernel IP-.
, ,
,
IP- (target IP).
ESX, . 28, vSwitch (
vSwitch1) IP- .
vSwitch ( vmnic1 vmnic2).
,
cross-stack EtherChannel trunk.

41

: VMware LACP, IEEE 802.3ad,

Ethernet.

28) NIC ESX multiswitch EtherChannel.

vSwitch1 VMkernel (VMkernel 1) IP-,
(teaming) NIC VMkernel :

VMkernel 1: IP- 192.168.1.101.

VMkernel 1 : Load-balancing Route based on IP hash.

29) VMkernel ESX multiswitch EtherChannel.

42

3.7 Ethernet
IP multiswitch link aggregation,

.
,
.
IP- ,
,
.
Single Mode
, single-mode ,
single-mode (active-passive) EtherChannel VIF. VIF
IP-, ,
IP- . vif favor ,
VIF
. ,
- .
Single Mode EtherChannel
(
).
.

. non-EtherChannel.

ESX/ESXi multiswitch hop.
Single Mode EtherChannel
VMkernel IP storage ports.
- IP .

43

 30) Single-mode VIF .

multimode
, ,
TR-3880: CLI Configuration Processes
for NetApp and VMware vSphere Storage Arrays Running Data ONTAP and ESX/ESXi Server.

IP-
, EtherChannel IP-
, VMkernel -
ESX/ESXi. ,
. Rapid
Cloning Utility (RCU), .
VMkernel
VMkernel NetApp
,
, . NetApp
VMkernel .
iSCSI NFS. VMkernel
IP .
iSCSI NFS ,
VMkernel .
31. VMkernel vSwitch,
vmnics vSwitch.
NFS, VMkernel vmnic,
vmnic standby. , vmnic
VMkernel.

44

 31) VMkernel iSCSI NFS.

32
ESX .

32) EtherChannel.
ESX iSCSI
ESX ( NIC),
, , .
multipathing VMware,
VMkernel.
, .
45

 ESX NFS
ESX (- NIC),
, , ,
, .
. ,
. EtherChannel .

, ,
, ,
, NetApp. ,
.

33) ESX vSwitch1.

34) ESX vSwitch .

IP- NetApp
, IP, NFS,
iSCSI, Ethernet
EtherChannel. EtherChannel
.
, ESX/ESXI.
NetApp EtherChannel, 802.3ad LACP, / static
EtherChannel. Data ONTAP EtherChannel NetApp virtual interfaces
(VIF). NetApp EtherChannel VIFs single mode
,
.
46

 , single-mode
EtherChannel VIF, IP-.
NSM.

35) LACP EtherChannel NIC .

: EtherChannel
;
NIC. ,
EtherChannel. ,
EtherChannel,
,
EtherChannel, .

47

 36) single-mode EtherChannel

NIC .

48

 37) single-mode EtherChannel

. , EtherChannel
.

49

 38) single-mode EtherChannel.

EtherChannel,
.
VMkernel ESX/ESXi
, ESX/ESXi
, , VMkernel,
. ,

IP-. VMkernel, (VMkernel)
,
vmnic vSwitch. ,
VMkernel .
IP- , ,
vmnic.
ESX/ESXi . 39, vSwitch ( vSwitch1)
IP- .
vSwitch ( vmnic1 vmnic2).
.

39) NIC ESX Ethernet.

vSwitch1, VMkernel (VMkernel 1 VMkernel 2). VMkernel
IP- , NIC teaming
VMkernel , :

VMkernel 1: IP- 192.168.1.101.

VMkernel 1:
Enable the override vSwitch failover order option.
NFS iSCSI, vmnic1.
NFS, standby- vmnic2.
iSCSI, unused.
VMkernel 2: IP- 192.168.2.101.
VMkernel2:
Enable the override vSwitch failover order option.
50

NFS iSCSI, vmnic2.

NFS, standby- vmnic1.
iSCSI, unused.

40) VMkernel ESX Ethernet.

3.8 Multiple TCP Sessions iSCSI

vSphere multiple TCP sessions iSCSI.
( round robin)
VMware native multipathing VMkernel vmnic
iSCSI.
iSCSI , VMkernel
vmnic. standby vmnics VMkernel .
VMkernel iSCSI, , ,
VMkernel NIC teaming, , ,
. iSCSI
native multipathing ESX/ESXi.
iSCSI , FC. multiple
TCP session support iSCSI ESX/ESXi, NFS, ,
, , NFS
iSCSI VMkernel, .

51

NetApp , iSCSI, NFS,

TCP NIC
teaming, .

41) VMkernel iSCSI multiple TCP

sessions.
: ESX/ESXi 4 iSCSI .
iSCSI VMKernels Multiple TCP Sessions iSCSI
1. vCenter Server.
2. ESX.
3. Configuration.
4. Hardware, Networking.
5. , Add Networking Add Network.
6. VMkernel Next.
: VMkernel Ethernet,
iSCSI. VMkernels .
7. VMkernel, .
default gateway VMkernel IP storage network.
8. VMkernel
, iSCSI VMkernel. VMkernel
standby-. . . 42 43.
iSCSI daemon VMkernel.
CLI.

52

9. ESX ESXi :
esxcli swiscsi nic add n <VMkernel ID> -d <Virtual HBA ID>

:
esxcli swiscsi nic add -n vmk0 -d vmhba33
esxcli swiscsi nic add -n vmk1 -d vmhba33
10. iSCSI-to-VMkernel. ESX ESXi

:
esxcli swiscsi nic list -d <Virtual HBA ID>

:
esxcli swiscsi nic list -d vmhba33

. 44.

42) iSCSI VMkernel 0: vmnic0.

53

 43) iSCSI VMkernel 1: vmnic1.

54

 44) (binding) iSCSI VMkernel multiple TCP sessions.

4

Virtual Infrastructure

4.1 :
NetApp, ,
,

55

 45) NetApp vCenter plug-in.

,
, .
,
(aggregates, FlexVol )
.
,
, VMware
, , LUN masking,
, ,
.
, ,
,
, VMware
.
, ,
VMware, NetApp, ,

.

56

 NetApp VCenter
NetApp vCenter, Virtual Storage Console (VSC) Rapid Cloning Utility (RCU),
VSC 2.0.
, VSC 2.0 NetApp SnapManager for Virtual
Infrastructure (SMVI), vSphere.
VSC ESX/ESXi .
, VSC 2.0
, VMware.
NetApp
, , ,
.

4.2
,
, , :

- .
, Ethernet- (NFS,
iSCSI, FCoE). () (VLAN),
.
, IP (NFS iSCSI),
IP .
.
IP (NFS iSCSI)
(EtherChannel), Ethernet . NetApp
VIF ( ). NetApp ,
, LACP VIF- multimode.

4.3 NetApp

.
,
. , VMware
, .
VM, VMware HA
DRS, VMware Fault
Tolerance, VM, ,
ESX VMotion storage VMotion, .
,
,
HBA,
, .
, ,
.

57

 RAID-DP
,
, , .
,
. NetApp RAID-DP RAID,
NetApp FAS. RAID-DP
RAID-. RAID-DP
; RAID- 12.5%.

RAID-DP , RAID 5 ,
RAID 10. NetApp RAID-DP RAID,
VMware.

46) NetApp RAID-DP.

Aggregates
aggregate NetApp,
flexible volume . Aggregate
,
IOPS, aggregate, IOPS
.
.
NetApp
root aggregate. , NetApp
aggregate. root aggregate ,
RAID-DP. ,
RAID type RAID-DP RAID-4. , ,
.
root volume root aggregate,
root volume aggregate.

aggregates. - VMware
(random) ,
,
-. FAS,
aggregate,
58

 .
aggregate.
Flexible
flexible LUN- ,
VMware ESX.
NetApp
VMware flexible- NetApp.
NetApp VMware desktop pools flexible NetApp.
VMware,
.

SnapMirror , NetApp
flexible volume.
LUN
LUN- SCSI- (FC, iSCSI, FCoE),
ESX/ESXi, VMFS raw
device mapping (RDM). iSCSI/Fibre Channel SAN configuration guide.

NetApp .
,
,
. ,
disaster recovery
.
NetApp :

aggregate: .
FlexVol: .
LUN VMFS: .
LUN RDM: , (volume
label) .

4.4 NetApp
,
NetApp.
NetApp System Manager
NetApp NetApp Systems Manager (NSM)
, NetApp
. OS Data ONTAP,
(CLI) - FilerView,
, ,
NetApp Operations Manager.
59

 NSM
. , ,
TR-3880: CLI Configuration Processes for NetApp and VMware vSphere
Storage Arrays Running Data ONTAP and ESX/ESXi Server.
Data ONTAP
NSM NOW Windows.
NetApp.
Ethernet ,
IP- DHCP.
,
setup.
NSM autodiscover
, IP- , .
NSM, NetApp Systems
Manager Quick Start Guide.

47) NetApp Systems Manager.

FC FCoE,
. Rapid
Cloning Utility (RCU) LUN,
, .

60

 48) NetApp Systems Manager.

IP-, NFS,
iSCSI, Ethernet
EtherChannel. , EtherChannel,
. ,
, 3.
NetApp System Manager
NetApp Systems Manager ,
NetApp FAS,
.
, thin provisioning
, aggregate.
,
, aggregate
.
NetApp e-mail
, SNMP.
NSM.

61

 49) NetApp Systems Manager.

NetApp Operations Manager
NetApp Operations Manager , , NetApp FAS
, . NetApp thin provisioning,
NetApp Operations Manager e-mail
. thin-provisioned,
aggregates.
,
, aggregate .
DataFabric Manager Server: Operations Manager
Administration Guide.
TCP/IP, VSC
Virtual Storage Console (VSC) vSphere client,
vCenter Server, VSC server, NetApp. ,
. VSC
6 , vCenter Server, VSC.
6) TCP/IP, VSC.

80
443

VMware vCenter Server NetApp

HTTP.
VMware vCenter Server NetApp
secure HTTP (SSL).
62

8043

8143

Backup and recovery https-.

backup and recovery, CLI
.
VSC https-.

4.5 VSC 2.0

root NetApp,
VSC, . ,
Data ONTAP service
account VSC. ,
VSC 2.0,
role-based access controls (RBAC) .
, RBAC VSC 2.0.

RBAC (Role-Based Access Control)
, , ,
.
Data ONTAP ( SSH, telnet).
: copy-paste
,
, .
.

NetApp CLI.
1: , , VSC Core Features
1. .
2. .
useradmin role add "role_name" -a login-http-admin,api-aggr-list-info,api-cf-getpartner,api-cf-status,api-disk-list-info,api-ems-autosupport-log,api-fcp-adapterlist-info,api-fcp-get-cfmode,api-license-list-info,api-lun-get-vdisk-attributes,apilun-list-info,api-lun-map-list-info,api-nfs-exportfs-list-rules,api-qtree-list,apisnmp-get,api-snmp-get-next,api-system-get-info,api-system-get-version,api-volumeautosize-get,api-volume-list-info,api-volume-options-list-info

63

 50) VSC core feature service account.

2:
1. , .
2. .
useradmin group add "group_name" -r "role_name"

51) VSC core feature service account.

3: , VSC
1. , ,
.
2. .
useradmin user add "user_name" -g "group_name"

: .

64

 52) VSC core feature service account.

4:
1. , .
2. .
useradmin user list "user_name"

,
.

53) VSC core feature service account.

service account VSC.

4.6 Provisioning and Cloning VSC 2.0

, .
Rapid Cloning Utility .

65

 ,
vCenter.
vCenter.

Provisioning and Cloning VSC 2.0

, 54.
. RBAC
, VSC.
Destroy Datastores:
Modify Datastores:
Create Datastores: SAN NAS
Create Clones: VM
54) VSC.
,
Provisioning and Cloning.
,
vCenter. root
, VMware
.
, VSC, 5.3.
:
1:
.
, VSC Provisioning and Cloning.

VSC.
.
4.5.
useradmin role add create_clones -a login-http-admin,api-system-get-version,apisystem-get-info,api-system-cli,api-license-list-info,cli-ifconfig,api-aggr-listinfo,api-volume-list-info,api-lun-list-info,api-lun-map-list-info,api-igroup-listinfo,api-ems-autosupport-log,api-file-get-file-info,api-clone-*,api-file-createdirectory,api-file-read-file,api-file-delete-file,api-file-write-file,cli-mv,apifile-delete-directory,cli-ndmpd,cli-ndmpcopy,api-useradmin-user-list,api-cfstatus,api-snapshot-list-info,api-volume-autosize-get,api-iscsi-session-listinfo,api-iscsi-portal-list-info,api-fcp-service-status,api-iscsi-service-status,clidf,api-snapmirror-get-volume-status,api-quota-report,api-qtree-list,api-system-apilist,api-vfiler-list-info

66

 2:
,
(provisioning) (cloning),
VSC .

4.5.
useradmin role add create_datastores -a api-volume-create,api-volume-set-option,apivolume-autosize-set,api-sis-enable,api-sis-start,api-snapshot-create,api-snapshotset-reserve,api-volume-clone-create,api-nfs-exportfs-list-rules-2,api-nfs-exportfsmodify-rule-2,api-nfs-exportfs-load-exports,api-igroup-create,api-lun-create-bysize,api-lun-map,api-lun-set-comment,api-igroup-add,cli-qtree,cli-iscsi

useradmin role add modify_datastores -a api-volume-create,api-volume-set-option,apivolume-autosize-set,api-sis-enable,api-sis-start,api-snapshot-create,api-snapshotset-reserve,api-volume-clone-create,api-nfs-exportfs-list-rules-2,api-nfs-exportfsmodify-rule-2,api-nfs-exportfs-load-exports,api-igroup-create,api-lun-create-bysize,api-lun-map,api-lun-set-comment,api-igroup-add,cli-qtree,cli-iscsi ,api-volumesize,api-sis-disable,api-sis-stop,api-lun-resize

useradmin role add destroy_datastores -a api-volume-create,api-volume-set-option,apivolume-autosize-set,api-sis-enable,api-sis-start,api-snapshot-create,api-snapshotset-reserve,api-volume-clone-create,api-nfs-exportfs-list-rules-2,api-nfs-exportfsmodify-rule-2,api-nfs-exportfs-load-exports,api-igroup-create,api-lun-create-bysize,api-lun-map,api-lun-set-comment,api-igroup-add,cli-qtree,cli-iscsi,api-volumesize,api-sis-disable,api-sis-stop,api-lun-resize,api-volume-offline,api-volumedestroy,api-lun-offline,api-lun-destroy

3: Provisioning Cloning
1. , .
2. .
useradmin group add "group_name" -r "role1_name" "role2_name"

create_clones .
4.5.
4:
1. , ,
.
2. .
4.5.
useradmin user add "user_name" -g "group_name"

: .
5:
1. , .
67

2. .
useradmin user list "user_name"

,
.
,
Provisioning and Cloning VSC 2.0.

5 vSphere

Virtual Infrastructure

5.1 vCenter
(. . 45), vSphere, NetApp
,

, NetApp.
Provisioning and Cloning, Virtual
Storage Console 2.0 vSphere plug-in.
NetApp Virtual Storage Console 2.0 vCenter plug-in
Virtual Storage Console (VSC) 2.0 , NetApp.
, VSC NetApp vSphere.
, VSC plug-in, :

Virtual Storage Console: VSC 2.0.

NetApp.
Provisioning and Cloning: ,
NetApp Rapid Cloning Utility (RCU). ,
, ,
, .
, Provisioning and Cloning, NetApp
FlexClone , .
Backup and Recovery: NetApp SnapManager
for Virtual Infrastructure, VSC 2.0 plug-in framework.
VMware , SMVI, SnapRestore. SnapMirror
,
FlexClone , NFS.

68

 55) Virtual Storage Console 2.0.

VSC
ESX/ESXi. VSC.

-
FC, FCoE, iSCSI, NFS ESX/ESXi

FC, FCoE, iSCSI
VAAI
, ESX/ESXi

,

, , ESX/ESXi,
-.

Provisioning and Cloning VSC

SAN NAS NetApp. Provisioning and Cloning
VSC 2.0:

Provisioning and Cloning.
FC, FCoE, iSCSI, NFS.
multipathing ,
VMware pluggable storage architecture LUN-,
ALUA, NFS round robin.
,
. LUN masking NFS exports.
FC, FCoE, iSCSI, NFS .

, , ESX/ESXi
, -.
69

5.2 Virtual Storage Console 2.0

VSC ESX/ESXi 4.0 ,
ESX/ESXi 3.5. VSC,
, .

vCenter Server version 4.0 . VSC vCenter Server

.
VM, 32 64-bit Windows Server 2008, 2003 SP1 , 32-bit XP
Professional SP2 .
Data ONTAP 7.3.1.1 .

firmware, NetApp Interoperability Matrix.

VSC 2.0
1. Windows
2.
3. .
VSC 2.0,
(. . 56). , ,
Virtual Storage Console ( ).
Provisioning and Cloning Backup and Recovery Rapid Cloning Utility
SnapManager for Virtual Infrastructure,
, . , NetApp
Provisioning and Cloning, , ,
.
4. VSC vCenter Server , ,
.
vCenter
.
VSC
plug-in vCenter Server. vCenter administrator
. ,
VSC 2.0, .

70

 56) VSC 2.0.

57) vCenter.

71

 58) VSC vCenter Server.

5.3 Virtual Storage Console

VSC ,
.
1. vSphere client, vCenter.
2. , NetApp.
3. Virtual Storage Console .
, VSC
, Data ONTAP, ESX/ESXi
. ,
ESX/ESXi vSphere client NetApp
. VSC ,
.
Controller Credentials,
, VSC .
root, VSC,
.

72

 59) VSC.

5.4 ESX/ESXi
VSC
ESX/ESXi 4.x, NetApp. VMware
ESX/ESXi,
.
HBA CNA, ,
- (NFS iSCSI).

73

 60) ESX/ESXi 4.x,

multipathing LUN.

5.5 Provisioning and Cloning

Provisioning and Cloning VSC 2.0 -
, , .
vSphere client:
1.
2.
3.
4.

vCenter
NetApp
Provisioning and Cloning .
Add .

74

 61) Provisioning and Cloning.

5. Storage Controllers
6. Add.
,
vCenter.
7. ,
. root
Provisioning and Cloning.

75

 62) RCU.

5.6 Provisioning and Cloning

, RCU ,
, .
, FlexVol ( SAN), aggregates (
NAS). :
1.
2.
3.
4.

Provisioning and Cloning VSC

Resources
.

prevent further changes (. 63),

,
. ,
,
VSC.

76

 63) VSC
.

5.7 vCenter
, Provisioning and Cloning VSC,
VMware vCenter.
1.
2.
3.
4.

, ESX/ESXi
NetApp;
Provisioning and Cloning,
Provision datastore.

77

 64) .
NetApp Datastore Provisioning wizard, :

;
(VMFS NFS);
, , , block size ( VMFS);
LUN thin provisioned.

.
, iSCSI, FC, FCoE, VSC , initiator groups,
ALUA, LUN masking, path selection policies, LUN
VMFS. , NFS, VSC
exports, .
, , thinprovisioned LUN-,
.
65 Fibre Channel TR3749 LUN
thin-provisioned, FlexVol.

78

 65) VMFS NetApp Datastore Provisioning wizard.

NFS VMFS.
NFS
autogrow. 66
thin provisioned autogrow.

79

 66) NFS NetApp Datastore Provisioning wizard.

5.8
, ,
.
.
, , ,
WAN, .

, VMware ,
, VM. VM
80

home directory. , VM home directory.

VM, , VMkernel
swapfile, VMware, NVRAM VMware, .
, VM home directory
. . 67 .

67) vswap VMware .

VM NetApp Snapshots
, NetApp
VMware NetApp,
SnapManager, SnapMirror / SnapVault.
NetApp
(-, ) , - ,
.
, - NetApp,
VMware ,
. ,
, , ,
,
, ,
.
: VMkernel swap
ESX VMkernel swap vswap VM.
; vswap ,
VM. ,
VM , Site Recovery Manager,
NetApp VMkernel swap VM
81

home directory, NetApp,

VMkernel swap. 68 .

68) : vswap
.
vswap.
VMware swap file , NetApp
thin-provisioned LUN FlexVol
autogrow. thin-provisioned LUN Autogrow FlexVol
swap-.
swap,
.
, vswap, . NetApp
ESX/ESXi vswap,
VMotion.
69 VMkernel
swap vCenter Server.

82

 69) VMkernel swap.

: VM swap/Pagefile
. ,
.
, , .
.
VM swap, pagefile ( ),
1,5 2 VM .
,
( ),
, business-critical .
swap pagefile OS
, , NetApp volume.
70 .

83

 70) :
VM pagefile pagefile datastore
vswap datastore .
, .
, ,
, -,
.
VMware
vCenter Site Recovery Manager. , -
DR-.
, SRM VM,
pagefile VMDK DR- SRM recovery
plan. SRM,
TR-3671: VMware vCenter Site Recovery Manager in a NetApp Environment.

5.9 vCenter
VSC 2.0, VMware
, . VSC
VMFS NFS,
NFS. ,
datastore vCenter, NetApp, Provisioning and Cloning,
Resize.

84

 71) .

72) VMFS.

5.10 vCenter
VSC VMware
, , , .
85

 SAN NAS ,
, .
NFS
NetApp vCenter, Storage Details - NAS Storage Details - SAN,
. ,
.
NFS.

73) Storage Details VSC.

Capacity
Capacity Storage Details , FlexVol,
aggregate . NFS, FlexVol
, .
aggregate.
VMFS, LUN ,
.
FlexVol aggregate. FlexVol LUN, LUN thin
provisioned, FlexVol autogrow ,
LUN. LUN
86

 FlexVol, , LUN offline,

, , .
85% aggregate.
aggregate
VM aggregate .
NetApp NetApp Data Motion,
, .
Volume
Volume FlexVol.
Provisioning and Cloning. VSC
NFS , VSC,
, NetApp.
Deduplication
, ,
Deduplication. NFS,
,
VM VM.
, NetApp,
Deduplication management.

74) NetApp .

87

 75) .

6

Virtual Infrastructure

6.1 VM OS Windows
Windows
-,
,
NTFS.
-,
. :
1. Windows VM.
2. Start > Run CMD.
3. : fsutil behavior set disablelastaccess 1.

, NetApp,
,
88

 WAFL OS.
OS
VM, , the NetApp Global Support Center.

6.2
VM SCSI BUS
VSC, GOS
timeout scripts, ISO,
, SCSI
.
VMware (GOS timeout scripts),
:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

ISO, VSC.
vCenter Server, VM .
edit settings.
CDROM ISO.
ISO, OS .
OK.
VM.
OS VM.
ISO.
VM.

6.3
VM VMFS
.
, ,
OS VM.
- ,
VMFS
.
- ,
, .
NetApp, VMware, ,
VMware
VMFS .
VMFS OS
:

Citrix, Microsoft, NetApp, VMware. TR-3747: Best Practices for File System Alignment in
Virtual Environments
Dell. Designing and Optimizing SAN Configurations
EMC. Celerra IP Storage with VMware Virtual Infrastructure
EMC. CLARiiON Integration with VMware ESX Server
89

IBM. Storage Block Alignment with VMware Virtual Infrastructure

Vizioncore. vOptimizer Pro FAQ
VMware. Recommendations for Aligning VMFS Partitions

NetApp VMFS
LUN- NetApp iSCSI, FC, FCoE. , LUN
, LUN VMware. ,
VMware NFS, .
, VMFS NFS, , OS,
.


NetApp FAS, 4096.
, Microsoft Windows 2000, 2003, XP
32256. 4096 .
, Microsoft Windows Server 2008
Windows 7, Windows Vista
1048576. .
: Windows 2008 Windows Vista
Windows, ,
, ,
.

6.4 VM


.
.
.
, -
.
, .
, -,
,
- . -
, .

90

6.5
OS Windows
,
OS Windows, VM msinfo32.
. msinfo32, Start > All Programs > Accessories > System Tools >
System Information ( . 76).

76) msinfo32
.
NetApp MBRtools:
NetApp MBRScan, ESX
Windows Linux,
VMFS NFS.
MBRScan ,
. VM (
), .
MBRScan
VM, msinfo32,
VM. MBRScan VSC.

91

6.6 VM
VM
,
,
(template).
.
NetApp MBRtools
VSC, NetApp , MBRAlign,
ESX MBR-
primary secondary. MBRAlign ,
(powered off) .
MBRAlign . ,
, thin thick
vmdk. MBRAlign NetApp. VM
, , ,
.
MBRAlign , VSC. NetApp
NetApp Global Support Center
.
: Linux, GRUB boot loader,
MBRAlign.
1.
2.
3.
4.

Linux VM CD Linux CDROM ISO.

VM.
boot from the CD.
CD GRUB .

,
VMDK. NetApp
, ,
VM.
,
, .
:

VMware vCenter Converter 5

Quest ( Vizioncore) vOptimizer Pro
Novell PlateSpin Migrate
Paragon Alignment Tool 3.0

92

6.7 VM
VMDK VM DISKPART

, OS,
. OS Windows,
Windows Preinstall Environment boot CD, "live CD/DVD".
. 77.
1.
2.
3.
4.
5.
6.

VM Microsoft WinPE CD.

Start > Run DISKPART.
Select disk0.
Create Partition Primary Align=32 .
.
OS .

77) diskpart .
VMDK fdisk
ESX.

6.8 VM
(VMDK)
ESX/ESXi 4 , VM .

; VM.
, C:\ Windows '/' Linux
, .
OS .
OS .
:
1. vCenter Server.
2. VM.
3. VM Properties.
93

4. ( . 78).
5. OK.
6. VM.

78) .
VMware ESX and ESXi Server
Configuration Guide.

94

 79) VMDK VM.

OS (NTFS EXT3)
RDM ,
. ,
OS, .
1. VM.
2. .
Windows VM, diskpart
. : http://support.microsoft.com/default.aspx?scid=kb;en-us;300415.

Linux VM, ext2resize

. : http://sourceforge.net/projects/ext2resize
VM Microsoft Windows, NetApp SnapDrive,
VM RDM.
OS
OS, OS,
.
Windows 2008. OS
, C:\ Windows VM '/' Linux VM
OS .
,
- . VMDK LUN, ,
OS, ,
6.8 . ,

. VM ,
, , VM.
.

7

Virtual Infrastructure

7.1 ,
VMware vSphere
. ,
, ,
,
. NetApp
1992 , NetApp,
95

 VMware, , ,
, .
VMware Snapshot (point-in-time) VM,
. VMware Snapshot
, ,
vCenter Server. VMware Snapshot
ESX vSphere.
VMware Snapshot, ,
vSphere Basic System Administration Guide.
NetApp Snapshot VMware,
crash-consistent
VM, VM,
. ,
.
VMware ,
,

(hardware-based)
Snapshot
, software-based . ,
vCenter Server,
. vSphere Basic System
Administration Guide VMware ESX and ESXi Server Configuration Guide.

7.2 NetApp
,
,
VMware . NetApp Backup and Recovery
Virtual Storage Console 2.0.
, SnapManager for Virtual Infrastructure (SMVI).
NetApp SnapManager,
,
VM,
, ,
, SMVI ,
.

96

 80) Backup and

Recovery VSC.

81) Backup and Recovery VSC.

97

 vSphere
NetApp Snapshot, TR-3737: SnapManager 2.0 for Virtual Infrastructure Best
Practices.

98

8 vSphere 5
vSphere 5.0, VMware ,
vSphere.
. vSphere,
.

8.1 VMware ESXi

vSphere 5 ,
ESX, Linux,
ESXi. :

, Busybox.
SSH .
.
.
VMkernel.
stateless server,
, RAID, SAN.

, VMware vSphere Hypervisor (ESXi), ,

, ESXi, vSphere,
, vCenter,
,
.
NetApp, , NetApp Virtual Storage Console,
, vCenter Server.
VMware ESXi
, -
.
vSphere installation bundle (VIB),
VMware ESXi
. NetApp VIB,
.

8.2 VMware vCenter Server

VMware vCenter Server ,
vSphere. vSphere 5 , vCenter Server
:

Windows, Microsoft Windows

Server 2003 2008, VM
vCenter Server Appliance (vCSA), virtual appliance, Linux,
, VM

NetApp , Virtual Storage Console

, , vCenter Windows
99

 vCSA. , vCSA Linux, NetApp, Virtual

Storage Console, VM ,
Windows.
, VSC, VM, ,
vCenter .

8.3
vSphere NetApp ,
(. 7).
7) vSphere NetApp.

ESXi shell

CLI

ESXi

vSphere CLI (vCLI)

CLI

ESXi
vCenter

vSphere Management
Assistant (vMA)
vSphere SDK

CLI

ESXi
vCenter
vCenter

PowerCLI

OO
CLI

vCenter

vSphere Client

GUI

ESXi
vCenter

UI

vSphere Web Client

GUI

vCenter

FilerView

Web
GUI

NetApp

NetApp System
Manager

GUI

DataFabric Manager
/Operations Manager

GUI

Virtual Storage
Console (2.1.1
)

Plug
-in

Data ONTAP
PowerShell Toolkit

OO
CLI

NetApp
Manageability SDK

SDK

NetApp

NetApp
ESXi /
vCenter

NetApp

NetApp

NetApp

SDK

VSC

SSH
. .
ESXi
Windows Linux,
ESXi
vCenter
vSphere CLI Linux
VM appliance.
Perl

-
(snap-in) Windows PowerShell.
NetApp
PowerShell cmdlets.
Windows.
GUI
, , VM,
.
, vSphere Site
Recovery Manager.
VM.
, ,
.
NetApp
.
.
Windows

-
(snap-in) Windows PowerShell

Manage ONTAP. s C
C++, Java, Perl, C#, VB.NET, PowerShell.

100

 , -
, NetApp ,
, vSphere client NetApp Virtual Storage Console,
vCenter Server.
, VSC 2.1.1 VSC vSphere 5.

8.4 Host Profiles

Host profiles ,
ESX/ESXi, vSphere 4. vSphere 5
. Host profiles ,
VSC, . ,
, VSC ,
ESXi host profile,
NetApp, VSC Monitoring and Host Configuration,
, .

9 vSphere 5
9.1
vSphere 5,
Data ONTAP.
8)

LUN

LUN/FS
Virtual RDM
Physical RDM
VMFS3
LUN VMFS3
VMFS52
LUN VMFS52
VMDK
queue depth
LUN/FS

FC/FCoE
VMFS RDM
256

iSCSI
VMFS RDM
256

NFS
NetApp WAFL
256

64TB

64TB

100TB1

(. )
2TB 512b
64TB
64TB
2TB
64TB
64TB

(. )
2TB 512b
64TB
64TB
2TB
64TB
64TB

100TB1
RDM NFS
RDM NFS
N/A
N/A
N/A
N/A

2TB 512b
64

2TB 512b
64

2TB 512b
N/A

4Gb 8Gb FC
10Gb Ethernet

1Gb
10Gb Ethernet

1Gb
10Gb Ethernet

- 100TB Data ONTAP 8.0 64-bit aggregates.

, VMFS3 VMFS5,
VMFS3.
2

101

 9) , VMware.

vMotion
Storage vMotion
VMware HA
DRS
Storage I/O Control
Datastore Cluster
Storage DRS
VCB
MSCS VM
Fault Tolerance

Site Recovery Manager

Virtual disk formats
Thin provisioned
Thick provisioned
Eager-zeroed Thick
VMware native
multipathing
Boot form SAN4

FC/FCoE

,
RDM shared LUN
, c eager-zeroed thick
VMDK virtual mode
RDM1

iSCSI

GuestOS
NetApp
, c eager-zeroed thick
VMDK virtual mode
RDM1

NFS

2
3

N/A

HBA

, c eager-zeroed thick
VMDK

- NetApp SnapDrive for Windows virtual-mode RDM.

- VM NetApp NFS.
3
- VAAI NFS.
4
- Autodeploy vSphere 5/ESXi 5.
TFTP HTTP, FC, FCoE, iSCSI,r NFS .
2

10) NetApp.

Thin provisioning
Resize datastore

SANscreen VM Insight
SnapDrive ( GuestOS)
ESX Host Utilities/VSC2.0
VSC2.0 Backup&Recovery
VSC2.0 Provisioning&Clo
Space Reclamation
VMDK/GuestOS
VMFS
1

FC/FCoE

iSCSI

NFS

RDM

RDM

1
N/A

SnapDrive 6.3 for Windows.

102

 11) .

Snapshot backup

backup SRM
SnapMirror
SnapVault
VMDK ()
VMDK ()
NDMP
SnapCreator
SnapProtect

FC/FCoE
Yes
Yes

iSCSI
Yes
Yes

NFS
Yes
Yes

RDM
RDM
VCB
VCB, Windows

RDM
RDM
VCB
VCB, Windows

RDM
RDM
VCB, VIC File Explorer
VCB, third-party apps
VM

9.2 VMFS 5
vSphere 5 vSphere Virtual Machine File
System (VMFS) 5. VMFS5 VMFS
, VAAI. 8, VMFS5
LUN 64TB, NetApp
16TB. , vSphere 5 physicalmode raw device map (RDM) LUN-, VM, 64TB.
Virtual-mode RDM - 2TB virtual-mode RDM
VMware, delta VMDK,
, VMDK VMFS5, 2TB.
, LUN- VMFS5
GUID partition tables (GPT) 16- SCSI. GPT
master boot record (MBR) , 2TB
. MBR 32- logical block address (LBA),
LBA 10- SCSI. 232 x 512
, 2TB . 64-bit
LBA GPT SCSI-16, VMFS5,
, , 32-bit LBA.
, - ,
vSphere ESXi 5. LUN- (256) (1024),
LUN ID (255), , .
VMFS5 ESX/ESXi vSphere 5; ESXi 5 .
, hardware-assisted locking (
atomic test and set, ATS) ,
SCSI ATS (,
NetApp).
, SCSI reservations,
VMFS ESX ESX 4.1, , VAAI.
, ATS-only VMFS,
VMFS5. , VMFS3, VMFS5, -
SCSI reservations.
103

 VMFS5 , all paths

down (APD). APD , , ESX LUN
.
.
LUN , ESX -,
VM, , , ESX purple
screen error .
, , LUN,
. LUN
NetApp, , Exchange SQL Server,
, . ,
, SnapDrive for Windows LUN.
LUN,
, .
LUN APD ESX 4.1, ESXi 5 VMFS5,
APD .

9.3 NFS vSphere 5

NFS Heap Memory
vSphere 5 NFS , ESXi 256
ESXi.
Net.TcpipHeapMax ( , TCP/IP, NFS, iSCSI,
, vMotion, ) NFS.
NetApp Net.TcpipHeapMax 128MB,
, ESXi 5 TCP/IP heap.

vSphere 5 NFS : SIOC, VAAI,
datastore clusters SDRS, .
IP- NFS
,
NFS IP- , ESX,
. ,
NetApp ( UUID).
, IP ,
.
,
. ,
ESX , ,
, ,
IP- NetApp :

/etc/hosts
IP-.
104

DNS IP- , ,
.
Round robin DNS IP
. , round robin DNS,
IP- ,
.

ESX . vCenter
IP. 5.0, vCenter IP
, . vCenter
, . vMotion DRS
, VM . 5.0, vCenter
, ,
IP, vMotion DRS VM,
IP- .
- , , ESX,
, ESX NetApp,
.
, ESX ,
VM . , NFS
, VM:

, FQDN, IP,
IP- IP, IP-
, FQDN
, FQDN

9.4 Storage Distributed Resource Scheduler (Storage DRS)

Storage DRS , VM
, - (I/O latency)
, VM, ,
(POD).
Storage DRS
(VMDK) .
Datastore Cluster (PODs)
(datastore cluster) ,
, .
(VMDK),
.

105

 82) .
,
,
. Provisioning and Cloning Virtual Storage
Console. ,
VSC provisioning wizard Datastore details (. . 83).

106

 83) VSC.
Storage DRS
Storage DRS
:

SDRS manual ,
.
(SAS, SATA, )
.
SDRS VMDK , ,
NetApp, , , ,
VMDK .
.
, SDRS VMDK,
.
SDRS , thin provisioned VMFS
, .

SDRS VMFS NFS.

107

.
ESXi 5.

Storage DRS
, vSphere
, . ,
,

-.

84) Storage DRS.

, ,
(POD)
-. POD.
Storage DRS vCenter Server
, , .
- 8 ,
(latency) 15ms. I/O latency , storage
DRS ,
.

108

 (Affiity Rules) Maintenance Mode

(affinity rules), Storage DRS,
, (POD). ,
. Storage DRS
, affinity rules:

VMDK affinity. .
VMDK anti-affinity.
, .
VM anti-affinity.
.

85) Affinity ( VMware).

, Storage DRS datastore maintenance mode,

.

86) maintenance mode.

SDRS Data ONTAP
12 VMware storage DRS Data
ONTAP.

109

 12) VMware storage DRS Data ONTAP.

NetApp
Snapshot

SDRS Initial
Placement

Deduplication

Thin Provisioning

SnapMirror

Metrocluster

SDRS



.

VASA .
storage DRS VMFS thin-pro.
, storage
vMotion
(break RPO)
.

.
NetApp Snapshot SDRS
NetApp , ,
. ,
- , .
SDRS VM , VM
. SDRS
, , ,
VM, ( ).
VM. VM VM,
, , ,
. - VM;
, , VM
. , ,
VM.
NetApp SDRS
VM, SDRS, ,
VM ,
SDRS ,
, VM, ,
VM, .
VM SDRS,
.
, ,
,
, VM , 100% .

110

NetApp Datastore Thin Provisioning SDRS

NFS, ,
, thin provisioned,
aggregate, thin provisioned.
thin-provisioned LUN , .
( VMFS) LUN,
. , LUN
, LUN,
. SDRS aggregate,
thin-provisioned VMFS, .

9.5 vSphere Storage I/O Control

vSphere storage I/O control (SIOC) vSphere 4.1
- , VMware ESX,
, iSCSI FC. vSphere 5 NFS.
SIOC , () .
- ,
, , disk shares.
SIOC,
resource shares (, ) VM .

87) SIOC NFS vSphere 5.0.

VM Resources, VM Edit Settings.
VM IOPS.

111

 88) SIOC VM vSphere 5.0.

:
Storage DRS Storage I/O Control
.
SIOC , SDRS.
Storage DRS I/O bottlenecks.
SIOC I/O bottlenecks, .
SIOC , 100%

10 vSphere 5
10.1 vSphere Standard vSwitches
vSwitch vSphere 5.
, ,
, GUI.
MTU/Jumbo Frames
vSphere client maximum transmission unit (MTU)
vSwitches VMkernel, jumbo frames. ESX,
,
vSwitch.

112

 89) MTU vSwitch.

90) VMkernel MTU.

Flow Control

, . NetApp,
flow control ESX
send, , 10 Gigabit Ethernet, flow
control ESXi, NetApp FAS, .
113

 ESXi 5, flow control vSphere client GUI. ethtool

flow control . flow
control: autoneg, tx, rx. tx send . ,
NIC, , , Intel, autoneg
, tx rx.
~ # esxcfg-nics -l
Name
PCI
vmnic0 0000:01:00.00
vmnic1 0000:01:00.01
vmnic2 0000:06:00.00
vmnic3 0000:06:00.01
vmnic4 0000:07:00.00
vmnic5 0000:07:00.01

Driver
igb
igb
e1000e
e1000e
e1000e
e1000e

Description
Intel Corporation
Intel Corporation
Intel Corporation
Intel Corporation
Intel Corporation
Intel Corporation

82575EB
82575EB
82571EB
82571EB
82571EB
82571EB

Gigabit
Gigabit
Gigabit
Gigabit
Gigabit
Gigabit

Network Connection
Network Connection
Ethernet Controller
Ethernet Controller
Ethernet Controller
Ethernet Controller

~ # ethtool -a vmnic2
Pause parameters for vmnic2:
Autonegotiate: on
RX:
on
TX:
on
~ # ethtool -A vmnic2 autoneg off rx off tx off
~ # ethtool -a vmnic2
Pause parameters for vmnic2:
Autonegotiate: off
RX:
off
TX:
off

, ,
/etc/rc.local.

10.2 Multiple TCP Sessions iSCSI

vSphere 5 multiple TCP sessions iSCSI.
round robin VMware native multipathing
VMkernel , iSCSI.
, VMkernel, iSCSI, vmnic
--. vmnic
VMkernel, vmnic
VMkernel. VMkernel
vSwitch, vmnic. , IP- VMkernel
IP-.
vSwitch, NIC teaming ,
vmnic standby vmnics VMkernel. vmnics
VMkernel.
: iSCSI VMkernel , , ,
iSCSI VMkernel
NIC teaming .
iSCSI ESXi
native multipathing. iSCSI
, FC. multiple TCP session iSCSI
ESXi, NFS,
, ,
114

, NFS, iSCSI
VMkernel, . NetApp
, iSCSI NFS,
TCP, NIC teaming ,
, multiple TCP session iSCSI.
, iSCSI VMkernel multiple
TCP sessions iSCSI, :
1.
2.
3.
4.
5.

vCenter Server.
ESXi.
, Configuration.
Hardware, Networking.
, Add Networking , Add
Network.
6. VMkernel Next.
: VMkernel Ethernet,
iSCSI. , VMkernel
IP-.
7. VMkernel, . default
gateway VMkernel IP storage network.
8. VMkernel ,
iSCSI VMkernel. , VMkernel
standby adapters (. . 91 92).
vSwitch,
vSwitch VMkernel, iSCSI.
vmnic, ,
93.
9. VMkernel, , software iSCSI
storage adapter.
Hardware ESXi, Storage Adapters.
10. iSCSI Software Adapter .
iSCSI Initiator Properties.
11. Network Configuration.
12. VMkernel, iSCSI (. .
94).
13. VMkernel, Add.
VMkernel, . ,
, VMkernel --
vmnics, .
14. VMkernel OK.
15. Close , .
115

16. , vSphere
iSCSI. ,
Network Configuration, , VMkernel
(. . 95).

91) iSCSI VMkernel 2: vmnic1.

92) iSCSI VMkernel 3: vmnic0.

116

 93) VMkernel vmnic

standby vmnics.

117

 94) VMkernel iSCSI software adapter.

118

 95) VMkernel .

119

11
VMware vSphere
.
, , ,
,
.
,
. -
.
NetApp, NetApp
VMware.
.
( ),
xdl-vgibutmevmtr@netapp.com ; TR-3749 v3.0.
, , , romx@mail.ru
.

120

12

Total Cost Comparison: IT Decision-Maker Perspectives on EMC, HP, and NetApp Storage Solutions in
Enterprise Database Environments
www.netapp.com/library/ar/ar1038.pdf
Wikipedia RAID Definitions and Explanations
en.wikipedia.org/wiki/Redundant_array_of_independent_disks
Microsoft: A Description of the Diskpart Command-Line Utility
www.support.microsoft.com/default.aspx?scid=kb;en-us;300415
GNU ext2resize
www.sourceforge.net/projects/ext2resize
IBM: Storage Block Alignment with VMware Virtual Infrastructure and IBM System Storage N series
ftp://service.boulder.ibm.com/storage/isv/NS3593-0.pdf
Using EMC Celerra IP Storage with VMware Infrastructure 3 over iSCSI and NFS
www.vmware.com/files/pdf/VMware_VI3_and_EMC_Celerra_IP.pdf
Dell/EMC SAN Configurations Part 2
www.dell.com/downloads/global/power/ps4q04-20040149-Mehis.pdf
EMC CLARiiON Integration with VMware ESX Server
www.vmware.com/pdf/clariion_wp_eng.pdf
Vizioncore: Quest vOptimizer Pro FAQ
www.vizioncore.com/products/vOptimizerPro/documents/vOptimizerProFAQ.pdf
Cisco Nexus 1000V Series Switches Deployment Guide Version 2
www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/guide_c07-556626.html

NetApp
NetApp VM Insight with SANscreen
www.netapp.com/us/products/management-software/sanscreen/sanscreen-vm-insight.html
TR-3348: Block Management with Data ONTAP 7G: FlexVol, FlexClone, and Space Guarantees
www.netapp.com/library/tr/3348.pdf
TR-3737: SnapManager 2.0 for Virtual Infrastructure Best Practices
media.netapp.com/documents/tr-3737.pdf
TR-3298: RAID-DP: NetApp Implementation of RAID Double Parity for Data Protection
media.netapp.com/documents/tr-3298.pdf
TR-3671: VMware vCenter Site Recovery Manager in a NetApp Environment
media.netapp.com/documents/tr-3671.pdf
Data ONTAP File Access and Protocol Management Guide
www.now.netapp.com/NOW/knowledge/docs/ontap/rel7311/html/ontap/filesag/accessing/task/t_oc_
accs_file_sharing_between_NFS_and_CIFS.html
DataFabric Manager Server 3.7: Operations Manager Administration Guide
www.now.netapp.com/NOW/knowledge/docs/DFM_win/rel371/html/software/opsmgr/index.htm
NetApp System Manager Quick Start Guide
www.now.netapp.com/NOW/knowledge/docs/netapp_sys_mgr/rel10/html/software/qkstart/index.ht
m
Setting RBAC Access with the VSC
https://kb.netapp.com/support/index?page=content&id=1012529
TR-3747: Best Practices for File System Alignment in Virtual Environments
www.media.netapp.com/documents/tr-3747.pdf
The Virtual Storage Guy Blog
www.communities.netapp.com/community/netapp-blogs/virtualstorageguy
TR-3880: CLI Configuration Processes for NetApp and VMware vSphere Storage Arrays Running Data
ONTAP and ESX/ESXi Server
www.media.netapp.com/documents/tr-3880.pdf
121

TR-3886: Understanding and Using vStorage APIs for Array Integration and NetApp Storage
www.media.netapp.com/documents/tr-3886.pdf
TR-3916: VMware vSphere 4.1 Storage Performance: Measuring FCoE, FC, iSCSI, and NFS Protocols
www.media.netapp.com/documents/tr-3916.pdf

VMware
ESXi Configuration Guides
www.vmware.com/pdf/vsphere4/r41/vsp_41_esxi_server_config.pdf
www.vmware.com/pdf/vsphere4/r40/vsp_40_esxi_server_config.pdf
vSphere Datacenter Administration Guides
www.vmware.com/pdf/vsphere4/r41/vsp_41_dc_admin_guide.pdf
www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdf
Fibre Channel SAN Configuration Guides
www.vmware.com/pdf/vsphere4/r41/vsp_41_san_cfg.pdf
www.vmware.com/pdf/vsphere4/r40/vsp_40_san_cfg.pdf
iSCSI SAN Configuration Guides
www.vmware.com/pdf/vsphere4/r41/vsp_41_iscsi_san_cfg.pdf
www.vmware.com/pdf/vsphere4/r40/vsp_40_iscsi_san_cfg.pdf
vSphere Upgrade Guide
www.vmware.com/pdf/vsphere4/r40/vsp_40_upgrade_guide.pdf
Performance Study of VMware vStorage Thin Provisioning
www.vmware.com/pdf/vsp_4_thinprov_perf.pdf
Configuration Maximums for VMware vSphere 4.1
www.vmware.com/pdf/vsphere4/r41/vsp_41_config_max.pdf
VMware vSphere 4:
www.vmware.com/support/pubs/vs_pubs.html

122

13

1.0
1.0.1
2.0

2.1

2.2

3.0

2009.
2009. . .
2010.
vCenter .
.
NSM, RCU VSC.
16 8.
2010. . vSphere 4.1
iSCSI multiple TCP
sessions NFS .
.
2011. . .
flow control, vicfg
esxcfg, TCP, VSC
2011. vSphere 5.0.
Flow Control 10G .
.

123