Вы находитесь на странице: 1из 40

Netwell

.
, ,
.
Netwell NetApp.

NETAPP TECHNICAL REPORT

VMware vSphere 5 NetApp MetroCluster


Santhosh Devaraju, Ashish Nainwal, NetApp
February 2013 | TR-4128


1 .................................................................................................................................................. 4
1.1 ........................................................................................................................... 4
1.2 ................................................................................................................................ 4
1.3 .......................................................................................... 4
2 ...................................................................................................................................................... 5
2.1 - ........................................................................................................................... 5
2.2 vSphere .......................................................... 5
vSphere Metro Storage Cluster? .......................................................................................... 6
3 NetApp MetroCluster ............................................................................................................ 9
3.1 MetroCluster ................................................................................................................ 9
3.2 MetroCluster...........................................................................................................................10
Stretched MetroCluster (SMC) .........................................................................................10
Stretched MetroCluster .............................................................................................11
Stretch MetroCluster Dual Twin ........................................................12
Fabric-Attached MetroCluster (FMC) ...............................................................................13
Fabric-Attached MetroCluster ...................................................................................13
4 vSphere ........................................................................................................................14
4.1 vCenter ................................................................................................14
1: vCenter Virtual Machine VMware HA Cluster ..................................15
2: vCenter .....................................................................15
3: vCenter VMware vCenter Heartbeat Solution ......................15
4.2 vSphere HA NetApp MetroCluster .........................................................................17
..................................................................................................................................................17
4.3 VMware DRS NetApp MetroCluster .......................................................................18
4.4 VMware Storage DRS NetApp MetroCluster ..........................................................18
5 .....................................................................19
5.1 NetApp Storage ..............................................................................................19
5.2 VMware vSphere ............................................................................................21
vSphere HA NetApp MetroCluster .............................................................21
VMware DRS Groups vSphere 5 NetApp MetroCluster .............................23
DRS Rules vSphere 5 NetApp MetroCluster ..............................................24
Datastore Cluster ...............................................................................................................26
6 MetroCluster ................................................................................................27
6.1 ..................................................................................27
2

6.2 ESX................................................................................................................................28
6.3 ESX .........................................................................................................................29
6.4 vCenter ....................................................................................................................29
6.5 .................................................................................................30
(Management Network) ............................30
(Storage Network) .........................................31
NetApp Fabric MetroCluster ............32
6.6 ISL .............................................................................33
6.7 .........................................................................................34
6.8 ....................................................................................................................36
6.9 ....................................................................................................................37
7 ( ) ...........................................................................38
8 ...................................................................................................................................................39
..................................................................................................................................................40
...................................................................................................................................40

1
NetApp MetroCluster
VMware vSphere 5.0 .
NetApp MetroCluster,
VMware vSphere.
, ,
.

1.1
:

NetApp ,
,
VMware vSphere 5.0 NetApp FAS.
,
, / (dev/test).

1.2
:

NetApp
VMware , ,
(, ),
.
,
, .
, professional
services NetApp VMware.
(
). ,
NetApp.

1.3
:

VMware :
o VMware vCenter 5.0
o VMware vSphere 5.0
NetApp OS NetApp Data ONTAP
4

2
2.1 -
-
. -
,
.

.
.
,
, ,
.
VMware, ,

IT-.
NetApp MetroCluster ,

,
. NetApp MetroCluster
,
.
,
RPO RTO .
VMware - high availability (HA) fault tolerance (FT)
NetApp MetroCluster - ,
.
,
,
.
2.2,
vSphere.

2.2 vSphere
NetApp Unified Storage Architecture .
NetApp FAS
Data ONTAP SAN (FC/FCoE,
iSCSI) NFS.
NetApp MetroCluster NetApp High Availability (controller failover, CFO)
. , MetroCluster
local SyncMirror, (
, , )
(controller failover on demand, CFOD).
5


. Local SyncMirror
MetroCluster, (plex,
): plex ( )
, plex ( )
.
,
. ,
().
MetroCluster. ,
, , , ( Fabric MetroCluster), ,
.
VMware HA/DRS , -
ESXi 5.0 5.1, vCenter Server 5.0 5.1. vSphere, vMotion,
, vCenter Server
HA/DRS, ESXi .
vSphere Metro Storage Cluster?
vSphere Metro Storage Cluster (vMSC) VMware.
vMSC ,
.
VMware
vMSC. VMware
Storage Compatibility Guide.
vSphere Metro Storage Cluster,
:

vSphere 5.x Support with NetApp MetroCluster


VMware vSphere Metro Storage Cluster - Case Study

, NetApp MetroCluster vSphere


:

Stretched MetroCluster
Fabric MetroCluster

1 Stretched MetroCluster.

. 1) Stretched MetroCluster.

2 Fabric MetroCluster.

. 1) Fabric MetroCluster.

3 NetApp MetroCluster
MetroCluster ,
,
. , , MetroCluster
,
. ,
.
(takeover process) ,
.
, , Disaster Recovery,
, , . MetroCluster
.

, Active-Active.
, (failover)
, .
SyncMirror. .
(failover).
Cluster Remote. ,

.

, 500
, FC-,
.

3.1 MetroCluster
NetApp MetroCluster ,
, 1 2.
1) MetroCluster.
No.
1.
2.

3.


HA- FAS, OS Data ONTAP.
Fibre Channel (FC) ,
NetApp, ( FabricMetroCluster).
( )
, .
MetroCluster,
- , MetroCluster.
, ,
FC
MetroCluster.
Fibre Channel-Virtual interface (FC-VI)
, Stretched MetroCluster, FC-VI
30m.
FAS31xx, 32xx, 62xx.
9

4.
5.
6.
7.
8.
9.

Copper/Fiber .
FAS9xx, 30xx, 60xx.
SFP.
FC initiator (storage adapters).
.
SAS, FibreBridges
6500N, SAS.
Fabric-attached MetroCluster dark fiber
( ) ,
xWDM, .

: NetApp MetroCluster Compatibility Matrix


, /fabric OS, Data ONTAP.
2 NetApp MetroCluster.
2) MetroCluster.
No.
FAS
Brocade Cisco
Brocade extended distance Cisco ENTERPRISE_PKG
1.
cluster
license
(failover)
buffer-to
(

10)
.
buffer credits
Brocade ports-on-demand
Cisco PORT_ACTIVATION_PKG
2.
syncmirror_local
(POD)





3.
cluster_remote
Cisco FM_SERVER_PKG

()

Cisco Fabric
Manager GUI

3.2 MetroCluster
MetroCluster .
Stretched MetroCluster (SMC)
Stretched MetroCluster , HA ( ,
).
500 . Stretched MetroCluster

(failover), .
mirrored active-active, Stretched MetroCluster
. (plex), ,
Data ONTAP , .
mirrored active-active, MetroCluster
, , ,
( / ) .
10

3 Stretched MetroCluster.

. 3) Stretched MetroCluster.
:
. , ,

, , , .
Stretched MetroCluster
4 Stretched MetroCluster
FAS32XX single controller , DS14MK4-FC.

11

. 4) Stretched MetroCluster.
Stretched MetroCluster :

HA- FAS32XX, 1, 2.
DS14MK4 1 2

: Stretched MetroCluster, 6
Stretched MetroCluster Considerations.
Stretch MetroCluster Dual Twin
Stretched MetroCluster dual MetroCluster
,
(single-head chassis) twin MetroCluster,
dual MetroCluster, A , A
, B , B
. 5 6 dual twin Stretch MetroCluster .

. 5) dual MetroCluster.
12

. 6) twin MetroCluster.
Fabric-Attached MetroCluster (FMC)
, Fabric-attached MetroCluster, ,
500 ( 160 ),

Brocade Cisco, dual-fabric
. Fabric-attached MetroCluster SyncMirror
,
. ,
aggregate.
7 Fabric-attached MetroCluster.

. 7) Fabric-attached MetroCluster.
:
. , ,

, , , .
Fabric-Attached MetroCluster
8 Fabric-attached MetroCluster
FAS62XX ,
DS14MK4-FC Brocade 5100.

13

. 8) Fabric-attached MetroCluster.
Fabric-attached MetroCluster :

HA- FAS32XX, 1, 2.
DS14MK4 1 2

4 vSphere
VMware vCenter ESX,
,
VM, vMotion, DRS, .
VMware View, vCloud. VMware
.

4.1 vCenter
, ,
.

14

1: vCenter Virtual Machine VMware HA Cluster


vCenter VMware HA cluster.
-, vCenter ESX,
HA. , vCenter
- , VM .
9 .

. 9) VMware HA.
2: vCenter
vCenter Server
, , .
, vCenter ,
NetApp CFOD recovery.
, Microsoft Cluster Service (MSCS), VMware KB.
Oracle RAC, Oracle Databases VMware RAC
Deployment Guide.
3: vCenter VMware vCenter Heartbeat Solution
VMware vCenter Server
,
15

. ,
VMware vCenter Server. VMware vCenter
Server Heartbeat.
VMware vCenter Server Heartbeat
VMware vCenter. , vCenter Server Heartbeat activepassive vCenter Server, .
vCenter Server Heartbeat
vCenter Server, , .
, , , vCenter Server Heartbeat
vCenter.
LAN, WAN.
VMware vCenter Server Heartbeat,
VMware vCenter Server Heartbeat.

VMware vCenter Server ,


VMware License Server
, VMware vMotion VMware DRS
, VMware vCenter
Server,
VMware vCenter Server


VMware vCenter Server
, , inventory,
, VMware vCenter Server,

10 vCenter Server Heartbeat,


. VM vCenter Server ESX
, NetApp
MetroCluster. vCenter Server Heartbeat
LAN.

16

. 10) vCenter Heartbeat solution.

4.2 vSphere HA NetApp MetroCluster

vSphere HA , VM ,
. , ,
, , .
NetApp MetroCluster solution for vSphere, VMware vSphere HA
,
VM ESX.
HA, , , ,
-. , slave- -,
network heartbeat. - heartbeats
. - , slave- ,
VM , . Slave-
VM - .
, heartbeats. -
, .
17

HA vSphere 5.0 ,
;
network heartbeat .
network heartbeat, VMware HA :
datastore heartbeating. Datastore heartbeating - ,
management network. datastore
heartbeating , ,
.
vSphere HA, VMware:
vSphere Availability.

heartbeat, NetApp
Isolation IP address vSphere HA.
.

4.3 VMware DRS NetApp MetroCluster


VMware DRS ,
, , .
VMware DRS CPU , VM
. VMware DRS
NetApp MetroCluster.
VMhost affinity VMware DRS,
A B, , VM ,
,
. VMhost affinity rules

.

4.4 VMware Storage DRS NetApp MetroCluster


VMware Storage DRS
, ,
storage I/O control.
Storage I/O control DRS- Storage DRS. Storage I/O
control (SIOC) -,
,
- , .
Storage DRS Storage vMotion ,
. NetApp MetroCluster ,
. ,
VM A, ,
A. ,
A, B,
, ,
- .
18

NetApp site affinity;


site affinity A site affinity B.

, VM , Storage
vMotion, NetApp , VMware DRS,
, .
affinity , ,
, .

5
5.1 NetApp Storage
/
, failover MetroCluster.
change_fsid OFF ( Data ONTAP version 7.2.4
),
MetroCluster failover UUID- LUN
, VMFS ESX Server
. , VMFS , VM
.

cf.takeover.change_fsid OFF.
Data ONTAP version 7.2.4 .
NetApp FAS, Data ONTAP 7.2.4,
MetroCluster failover LUN- LUN UUID
LUN. LUN- VMFS-3,
ESX Server 3.x Snapshot LUN. , RAW LUN,
RDM (Raw Device Mapping) MetroCluster,
RDM ,
LUN. ESX
VMFS LUN-, . VMware KB 1001783.

. 11) cf.takeover.change_fsid OFF.


FAS :

cluster, cluster_remote, syncmirror_local

iSCSI, FCP / NFS


19

MetroCluster software-based disk ownership


:
o High-Availability and MetroCluster Configuration Guide
o MetroCluster Design and Implementation Guide
FAS , flexible
aggregate , ESX: VMFS (FC
iSCSI) NFS.

Virtual Storage Console for VMware vSphere vCenter Server,


VMware.
NetApp NetApp Virtual Storage Console for vSphere
NetApp VMware vSphere.

12 NetApp
MetroCluster.

. 12) NetApp FAS


MetroCluster.

20

5.2 VMware vSphere


vSphere HA NetApp MetroCluster
HA
ESX.
1.
2.
3.
4.
5.
6.

vCenter, vSphere Client.


Hosts and Clusters.
Edit Settings.
Turn On vSphere HA.
vSphere HA , Host Monitoring .
disable Admission Control,
,
.

7. . Advanced Options , Isolation


address das.isolationaddress IP-.

21

8. VM ,
Medium.
9. Host Isolation response NetApp Leave Powered On.

Host Isolation Leave Powered On,


,
.

: iSCSI/NFS, (management network)


IP storage network, ,
.
Shutdown, ,
, .
split-brain.

22

10. VM Monitoring .
11. Datastore Heartbeating Select any of the cluster datastores taking into
account my preferences.
12. OK.

VMware DRS Groups vSphere 5 NetApp MetroCluster


DRS VM .
.
1. vCenter, vSphere Client.

23

2. vSphere inventory
Edit Settings.
3. DRS Group Manager.
4. virtual machine DRS groups, ,
.
5. VM .
6. DRS- (host DRS groups), ,
.
7. .

DRS Rules vSphere 5 NetApp MetroCluster


DRS A B.
1. vSphere, inventory Edit
Settings.
2. Rules.
3. Add.
4. Rule .
5. Type, Virtual Machines to Hosts.
6. virtual machine DRS group host DRS group
.
24

7. , VM host DRS groups, A, ,


A.
8. Should run on hosts in group.
9. 3 7 B.
10. OK.

NetApp Should run on hosts in group


Must run on hosts in group. A, VM A
B vSphere HA,
HA VMs B, .
, , HA,
, .

: (event), (alarm),
, VM VM-Host affinity rule. vSphere
Client, , VM
is violating VM-Host Affinity Rule .
(alarms),
vSphere Monitoring and Performance.
13 , DRS.

. 13) DRS.
25

Datastore Cluster
.
1. vCenter vSphere Client.
2. Datastores and Datastore Clusters vSphere Client inventory,
datacenter New Datastore Cluster.
3. , , Turn on Storage DRS .

4. SDRS automation level, , No Automation (Manual Mode).

NetApp Storage DRS ,


, .

5. , Enable I/O metric for SDRS recommendations ;


.

6. .
7. Select Datastores , A;
, A.
8. 1 6 B, ,
B.
26

6 MetroCluster
6.1
, , HBA, ,
, FC/Ethernet ,
(dead) ESX.

HBA//, ESX .
,
, .
MetroCluster ,
.

, , NFS/iSCSI, NetApp
, vmkernel vSwitch
, NFS vmkernel distributed vSwitch. NIC teaming

active-active

active-standby.
, iSCSI LUNs, multipathing vmkernel
iSCSI. vSphere.

, Fibre Channel LUNs, NetApp , ,


HBA, HBA/port. NetApp

single
initiator
to
single
target.
Virtual Storage Console (VSC) multipathing.

27

6.2 ESX
14 ESX.

. 14) ESX.
, ESX , (master node) VMware HA
, network
heartbeats. , , ,
, , master node
datastore heartbeats , , , ,
(management network).
, master node , ,
, ,
.
, ESX ,
, . DRS affinity rules
, , VM ,
. , DRS
, NetApp DRS
.
MetroCluster ,
.

28

6.3 ESX
15 ESX .

. 15) ESX.
, (management network) ESX
, master node VMware HA network heartbeats
, .
,
-, datastore heartbeat. ,
master node , , .
, ,
, , , VM
. 30 .
MetroCluster ,
.

6.4 vCenter
, vCenter , ,
, DRS, ,
VM DRS .
,
, VMware HA VM
. , VM
, ,
latency .
vCenter, DRS host-VM
affinity rule.
MetroCluster ,
.
29

6.5
(Management Network)
16 (management
network).

. 16) .
, ,
ESX A ESX .
, ESX network
heartbeats master node VMware HA-. ,
, master node , VM
.
MetroCluster ,
,
.

30

(Storage Network)
17 .

. 17) .
, (ISL) ,
A /LUN- B . VMware
DRS ,
, , , ,
VM ,
.
MetroCluster ,
,
.
- (affinity rule) , A
( LUN-) B,
ISL. ISL , VM A,
B, . , , VMware HA ,
31

- , heartbeats.
,
- VM .
NetApp Fabric MetroCluster
18 NetApp
Fabric MetroCluster.

. 18) NetApp Fabric MetroCluster.


, ISL Fabric
MetroCluster , ,
/LUN-
.
, ,
cf status.

32

6.6 ISL
19 ISL (interswitch links, ) .

Figure 19) ISL .


, ISL
. , , ISL
(management network) (storage network),
, ISL.
, ESX , vSphere HA
datastore heartbeats , , ESX datastore
33

heartbeats /LUN. A , ,
B , ,
datastore heartbeats. vSphere HA A
B, , B
(ISL).
B.
NetApp , - DRS.
, , ,
, vSphere HA VM
. , ISL , ,
, ,
VM MAC-.

6.7
20 .

. 20) .
,
,
34

. ,
. ,
ESX
, ., ISL
.
,
/LUN-. , vSphere HA
, master node - network heartbeats.
,
, . ,

controller failover on demand (CFOD);
, plexes aggregates,
read-only .
, , ,
, ,
, backend- .
, ,
failback, ,
, . ,
failback , failback
.
, - ,
,
.
, , , A
kernel panic, A B.

, ,
OS firmware . ,
.
Tier 1 2. NetApp FAS,
HA-, (
, OS,
),
, . ,
, (failback)
, .
35

, B ,
B,
A.

6.8
21 .

. 21) .
, A,
/LUN-, A (
), B,
, read-write ESX.
36

MetroCluster , ,
, ,
.
, - ,
,
.

6.9
22 .

. 22) .
, A, ESX B
heartbeat ESX A, ,
ESX B heartbeats. heartbeats
, ESX A ,
, A, B.
, controller failure on demand,
A B. , LUN- A
37

B, A
.

controller failover on demand 30


, vSphere HA VM 30
.

B.
A
(controller failure on demand, CFOD)
A.

7 ( )
3) 1: - ESX 1 2.
Tests Performed
Expected Results

Actual Results
MetroCluster Behavior
VMware HA Behavior

1. ESX 1.
2. 2.
1
.
VM, 1, 2.
.
.
- 1
.
VM, ,
.

38

4) 2: .


MetroCluster
VMware HA

1. 0 1.
2. 0 2.
VM -
.
VM DRS- .
.
.
MetroCluster .
.
HA
.

5) 3: .


MetroCluster
VMware HA

1. 1.
2. 0 2.
VM -
.
VM DRS- .
.
.

.
HA .
.

8

, ,
- .
NetApp MetroCluster
VMware vSphere HA, .
-
,
RPO RTO. ,
,
. VMware NetApp,
High Available Fault Tolerance .

.

39


Ashish Nainwal Santhosh Devaraju NetApp
Infrastructure and Cloud Enablement group,
, NetApp.

1.0


2013

40